2024-09-21T06:15:38.515 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T06:15:38.608 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T06:15:38.704 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913544 branch: wip-pdonnell-testing-20240920.212106-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/iogen}} email: pdonnell@redhat.com first_in_suite: false job_id: '7913544' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240920.212106-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 1 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 1843221895adccfb3b692060251dd6a0183bc410 subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: dry_run: true install: ceph: flavor: default sha1: 1843221895adccfb3b692060251dd6a0183bc410 kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240920.212106-debug sha1: 1843221895adccfb3b692060251dd6a0183bc410 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7864 sha1: 1843221895adccfb3b692060251dd6a0183bc410 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240920.212106-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1843221895adccfb3b692060251dd6a0183bc410 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi067.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBxpHEky477d52e2IiSOdnAH6q+FiDq257MwLTJlmsZqbjCkky/fMOj8EecHMigfszdFoCkucVOgJTGW80VUHcc= smithi100.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL7K1IRmLMFcEdTYBQuuh8Q0G5JbpLKQ1L/wtSsHxr+ajGZ7wSVORp9MApC8QfBgN5Pu7yR7iZwbwlxhbWTyJew= smithi203.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF6QMvc3usVLkioEbZMuck6Wxov9r2behZ4cRQ2NMUmP9PlfzUB3ZXy1wMDzg+rPc17iqcEemg54QFkuUxc6qYc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - ceph-fuse: null - check-counter: {} - quiescer: null - workunit: clients: all: - suites/iogen.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_23:48:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T06:15:38.704 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa; will attempt to use it 2024-09-21T06:15:38.705 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks 2024-09-21T06:15:38.705 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T06:15:38.718 INFO:teuthology.task.internal:Checking packages... 2024-09-21T06:15:38.737 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1843221895adccfb3b692060251dd6a0183bc410' 2024-09-21T06:15:38.737 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T06:15:38.737 INFO:teuthology.packaging:ref: None 2024-09-21T06:15:38.737 INFO:teuthology.packaging:tag: None 2024-09-21T06:15:38.737 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240920.212106-debug 2024-09-21T06:15:38.737 INFO:teuthology.packaging:sha1: 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:15:38.738 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240920.212106-debug 2024-09-21T06:15:38.878 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5120.g18432218 2024-09-21T06:15:38.880 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T06:15:38.906 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T06:15:38.906 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T06:15:38.941 INFO:teuthology.task.internal:Saving configuration 2024-09-21T06:15:38.959 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T06:15:38.965 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T06:15:38.988 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi067.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913544', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 06:10:20.645564', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBxpHEky477d52e2IiSOdnAH6q+FiDq257MwLTJlmsZqbjCkky/fMOj8EecHMigfszdFoCkucVOgJTGW80VUHcc='} 2024-09-21T06:15:39.013 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913544', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 06:10:20.643622', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL7K1IRmLMFcEdTYBQuuh8Q0G5JbpLKQ1L/wtSsHxr+ajGZ7wSVORp9MApC8QfBgN5Pu7yR7iZwbwlxhbWTyJew='} 2024-09-21T06:15:39.034 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi203.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913544', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 06:10:20.646572', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF6QMvc3usVLkioEbZMuck6Wxov9r2behZ4cRQ2NMUmP9PlfzUB3ZXy1wMDzg+rPc17iqcEemg54QFkuUxc6qYc='} 2024-09-21T06:15:39.034 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T06:15:39.042 INFO:teuthology.task.internal:roles: ubuntu@smithi067.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-21T06:15:39.042 INFO:teuthology.task.internal:roles: ubuntu@smithi100.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-21T06:15:39.043 INFO:teuthology.task.internal:roles: ubuntu@smithi203.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-21T06:15:39.043 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T06:15:39.138 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f65a694c160>, signals=[15]) 2024-09-21T06:15:39.138 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T06:15:39.149 INFO:teuthology.task.internal:Opening connections... 2024-09-21T06:15:39.150 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi067.front.sepia.ceph.com 2024-09-21T06:15:39.151 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:15:39.225 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi100.front.sepia.ceph.com 2024-09-21T06:15:39.226 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:15:39.296 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi203.front.sepia.ceph.com 2024-09-21T06:15:39.296 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:15:39.369 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T06:15:39.378 DEBUG:teuthology.orchestra.run.smithi067:> uname -m 2024-09-21T06:15:39.396 INFO:teuthology.orchestra.run.smithi067.stdout:x86_64 2024-09-21T06:15:39.396 DEBUG:teuthology.orchestra.run.smithi067:> cat /etc/os-release 2024-09-21T06:15:39.452 INFO:teuthology.orchestra.run.smithi067.stdout:NAME="CentOS Stream" 2024-09-21T06:15:39.452 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION="9" 2024-09-21T06:15:39.452 INFO:teuthology.orchestra.run.smithi067.stdout:ID="centos" 2024-09-21T06:15:39.452 INFO:teuthology.orchestra.run.smithi067.stdout:ID_LIKE="rhel fedora" 2024-09-21T06:15:39.452 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION_ID="9" 2024-09-21T06:15:39.452 INFO:teuthology.orchestra.run.smithi067.stdout:PLATFORM_ID="platform:el9" 2024-09-21T06:15:39.453 INFO:teuthology.orchestra.run.smithi067.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T06:15:39.453 INFO:teuthology.orchestra.run.smithi067.stdout:ANSI_COLOR="0;31" 2024-09-21T06:15:39.453 INFO:teuthology.orchestra.run.smithi067.stdout:LOGO="fedora-logo-icon" 2024-09-21T06:15:39.453 INFO:teuthology.orchestra.run.smithi067.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T06:15:39.453 INFO:teuthology.orchestra.run.smithi067.stdout:HOME_URL="https://centos.org/" 2024-09-21T06:15:39.453 INFO:teuthology.orchestra.run.smithi067.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T06:15:39.453 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T06:15:39.453 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T06:15:39.453 INFO:teuthology.lock.ops:Updating smithi067.front.sepia.ceph.com on lock server 2024-09-21T06:15:39.475 DEBUG:teuthology.orchestra.run.smithi100:> uname -m 2024-09-21T06:15:39.494 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64 2024-09-21T06:15:39.494 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2024-09-21T06:15:39.551 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="CentOS Stream" 2024-09-21T06:15:39.551 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="9" 2024-09-21T06:15:39.551 INFO:teuthology.orchestra.run.smithi100.stdout:ID="centos" 2024-09-21T06:15:39.552 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE="rhel fedora" 2024-09-21T06:15:39.552 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="9" 2024-09-21T06:15:39.552 INFO:teuthology.orchestra.run.smithi100.stdout:PLATFORM_ID="platform:el9" 2024-09-21T06:15:39.552 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T06:15:39.552 INFO:teuthology.orchestra.run.smithi100.stdout:ANSI_COLOR="0;31" 2024-09-21T06:15:39.552 INFO:teuthology.orchestra.run.smithi100.stdout:LOGO="fedora-logo-icon" 2024-09-21T06:15:39.552 INFO:teuthology.orchestra.run.smithi100.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T06:15:39.552 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://centos.org/" 2024-09-21T06:15:39.552 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T06:15:39.552 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T06:15:39.552 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T06:15:39.552 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2024-09-21T06:15:39.575 DEBUG:teuthology.orchestra.run.smithi203:> uname -m 2024-09-21T06:15:39.593 INFO:teuthology.orchestra.run.smithi203.stdout:x86_64 2024-09-21T06:15:39.593 DEBUG:teuthology.orchestra.run.smithi203:> cat /etc/os-release 2024-09-21T06:15:39.652 INFO:teuthology.orchestra.run.smithi203.stdout:NAME="CentOS Stream" 2024-09-21T06:15:39.652 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION="9" 2024-09-21T06:15:39.652 INFO:teuthology.orchestra.run.smithi203.stdout:ID="centos" 2024-09-21T06:15:39.652 INFO:teuthology.orchestra.run.smithi203.stdout:ID_LIKE="rhel fedora" 2024-09-21T06:15:39.652 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION_ID="9" 2024-09-21T06:15:39.652 INFO:teuthology.orchestra.run.smithi203.stdout:PLATFORM_ID="platform:el9" 2024-09-21T06:15:39.652 INFO:teuthology.orchestra.run.smithi203.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T06:15:39.652 INFO:teuthology.orchestra.run.smithi203.stdout:ANSI_COLOR="0;31" 2024-09-21T06:15:39.652 INFO:teuthology.orchestra.run.smithi203.stdout:LOGO="fedora-logo-icon" 2024-09-21T06:15:39.653 INFO:teuthology.orchestra.run.smithi203.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T06:15:39.653 INFO:teuthology.orchestra.run.smithi203.stdout:HOME_URL="https://centos.org/" 2024-09-21T06:15:39.653 INFO:teuthology.orchestra.run.smithi203.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T06:15:39.653 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T06:15:39.653 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T06:15:39.653 INFO:teuthology.lock.ops:Updating smithi203.front.sepia.ceph.com on lock server 2024-09-21T06:15:39.675 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T06:15:39.685 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T06:15:39.698 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T06:15:39.698 DEBUG:teuthology.orchestra.run.smithi067:> test '!' -e /home/ubuntu/cephtest 2024-09-21T06:15:39.701 DEBUG:teuthology.orchestra.run.smithi100:> test '!' -e /home/ubuntu/cephtest 2024-09-21T06:15:39.703 DEBUG:teuthology.orchestra.run.smithi203:> test '!' -e /home/ubuntu/cephtest 2024-09-21T06:15:39.719 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T06:15:39.731 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T06:15:39.731 DEBUG:teuthology.orchestra.run.smithi067:> test -z $(ls -A /var/lib/ceph) 2024-09-21T06:15:39.758 DEBUG:teuthology.orchestra.run.smithi100:> test -z $(ls -A /var/lib/ceph) 2024-09-21T06:15:39.760 DEBUG:teuthology.orchestra.run.smithi203:> test -z $(ls -A /var/lib/ceph) 2024-09-21T06:15:39.875 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T06:15:39.968 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T06:15:39.986 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-21T06:15:39.987 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-21T06:15:39.987 DEBUG:teuthology.orchestra.run.smithi067:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T06:15:39.988 DEBUG:teuthology.orchestra.run.smithi100:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T06:15:39.988 DEBUG:teuthology.orchestra.run.smithi203:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T06:15:40.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:15:40.005 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-09-21T06:15:40.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:15:40.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:15:40.006 DEBUG:teuthology.orchestra.run.smithi203:> uname -r 2024-09-21T06:15:40.007 DEBUG:teuthology.orchestra.run.smithi067:> uname -r 2024-09-21T06:15:40.063 INFO:teuthology.orchestra.run.smithi100.stdout:5.14.0-503.el9.x86_64 2024-09-21T06:15:40.063 INFO:teuthology.task.kernel:Running kernel on smithi100: 5.14.0-503.el9.x86_64 2024-09-21T06:15:40.063 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum install -y kernel 2024-09-21T06:15:40.064 INFO:teuthology.orchestra.run.smithi067.stdout:5.14.0-503.el9.x86_64 2024-09-21T06:15:40.065 INFO:teuthology.task.kernel:Running kernel on smithi067: 5.14.0-503.el9.x86_64 2024-09-21T06:15:40.065 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum install -y kernel 2024-09-21T06:15:40.066 INFO:teuthology.orchestra.run.smithi203.stdout:5.14.0-503.el9.x86_64 2024-09-21T06:15:40.066 INFO:teuthology.task.kernel:Running kernel on smithi203: 5.14.0-503.el9.x86_64 2024-09-21T06:15:40.066 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum install -y kernel 2024-09-21T06:15:43.465 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.3 MB 00:01 2024-09-21T06:15:43.830 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - BaseOS 4.1 MB/s | 8.3 MB 00:02 2024-09-21T06:15:44.480 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - BaseOS 3.2 MB/s | 8.3 MB 00:02 2024-09-21T06:15:45.990 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - AppStream 20 MB/s | 20 MB 00:01 2024-09-21T06:15:47.449 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - AppStream 7.0 MB/s | 20 MB 00:02 2024-09-21T06:15:49.338 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - AppStream 5.4 MB/s | 20 MB 00:03 2024-09-21T06:15:50.574 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - CRB 9.5 MB/s | 6.5 MB 00:00 2024-09-21T06:15:52.266 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 19 kB 00:00 2024-09-21T06:15:52.983 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.5 MB 00:01 2024-09-21T06:15:53.356 INFO:teuthology.orchestra.run.smithi067.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-21T06:15:57.458 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - Extras packages 5.8 kB/s | 19 kB 00:03 2024-09-21T06:15:58.204 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 37 MB/s | 23 MB 00:00 2024-09-21T06:15:58.927 INFO:teuthology.orchestra.run.smithi067.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-21T06:16:00.941 INFO:teuthology.orchestra.run.smithi067.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T06:16:00.941 INFO:teuthology.orchestra.run.smithi067.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T06:16:01.029 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-21T06:16:01.034 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-21T06:16:01.035 INFO:teuthology.orchestra.run.smithi067.stdout: Package Architecture Version Repository Size 2024-09-21T06:16:01.035 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-21T06:16:01.035 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2024-09-21T06:16:01.035 INFO:teuthology.orchestra.run.smithi067.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T06:16:01.035 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2024-09-21T06:16:01.035 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T06:16:01.035 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T06:16:01.035 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T06:16:01.035 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:16:01.035 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-09-21T06:16:01.035 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-21T06:16:01.035 INFO:teuthology.orchestra.run.smithi067.stdout:Install 4 Packages 2024-09-21T06:16:01.035 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:16:01.036 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 79 M 2024-09-21T06:16:01.036 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 126 M 2024-09-21T06:16:01.036 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-09-21T06:16:01.161 INFO:teuthology.orchestra.run.smithi067.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 458 kB/s | 36 kB 00:00 2024-09-21T06:16:01.196 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - CRB 828 kB/s | 6.5 MB 00:08 2024-09-21T06:16:03.455 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 19 kB 00:01 2024-09-21T06:16:03.584 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-21T06:16:04.234 INFO:teuthology.orchestra.run.smithi203.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-21T06:16:05.061 INFO:teuthology.orchestra.run.smithi067.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 3.9 MB/s | 16 MB 00:03 2024-09-21T06:16:05.483 INFO:teuthology.orchestra.run.smithi100.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T06:16:05.483 INFO:teuthology.orchestra.run.smithi100.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T06:16:05.579 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-09-21T06:16:05.586 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-09-21T06:16:05.586 INFO:teuthology.orchestra.run.smithi100.stdout: Package Architecture Version Repository Size 2024-09-21T06:16:05.586 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-09-21T06:16:05.586 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-09-21T06:16:05.586 INFO:teuthology.orchestra.run.smithi100.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T06:16:05.586 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-09-21T06:16:05.587 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T06:16:05.587 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T06:16:05.587 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T06:16:05.587 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-21T06:16:05.587 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-09-21T06:16:05.587 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-09-21T06:16:05.587 INFO:teuthology.orchestra.run.smithi100.stdout:Install 4 Packages 2024-09-21T06:16:05.588 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-21T06:16:05.588 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 79 M 2024-09-21T06:16:05.588 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 126 M 2024-09-21T06:16:05.588 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-09-21T06:16:06.269 INFO:teuthology.orchestra.run.smithi100.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 65 kB/s | 36 kB 00:00 2024-09-21T06:16:07.103 INFO:teuthology.orchestra.run.smithi100.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-21T06:16:07.736 INFO:teuthology.orchestra.run.smithi100.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 17 MB/s | 35 MB 00:02 2024-09-21T06:16:08.286 INFO:teuthology.orchestra.run.smithi067.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 4.0 MB/s | 29 MB 00:07 2024-09-21T06:16:08.554 INFO:teuthology.orchestra.run.smithi100.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 13 MB/s | 29 MB 00:02 2024-09-21T06:16:08.554 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-09-21T06:16:08.554 INFO:teuthology.orchestra.run.smithi100.stdout:Total 27 MB/s | 79 MB 00:02 2024-09-21T06:16:09.237 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-09-21T06:16:09.575 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-09-21T06:16:09.575 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-09-21T06:16:09.690 INFO:teuthology.orchestra.run.smithi203.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-21T06:16:10.011 INFO:teuthology.orchestra.run.smithi067.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 3.9 MB/s | 35 MB 00:08 2024-09-21T06:16:10.011 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-09-21T06:16:10.011 INFO:teuthology.orchestra.run.smithi067.stdout:Total 8.8 MB/s | 79 MB 00:08 2024-09-21T06:16:10.541 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-09-21T06:16:10.541 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-09-21T06:16:10.720 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-09-21T06:16:11.081 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-09-21T06:16:11.081 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-09-21T06:16:11.663 INFO:teuthology.orchestra.run.smithi203.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T06:16:11.663 INFO:teuthology.orchestra.run.smithi203.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T06:16:11.754 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-09-21T06:16:11.766 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-09-21T06:16:11.767 INFO:teuthology.orchestra.run.smithi203.stdout: Package Architecture Version Repository Size 2024-09-21T06:16:11.767 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-09-21T06:16:11.767 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2024-09-21T06:16:11.767 INFO:teuthology.orchestra.run.smithi203.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T06:16:11.767 INFO:teuthology.orchestra.run.smithi203.stdout:Installing dependencies: 2024-09-21T06:16:11.767 INFO:teuthology.orchestra.run.smithi203.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T06:16:11.767 INFO:teuthology.orchestra.run.smithi203.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T06:16:11.767 INFO:teuthology.orchestra.run.smithi203.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T06:16:11.767 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T06:16:11.767 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-09-21T06:16:11.767 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-09-21T06:16:11.767 INFO:teuthology.orchestra.run.smithi203.stdout:Install 4 Packages 2024-09-21T06:16:11.767 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T06:16:11.768 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 79 M 2024-09-21T06:16:11.768 INFO:teuthology.orchestra.run.smithi203.stdout:Installed size: 126 M 2024-09-21T06:16:11.768 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-09-21T06:16:12.040 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-09-21T06:16:12.041 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-09-21T06:16:12.108 INFO:teuthology.orchestra.run.smithi203.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 426 kB/s | 36 kB 00:00 2024-09-21T06:16:12.500 INFO:teuthology.orchestra.run.smithi203.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-09-21T06:16:12.967 INFO:teuthology.orchestra.run.smithi203.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 37 MB/s | 35 MB 00:00 2024-09-21T06:16:12.997 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-09-21T06:16:13.325 INFO:teuthology.orchestra.run.smithi203.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 23 MB/s | 29 MB 00:01 2024-09-21T06:16:13.326 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-09-21T06:16:13.326 INFO:teuthology.orchestra.run.smithi203.stdout:Total 51 MB/s | 79 MB 00:01 2024-09-21T06:16:13.690 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T06:16:13.796 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T06:16:14.004 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-09-21T06:16:14.378 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-09-21T06:16:14.378 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-09-21T06:16:14.605 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-09-21T06:16:15.264 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T06:16:15.344 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T06:16:15.355 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-09-21T06:16:15.355 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-09-21T06:16:16.549 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T06:16:16.684 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T06:16:17.794 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-09-21T06:16:18.097 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T06:16:18.223 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T06:16:18.436 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T06:16:18.542 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T06:16:21.317 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T06:16:21.452 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T06:16:21.542 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T06:16:21.576 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T06:16:23.346 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T06:16:23.390 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T06:16:26.257 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T06:16:26.302 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T06:16:26.478 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T06:16:28.528 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T06:16:31.147 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T06:17:04.725 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T06:17:04.765 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T06:17:05.059 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T06:17:05.059 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T06:17:05.059 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T06:17:05.059 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T06:17:05.897 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T06:17:05.897 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-21T06:17:05.897 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-09-21T06:17:05.897 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T06:17:05.897 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T06:17:05.897 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T06:17:05.897 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T06:17:05.897 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-21T06:17:05.898 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-09-21T06:17:06.237 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T06:17:06.323 INFO:teuthology.orchestra.run.smithi100.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T06:17:06.324 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T06:17:06.324 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T06:17:06.324 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T06:17:06.324 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T06:17:06.324 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-21T06:17:06.324 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi100.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T06:17:06.325 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T06:17:06.382 INFO:teuthology.orchestra.run.smithi100.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T06:17:06.382 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T06:17:06.382 DEBUG:teuthology.orchestra.run.smithi100:> sudo rpm -qi grub2-tools 2024-09-21T06:17:06.417 INFO:teuthology.orchestra.run.smithi100.stdout:Name : grub2-tools 2024-09-21T06:17:06.417 INFO:teuthology.orchestra.run.smithi100.stdout:Epoch : 1 2024-09-21T06:17:06.417 INFO:teuthology.orchestra.run.smithi100.stdout:Version : 2.06 2024-09-21T06:17:06.417 INFO:teuthology.orchestra.run.smithi100.stdout:Release : 82.el9 2024-09-21T06:17:06.417 INFO:teuthology.orchestra.run.smithi100.stdout:Architecture: x86_64 2024-09-21T06:17:06.418 INFO:teuthology.orchestra.run.smithi100.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T06:17:06.418 INFO:teuthology.orchestra.run.smithi100.stdout:Group : Unspecified 2024-09-21T06:17:06.418 INFO:teuthology.orchestra.run.smithi100.stdout:Size : 8274670 2024-09-21T06:17:06.418 INFO:teuthology.orchestra.run.smithi100.stdout:License : GPLv3+ 2024-09-21T06:17:06.418 INFO:teuthology.orchestra.run.smithi100.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T06:17:06.418 INFO:teuthology.orchestra.run.smithi100.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T06:17:06.418 INFO:teuthology.orchestra.run.smithi100.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T06:17:06.418 INFO:teuthology.orchestra.run.smithi100.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T06:17:06.418 INFO:teuthology.orchestra.run.smithi100.stdout:Packager : builder@centos.org 2024-09-21T06:17:06.418 INFO:teuthology.orchestra.run.smithi100.stdout:Vendor : CentOS 2024-09-21T06:17:06.418 INFO:teuthology.orchestra.run.smithi100.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T06:17:06.418 INFO:teuthology.orchestra.run.smithi100.stdout:Summary : Support tools for GRUB. 2024-09-21T06:17:06.418 INFO:teuthology.orchestra.run.smithi100.stdout:Description : 2024-09-21T06:17:06.418 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-21T06:17:06.418 INFO:teuthology.orchestra.run.smithi100.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T06:17:06.419 INFO:teuthology.orchestra.run.smithi100.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T06:17:06.419 INFO:teuthology.orchestra.run.smithi100.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T06:17:06.419 INFO:teuthology.orchestra.run.smithi100.stdout:hardware devices. 2024-09-21T06:17:06.419 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-21T06:17:06.419 INFO:teuthology.orchestra.run.smithi100.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T06:17:06.420 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T06:17:06.420 INFO:teuthology.task.kernel:Updating grub on smithi100 to boot 5.14.0-508.el9.x86_64 2024-09-21T06:17:06.420 DEBUG:teuthology.orchestra.run.smithi100:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T06:17:07.078 INFO:teuthology.orchestra.run.smithi100.stderr:Generating grub configuration file ... 2024-09-21T06:17:07.674 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T06:17:07.716 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T06:17:08.045 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T06:17:08.046 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T06:17:08.046 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T06:17:08.046 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T06:17:08.447 INFO:teuthology.orchestra.run.smithi100.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T06:17:08.469 INFO:teuthology.orchestra.run.smithi100.stderr:done 2024-09-21T06:17:08.472 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2024-09-21T06:17:08.487 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.ocxC8VHnPw 2024-09-21T06:17:08.488 DEBUG:teuthology.orchestra.run.smithi100:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ocxC8VHnPw 2024-09-21T06:17:08.563 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.ocxC8VHnPw 2024-09-21T06:17:08.704 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.ocxC8VHnPw is 6KB 2024-09-21T06:17:08.753 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.ocxC8VHnPw 2024-09-21T06:17:08.768 DEBUG:teuthology.orchestra.run.smithi100:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T06:17:08.836 INFO:teuthology.orchestra.run.smithi100.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T06:17:08.836 INFO:teuthology.orchestra.run.smithi100.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T06:17:08.836 INFO:teuthology.orchestra.run.smithi100.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T06:17:08.836 INFO:teuthology.orchestra.run.smithi100.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T06:17:08.838 DEBUG:teuthology.orchestra.run.smithi100:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T06:17:08.900 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T06:17:08.901 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:17:08.902 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-09-21T06:17:08.902 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T06:17:08.902 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T06:17:08.902 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T06:17:08.902 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T06:17:08.902 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:17:08.903 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-21T06:17:08.992 DEBUG:teuthology.orchestra.run.smithi100:> sudo shutdown -r now 2024-09-21T06:17:09.243 DEBUG:teuthology.orchestra.run.smithi067:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T06:17:09.331 INFO:teuthology.orchestra.run.smithi067.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T06:17:09.332 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T06:17:09.332 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T06:17:09.332 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T06:17:09.332 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T06:17:09.332 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-21T06:17:09.332 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi067.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T06:17:09.332 DEBUG:teuthology.orchestra.run.smithi067:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T06:17:09.391 INFO:teuthology.orchestra.run.smithi067.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T06:17:09.392 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T06:17:09.392 DEBUG:teuthology.orchestra.run.smithi067:> sudo rpm -qi grub2-tools 2024-09-21T06:17:09.423 INFO:teuthology.orchestra.run.smithi067.stdout:Name : grub2-tools 2024-09-21T06:17:09.423 INFO:teuthology.orchestra.run.smithi067.stdout:Epoch : 1 2024-09-21T06:17:09.423 INFO:teuthology.orchestra.run.smithi067.stdout:Version : 2.06 2024-09-21T06:17:09.423 INFO:teuthology.orchestra.run.smithi067.stdout:Release : 82.el9 2024-09-21T06:17:09.423 INFO:teuthology.orchestra.run.smithi067.stdout:Architecture: x86_64 2024-09-21T06:17:09.423 INFO:teuthology.orchestra.run.smithi067.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T06:17:09.423 INFO:teuthology.orchestra.run.smithi067.stdout:Group : Unspecified 2024-09-21T06:17:09.423 INFO:teuthology.orchestra.run.smithi067.stdout:Size : 8274670 2024-09-21T06:17:09.423 INFO:teuthology.orchestra.run.smithi067.stdout:License : GPLv3+ 2024-09-21T06:17:09.423 INFO:teuthology.orchestra.run.smithi067.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T06:17:09.424 INFO:teuthology.orchestra.run.smithi067.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T06:17:09.424 INFO:teuthology.orchestra.run.smithi067.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T06:17:09.424 INFO:teuthology.orchestra.run.smithi067.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T06:17:09.424 INFO:teuthology.orchestra.run.smithi067.stdout:Packager : builder@centos.org 2024-09-21T06:17:09.424 INFO:teuthology.orchestra.run.smithi067.stdout:Vendor : CentOS 2024-09-21T06:17:09.424 INFO:teuthology.orchestra.run.smithi067.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T06:17:09.424 INFO:teuthology.orchestra.run.smithi067.stdout:Summary : Support tools for GRUB. 2024-09-21T06:17:09.424 INFO:teuthology.orchestra.run.smithi067.stdout:Description : 2024-09-21T06:17:09.424 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:17:09.424 INFO:teuthology.orchestra.run.smithi067.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T06:17:09.424 INFO:teuthology.orchestra.run.smithi067.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T06:17:09.424 INFO:teuthology.orchestra.run.smithi067.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T06:17:09.424 INFO:teuthology.orchestra.run.smithi067.stdout:hardware devices. 2024-09-21T06:17:09.424 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:17:09.425 INFO:teuthology.orchestra.run.smithi067.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T06:17:09.425 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T06:17:09.425 INFO:teuthology.task.kernel:Updating grub on smithi067 to boot 5.14.0-508.el9.x86_64 2024-09-21T06:17:09.426 DEBUG:teuthology.orchestra.run.smithi067:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T06:17:09.436 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T06:17:09.476 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T06:17:09.741 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T06:17:09.741 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T06:17:09.741 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T06:17:09.742 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T06:17:10.074 INFO:teuthology.orchestra.run.smithi067.stderr:Generating grub configuration file ... 2024-09-21T06:17:10.633 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T06:17:10.633 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T06:17:10.633 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2024-09-21T06:17:10.633 INFO:teuthology.orchestra.run.smithi203.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T06:17:10.633 INFO:teuthology.orchestra.run.smithi203.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T06:17:10.634 INFO:teuthology.orchestra.run.smithi203.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T06:17:10.634 INFO:teuthology.orchestra.run.smithi203.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T06:17:10.634 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T06:17:10.634 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-09-21T06:17:11.021 DEBUG:teuthology.orchestra.run.smithi203:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T06:17:11.113 INFO:teuthology.orchestra.run.smithi203.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T06:17:11.113 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T06:17:11.114 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T06:17:11.114 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T06:17:11.114 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T06:17:11.114 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-09-21T06:17:11.114 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi203.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T06:17:11.114 DEBUG:teuthology.orchestra.run.smithi203:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T06:17:11.172 INFO:teuthology.orchestra.run.smithi203.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T06:17:11.172 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T06:17:11.173 DEBUG:teuthology.orchestra.run.smithi203:> sudo rpm -qi grub2-tools 2024-09-21T06:17:11.204 INFO:teuthology.orchestra.run.smithi203.stdout:Name : grub2-tools 2024-09-21T06:17:11.204 INFO:teuthology.orchestra.run.smithi203.stdout:Epoch : 1 2024-09-21T06:17:11.204 INFO:teuthology.orchestra.run.smithi203.stdout:Version : 2.06 2024-09-21T06:17:11.204 INFO:teuthology.orchestra.run.smithi203.stdout:Release : 82.el9 2024-09-21T06:17:11.204 INFO:teuthology.orchestra.run.smithi203.stdout:Architecture: x86_64 2024-09-21T06:17:11.204 INFO:teuthology.orchestra.run.smithi203.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T06:17:11.205 INFO:teuthology.orchestra.run.smithi203.stdout:Group : Unspecified 2024-09-21T06:17:11.205 INFO:teuthology.orchestra.run.smithi203.stdout:Size : 8274670 2024-09-21T06:17:11.205 INFO:teuthology.orchestra.run.smithi203.stdout:License : GPLv3+ 2024-09-21T06:17:11.205 INFO:teuthology.orchestra.run.smithi203.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T06:17:11.205 INFO:teuthology.orchestra.run.smithi203.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T06:17:11.205 INFO:teuthology.orchestra.run.smithi203.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T06:17:11.205 INFO:teuthology.orchestra.run.smithi203.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T06:17:11.205 INFO:teuthology.orchestra.run.smithi203.stdout:Packager : builder@centos.org 2024-09-21T06:17:11.205 INFO:teuthology.orchestra.run.smithi203.stdout:Vendor : CentOS 2024-09-21T06:17:11.205 INFO:teuthology.orchestra.run.smithi203.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T06:17:11.205 INFO:teuthology.orchestra.run.smithi203.stdout:Summary : Support tools for GRUB. 2024-09-21T06:17:11.205 INFO:teuthology.orchestra.run.smithi203.stdout:Description : 2024-09-21T06:17:11.205 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T06:17:11.205 INFO:teuthology.orchestra.run.smithi203.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T06:17:11.205 INFO:teuthology.orchestra.run.smithi203.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T06:17:11.206 INFO:teuthology.orchestra.run.smithi203.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T06:17:11.206 INFO:teuthology.orchestra.run.smithi203.stdout:hardware devices. 2024-09-21T06:17:11.206 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T06:17:11.206 INFO:teuthology.orchestra.run.smithi203.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T06:17:11.206 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T06:17:11.207 INFO:teuthology.task.kernel:Updating grub on smithi203 to boot 5.14.0-508.el9.x86_64 2024-09-21T06:17:11.207 DEBUG:teuthology.orchestra.run.smithi203:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T06:17:11.383 INFO:teuthology.orchestra.run.smithi067.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T06:17:11.417 INFO:teuthology.orchestra.run.smithi067.stderr:done 2024-09-21T06:17:11.419 DEBUG:teuthology.orchestra.run.smithi067:> mktemp 2024-09-21T06:17:11.434 INFO:teuthology.orchestra.run.smithi067.stdout:/tmp/tmp.8TUSUjfvWx 2024-09-21T06:17:11.434 DEBUG:teuthology.orchestra.run.smithi067:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.8TUSUjfvWx 2024-09-21T06:17:11.514 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0666 /tmp/tmp.8TUSUjfvWx 2024-09-21T06:17:11.655 DEBUG:teuthology.orchestra.remote:smithi067:/tmp/tmp.8TUSUjfvWx is 6KB 2024-09-21T06:17:11.674 DEBUG:teuthology.orchestra.run.smithi067:> rm -fr /tmp/tmp.8TUSUjfvWx 2024-09-21T06:17:11.688 DEBUG:teuthology.orchestra.run.smithi067:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T06:17:11.758 INFO:teuthology.orchestra.run.smithi067.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T06:17:11.758 INFO:teuthology.orchestra.run.smithi067.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T06:17:11.758 INFO:teuthology.orchestra.run.smithi067.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T06:17:11.759 INFO:teuthology.orchestra.run.smithi067.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T06:17:11.760 DEBUG:teuthology.orchestra.run.smithi067:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T06:17:11.870 INFO:teuthology.orchestra.run.smithi203.stderr:Generating grub configuration file ... 2024-09-21T06:17:11.908 DEBUG:teuthology.orchestra.run.smithi067:> sudo shutdown -r now 2024-09-21T06:17:13.181 INFO:teuthology.orchestra.run.smithi203.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T06:17:13.216 INFO:teuthology.orchestra.run.smithi203.stderr:done 2024-09-21T06:17:13.218 DEBUG:teuthology.orchestra.run.smithi203:> mktemp 2024-09-21T06:17:13.234 INFO:teuthology.orchestra.run.smithi203.stdout:/tmp/tmp.hUPLrGgku8 2024-09-21T06:17:13.234 DEBUG:teuthology.orchestra.run.smithi203:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.hUPLrGgku8 2024-09-21T06:17:13.303 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0666 /tmp/tmp.hUPLrGgku8 2024-09-21T06:17:13.437 DEBUG:teuthology.orchestra.remote:smithi203:/tmp/tmp.hUPLrGgku8 is 6KB 2024-09-21T06:17:13.458 DEBUG:teuthology.orchestra.run.smithi203:> rm -fr /tmp/tmp.hUPLrGgku8 2024-09-21T06:17:13.473 DEBUG:teuthology.orchestra.run.smithi203:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T06:17:13.536 INFO:teuthology.orchestra.run.smithi203.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T06:17:13.536 INFO:teuthology.orchestra.run.smithi203.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T06:17:13.536 INFO:teuthology.orchestra.run.smithi203.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T06:17:13.537 INFO:teuthology.orchestra.run.smithi203.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T06:17:13.538 DEBUG:teuthology.orchestra.run.smithi203:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T06:17:13.686 DEBUG:teuthology.orchestra.run.smithi203:> sudo shutdown -r now 2024-09-21T06:17:39.010 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-21T06:17:39.010 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-21T06:17:39.011 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:17:41.912 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-21T06:17:41.912 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-09-21T06:17:41.913 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:17:43.690 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-09-21T06:17:43.691 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-09-21T06:17:43.691 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:17:57.487 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-09-21T06:18:02.095 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.203 2024-09-21T06:18:06.492 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-21T06:18:06.493 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:18:09.551 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-09-21T06:18:11.097 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-09-21T06:18:11.098 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:18:14.159 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.203 2024-09-21T06:18:16.431 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.67 2024-09-21T06:18:21.554 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-21T06:18:21.555 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:18:25.435 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-09-21T06:18:25.436 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:18:26.160 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-09-21T06:18:26.161 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:18:43.823 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.67 2024-09-21T06:18:44.591 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.203 2024-09-21T06:18:53.330 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-09-21T06:18:53.958 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-21T06:18:53.958 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-21T06:18:53.958 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-09-21T06:18:53.976 INFO:teuthology.orchestra.run.smithi100.stdout:5.14.0-508.el9.x86_64 2024-09-21T06:18:53.976 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T06:18:53.976 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T06:18:53.977 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T06:18:54.977 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-21T06:18:54.977 DEBUG:teuthology.orchestra.run.smithi100:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T06:18:55.042 INFO:teuthology.orchestra.run.smithi100.stdout:ttyS1 2024-09-21T06:18:55.061 DEBUG:teuthology.parallel:result is None 2024-09-21T06:18:55.824 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-09-21T06:18:55.824 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:18:56.197 DEBUG:teuthology.orchestra.run.smithi067:> true 2024-09-21T06:18:56.513 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-09-21T06:18:56.513 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-21T06:18:56.513 DEBUG:teuthology.orchestra.run.smithi067:> uname -r 2024-09-21T06:18:56.528 INFO:teuthology.orchestra.run.smithi067.stdout:5.14.0-508.el9.x86_64 2024-09-21T06:18:56.528 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T06:18:56.528 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T06:18:56.528 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T06:18:57.529 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-21T06:18:57.529 DEBUG:teuthology.orchestra.run.smithi067:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T06:18:57.615 INFO:teuthology.orchestra.run.smithi067.stdout:ttyS1 2024-09-21T06:18:57.643 DEBUG:teuthology.parallel:result is None 2024-09-21T06:18:59.592 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-09-21T06:18:59.592 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:18:59.952 DEBUG:teuthology.orchestra.run.smithi203:> true 2024-09-21T06:19:00.419 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-09-21T06:19:00.419 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-508.el9.x86_64"... 2024-09-21T06:19:00.419 DEBUG:teuthology.orchestra.run.smithi203:> uname -r 2024-09-21T06:19:00.433 INFO:teuthology.orchestra.run.smithi203.stdout:5.14.0-508.el9.x86_64 2024-09-21T06:19:00.433 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T06:19:00.433 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T06:19:00.434 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T06:19:01.435 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-09-21T06:19:01.435 DEBUG:teuthology.orchestra.run.smithi203:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T06:19:01.523 INFO:teuthology.orchestra.run.smithi203.stdout:ttyS1 2024-09-21T06:19:01.552 DEBUG:teuthology.parallel:result is None 2024-09-21T06:19:01.552 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T06:19:01.561 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T06:19:01.561 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T06:19:01.563 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T06:19:01.566 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T06:19:01.582 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T06:19:01.588 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T06:19:01.595 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T06:19:01.595 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T06:19:01.618 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T06:19:01.625 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T06:19:01.876 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T06:19:01.882 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T06:19:01.883 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T06:19:01.885 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T06:19:01.887 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T06:19:01.919 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T06:19:01.921 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T06:19:01.929 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T06:19:01.933 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T06:19:02.082 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T06:19:02.095 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T06:19:02.097 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T06:19:02.111 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T06:19:02.111 DEBUG:teuthology.orchestra.run.smithi067:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T06:19:02.113 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T06:19:02.116 DEBUG:teuthology.orchestra.run.smithi203:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T06:19:02.142 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T06:19:02.150 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T06:19:02.150 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T06:19:02.180 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T06:19:02.185 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T06:19:02.200 DEBUG:teuthology.orchestra.run.smithi067:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T06:19:02.280 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T06:19:02.325 DEBUG:teuthology.orchestra.run.smithi067:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T06:19:02.402 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T06:19:02.441 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-21T06:19:02.441 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T06:19:02.506 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T06:19:02.551 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T06:19:02.609 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T06:19:02.693 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T06:19:02.736 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-21T06:19:02.736 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T06:19:02.803 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T06:19:02.845 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T06:19:02.896 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T06:19:02.975 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T06:19:03.018 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-21T06:19:03.019 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T06:19:03.084 DEBUG:teuthology.orchestra.run.smithi067:> sudo service rsyslog restart 2024-09-21T06:19:03.086 DEBUG:teuthology.orchestra.run.smithi100:> sudo service rsyslog restart 2024-09-21T06:19:03.089 DEBUG:teuthology.orchestra.run.smithi203:> sudo service rsyslog restart 2024-09-21T06:19:03.129 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T06:19:03.136 INFO:teuthology.orchestra.run.smithi067.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T06:19:03.188 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T06:19:03.645 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T06:19:03.652 INFO:teuthology.task.internal:Starting timer... 2024-09-21T06:19:03.652 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T06:19:03.662 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T06:19:03.670 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-21T06:19:03.670 DEBUG:teuthology.orchestra.run.smithi067:> sudo service auditd rotate 2024-09-21T06:19:03.735 INFO:teuthology.orchestra.run.smithi067.stdout:Rotating logs: 2024-09-21T06:19:03.737 DEBUG:teuthology.orchestra.run.smithi100:> sudo service auditd rotate 2024-09-21T06:19:03.801 INFO:teuthology.orchestra.run.smithi100.stdout:Rotating logs: 2024-09-21T06:19:03.803 DEBUG:teuthology.orchestra.run.smithi203:> sudo service auditd rotate 2024-09-21T06:19:03.857 INFO:teuthology.orchestra.run.smithi203.stdout:Rotating logs: 2024-09-21T06:19:03.859 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T06:19:03.859 DEBUG:teuthology.orchestra.run.smithi067:> /usr/sbin/getenforce 2024-09-21T06:19:03.900 INFO:teuthology.orchestra.run.smithi067.stdout:Permissive 2024-09-21T06:19:03.900 DEBUG:teuthology.orchestra.run.smithi100:> /usr/sbin/getenforce 2024-09-21T06:19:03.918 INFO:teuthology.orchestra.run.smithi100.stdout:Permissive 2024-09-21T06:19:03.918 DEBUG:teuthology.orchestra.run.smithi203:> /usr/sbin/getenforce 2024-09-21T06:19:03.938 INFO:teuthology.orchestra.run.smithi203.stdout:Permissive 2024-09-21T06:19:03.938 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi067.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi100.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi203.front.sepia.ceph.com': 'permissive'} 2024-09-21T06:19:03.938 DEBUG:teuthology.orchestra.run.smithi067:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T06:19:03.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:19:03.969 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T06:19:03.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:19:03.997 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T06:19:04.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:19:04.022 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T06:19:04.022 DEBUG:teuthology.orchestra.run.smithi067:> sudo /usr/sbin/setenforce permissive 2024-09-21T06:19:04.054 DEBUG:teuthology.orchestra.run.smithi100:> sudo /usr/sbin/setenforce permissive 2024-09-21T06:19:04.084 DEBUG:teuthology.orchestra.run.smithi203:> sudo /usr/sbin/setenforce permissive 2024-09-21T06:19:04.111 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T06:19:04.121 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T06:19:04.130 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T06:19:04.131 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi067.front.sepia.ceph.com,smithi100.front.sepia.ceph.com,smithi203.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T06:22:49.954 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi067.front.sepia.ceph.com'), Remote(name='ubuntu@smithi100.front.sepia.ceph.com'), Remote(name='ubuntu@smithi203.front.sepia.ceph.com')] 2024-09-21T06:22:49.955 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-09-21T06:22:49.956 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:22:50.040 DEBUG:teuthology.orchestra.run.smithi067:> true 2024-09-21T06:22:50.124 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-09-21T06:22:50.125 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-21T06:22:50.126 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:22:50.201 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-09-21T06:22:50.285 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-21T06:22:50.285 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-09-21T06:22:50.286 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:22:50.370 DEBUG:teuthology.orchestra.run.smithi203:> true 2024-09-21T06:22:50.446 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-09-21T06:22:50.447 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T06:22:50.458 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T06:22:50.458 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T06:22:50.459 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T06:22:50.461 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T06:22:50.461 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T06:22:50.464 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T06:22:50.464 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T06:22:50.494 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T06:22:50.495 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T06:22:50.511 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T06:22:50.511 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T06:22:50.518 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T06:22:50.531 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T06:22:50.536 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: ntpd: command not found 2024-09-21T06:22:50.538 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: ntpd: command not found 2024-09-21T06:22:50.548 INFO:teuthology.orchestra.run.smithi100.stdout:506 Cannot talk to daemon 2024-09-21T06:22:50.551 INFO:teuthology.orchestra.run.smithi067.stdout:506 Cannot talk to daemon 2024-09-21T06:22:50.556 INFO:teuthology.orchestra.run.smithi203.stderr:sudo: ntpd: command not found 2024-09-21T06:22:50.563 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T06:22:50.566 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T06:22:50.567 INFO:teuthology.orchestra.run.smithi203.stdout:506 Cannot talk to daemon 2024-09-21T06:22:50.578 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T06:22:50.581 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T06:22:50.581 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T06:22:50.595 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T06:22:50.631 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2024-09-21T06:22:50.635 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T06:22:50.635 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2024-09-21T06:22:50.635 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:22:50.635 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:22:50.635 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:22:50.635 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:22:50.636 INFO:teuthology.orchestra.run.smithi067.stderr:bash: line 1: ntpq: command not found 2024-09-21T06:22:50.640 INFO:teuthology.orchestra.run.smithi067.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T06:22:50.640 INFO:teuthology.orchestra.run.smithi067.stdout:=============================================================================== 2024-09-21T06:22:50.640 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:22:50.640 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:22:50.640 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:22:50.640 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:22:50.652 INFO:teuthology.orchestra.run.smithi203.stderr:bash: line 1: ntpq: command not found 2024-09-21T06:22:50.656 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T06:22:50.656 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2024-09-21T06:22:50.656 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:22:50.656 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:22:50.656 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:22:50.656 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:22:50.656 INFO:teuthology.run_tasks:Running task install... 2024-09-21T06:22:50.664 DEBUG:teuthology.task.install:project ceph 2024-09-21T06:22:50.664 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}} 2024-09-21T06:22:50.664 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'} 2024-09-21T06:22:50.664 INFO:teuthology.task.install:Using flavor: default 2024-09-21T06:22:50.669 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T06:22:50.670 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-21T06:22:50.670 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T06:22:50.670 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:22:50.672 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T06:22:50.672 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:22:50.673 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T06:22:50.673 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:22:50.816 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T06:22:50.817 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T06:22:50.826 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T06:22:50.826 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T06:22:50.831 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T06:22:50.831 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T06:22:50.937 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T06:22:50.937 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-21T06:22:50.937 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T06:22:50.949 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T06:22:50.949 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-21T06:22:50.950 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T06:22:50.957 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T06:22:50.957 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-21T06:22:50.957 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T06:22:50.965 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T06:22:50.965 DEBUG:teuthology.orchestra.run.smithi067:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T06:22:50.975 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T06:22:50.975 DEBUG:teuthology.orchestra.run.smithi100:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T06:22:50.983 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T06:22:50.983 DEBUG:teuthology.orchestra.run.smithi203:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T06:22:51.033 DEBUG:teuthology.orchestra.run.smithi067:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T06:22:51.042 DEBUG:teuthology.orchestra.run.smithi100:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T06:22:51.050 DEBUG:teuthology.orchestra.run.smithi203:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T06:22:51.111 DEBUG:teuthology.orchestra.run.smithi067:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T06:22:51.118 DEBUG:teuthology.orchestra.run.smithi100:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T06:22:51.125 DEBUG:teuthology.orchestra.run.smithi203:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T06:22:51.176 INFO:teuthology.orchestra.run.smithi067.stdout:check_obsoletes = 1 2024-09-21T06:22:51.178 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum clean all 2024-09-21T06:22:51.182 INFO:teuthology.orchestra.run.smithi100.stdout:check_obsoletes = 1 2024-09-21T06:22:51.183 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum clean all 2024-09-21T06:22:51.190 INFO:teuthology.orchestra.run.smithi203.stdout:check_obsoletes = 1 2024-09-21T06:22:51.191 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum clean all 2024-09-21T06:22:51.415 INFO:teuthology.orchestra.run.smithi100.stdout:45 files removed 2024-09-21T06:22:51.417 INFO:teuthology.orchestra.run.smithi067.stdout:45 files removed 2024-09-21T06:22:51.430 INFO:teuthology.orchestra.run.smithi203.stdout:45 files removed 2024-09-21T06:22:51.442 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T06:22:51.444 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T06:22:51.457 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T06:22:51.989 INFO:teuthology.orchestra.run.smithi100.stdout:ceph packages for x86_64 323 kB/s | 84 kB 00:00 2024-09-21T06:22:52.012 INFO:teuthology.orchestra.run.smithi203.stdout:ceph packages for x86_64 312 kB/s | 84 kB 00:00 2024-09-21T06:22:52.018 INFO:teuthology.orchestra.run.smithi067.stdout:ceph packages for x86_64 305 kB/s | 84 kB 00:00 2024-09-21T06:22:52.273 INFO:teuthology.orchestra.run.smithi100.stdout:ceph noarch packages 100 kB/s | 24 kB 00:00 2024-09-21T06:22:52.295 INFO:teuthology.orchestra.run.smithi203.stdout:ceph noarch packages 101 kB/s | 24 kB 00:00 2024-09-21T06:22:52.311 INFO:teuthology.orchestra.run.smithi067.stdout:ceph noarch packages 98 kB/s | 24 kB 00:00 2024-09-21T06:22:52.548 INFO:teuthology.orchestra.run.smithi100.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-21T06:22:52.596 INFO:teuthology.orchestra.run.smithi203.stdout:ceph source packages 7.2 kB/s | 1.9 kB 00:00 2024-09-21T06:22:52.795 INFO:teuthology.orchestra.run.smithi067.stdout:ceph source packages 4.3 kB/s | 1.9 kB 00:00 2024-09-21T06:22:54.382 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.3 MB 00:01 2024-09-21T06:22:54.637 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.3 MB 00:01 2024-09-21T06:22:55.196 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - BaseOS 3.2 MB/s | 8.3 MB 00:02 2024-09-21T06:22:56.422 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - AppStream 32 MB/s | 20 MB 00:00 2024-09-21T06:22:59.488 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - AppStream 6.3 MB/s | 20 MB 00:03 2024-09-21T06:23:00.939 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-21T06:23:02.674 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 19 kB 00:00 2024-09-21T06:23:03.465 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - AppStream 2.5 MB/s | 20 MB 00:08 2024-09-21T06:23:03.629 INFO:teuthology.orchestra.run.smithi067.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-21T06:23:05.439 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.5 MB 00:01 2024-09-21T06:23:06.965 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - Extras packages 116 kB/s | 19 kB 00:00 2024-09-21T06:23:07.691 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-21T06:23:07.895 INFO:teuthology.orchestra.run.smithi203.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-21T06:23:09.217 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 19 kB 00:00 2024-09-21T06:23:09.235 INFO:teuthology.orchestra.run.smithi067.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-21T06:23:09.972 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-21T06:23:11.496 INFO:teuthology.orchestra.run.smithi067.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T06:23:11.497 INFO:teuthology.orchestra.run.smithi067.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T06:23:11.509 INFO:teuthology.orchestra.run.smithi067.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T06:23:11.509 INFO:teuthology.orchestra.run.smithi067.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T06:23:11.510 INFO:teuthology.orchestra.run.smithi067.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T06:23:11.511 INFO:teuthology.orchestra.run.smithi067.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T06:23:11.513 INFO:teuthology.orchestra.run.smithi067.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T06:23:11.515 INFO:teuthology.orchestra.run.smithi067.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T06:23:11.516 INFO:teuthology.orchestra.run.smithi067.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T06:23:11.583 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-21T06:23:11.612 INFO:teuthology.orchestra.run.smithi067.stdout:======================================================================================= 2024-09-21T06:23:11.612 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2024-09-21T06:23:11.612 INFO:teuthology.orchestra.run.smithi067.stdout:======================================================================================= 2024-09-21T06:23:11.612 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2024-09-21T06:23:11.612 INFO:teuthology.orchestra.run.smithi067.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T06:23:11.612 INFO:teuthology.orchestra.run.smithi067.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T06:23:11.612 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T06:23:11.612 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T06:23:11.612 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T06:23:11.612 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T06:23:11.612 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T06:23:11.612 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T06:23:11.612 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T06:23:11.612 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T06:23:11.613 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T06:23:11.613 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T06:23:11.613 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T06:23:11.613 INFO:teuthology.orchestra.run.smithi067.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T06:23:11.613 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T06:23:11.613 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T06:23:11.613 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T06:23:11.613 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T06:23:11.613 INFO:teuthology.orchestra.run.smithi067.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T06:23:11.613 INFO:teuthology.orchestra.run.smithi067.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T06:23:11.613 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T06:23:11.613 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T06:23:11.613 INFO:teuthology.orchestra.run.smithi067.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T06:23:11.613 INFO:teuthology.orchestra.run.smithi067.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T06:23:11.613 INFO:teuthology.orchestra.run.smithi067.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T06:23:11.614 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T06:23:11.614 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T06:23:11.614 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T06:23:11.614 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T06:23:11.614 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T06:23:11.614 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T06:23:11.614 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T06:23:11.614 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T06:23:11.614 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T06:23:11.614 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T06:23:11.614 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T06:23:11.614 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T06:23:11.614 INFO:teuthology.orchestra.run.smithi067.stdout:Upgrading: 2024-09-21T06:23:11.614 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T06:23:11.614 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T06:23:11.615 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T06:23:11.615 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T06:23:11.615 INFO:teuthology.orchestra.run.smithi067.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T06:23:11.615 INFO:teuthology.orchestra.run.smithi067.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T06:23:11.615 INFO:teuthology.orchestra.run.smithi067.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T06:23:11.615 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T06:23:11.615 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2024-09-21T06:23:11.615 INFO:teuthology.orchestra.run.smithi067.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T06:23:11.615 INFO:teuthology.orchestra.run.smithi067.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T06:23:11.615 INFO:teuthology.orchestra.run.smithi067.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T06:23:11.615 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T06:23:11.615 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T06:23:11.615 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T06:23:11.615 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T06:23:11.615 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T06:23:11.616 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T06:23:11.616 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T06:23:11.616 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T06:23:11.616 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T06:23:11.616 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T06:23:11.616 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T06:23:11.616 INFO:teuthology.orchestra.run.smithi067.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T06:23:11.616 INFO:teuthology.orchestra.run.smithi067.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T06:23:11.617 INFO:teuthology.orchestra.run.smithi067.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T06:23:11.617 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T06:23:11.617 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T06:23:11.617 INFO:teuthology.orchestra.run.smithi067.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T06:23:11.617 INFO:teuthology.orchestra.run.smithi067.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T06:23:11.617 INFO:teuthology.orchestra.run.smithi067.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T06:23:11.617 INFO:teuthology.orchestra.run.smithi067.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T06:23:11.617 INFO:teuthology.orchestra.run.smithi067.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T06:23:11.617 INFO:teuthology.orchestra.run.smithi067.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T06:23:11.617 INFO:teuthology.orchestra.run.smithi067.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T06:23:11.617 INFO:teuthology.orchestra.run.smithi067.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T06:23:11.617 INFO:teuthology.orchestra.run.smithi067.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T06:23:11.617 INFO:teuthology.orchestra.run.smithi067.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T06:23:11.617 INFO:teuthology.orchestra.run.smithi067.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T06:23:11.618 INFO:teuthology.orchestra.run.smithi067.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T06:23:11.618 INFO:teuthology.orchestra.run.smithi067.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T06:23:11.618 INFO:teuthology.orchestra.run.smithi067.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T06:23:11.618 INFO:teuthology.orchestra.run.smithi067.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T06:23:11.618 INFO:teuthology.orchestra.run.smithi067.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T06:23:11.618 INFO:teuthology.orchestra.run.smithi067.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T06:23:11.618 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T06:23:11.618 INFO:teuthology.orchestra.run.smithi067.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T06:23:11.618 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T06:23:11.618 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T06:23:11.618 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T06:23:11.618 INFO:teuthology.orchestra.run.smithi067.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T06:23:11.619 INFO:teuthology.orchestra.run.smithi067.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T06:23:11.619 INFO:teuthology.orchestra.run.smithi067.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T06:23:11.619 INFO:teuthology.orchestra.run.smithi067.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T06:23:11.619 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T06:23:11.619 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T06:23:11.619 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T06:23:11.619 INFO:teuthology.orchestra.run.smithi067.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T06:23:11.619 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T06:23:11.619 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T06:23:11.619 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T06:23:11.619 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T06:23:11.619 INFO:teuthology.orchestra.run.smithi067.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T06:23:11.620 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T06:23:11.620 INFO:teuthology.orchestra.run.smithi067.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T06:23:11.620 INFO:teuthology.orchestra.run.smithi067.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T06:23:11.620 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T06:23:11.620 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T06:23:11.620 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T06:23:11.620 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T06:23:11.620 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T06:23:11.620 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T06:23:11.620 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T06:23:11.620 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T06:23:11.620 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T06:23:11.620 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T06:23:11.621 INFO:teuthology.orchestra.run.smithi067.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T06:23:11.621 INFO:teuthology.orchestra.run.smithi067.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T06:23:11.621 INFO:teuthology.orchestra.run.smithi067.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T06:23:11.621 INFO:teuthology.orchestra.run.smithi067.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T06:23:11.621 INFO:teuthology.orchestra.run.smithi067.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T06:23:11.621 INFO:teuthology.orchestra.run.smithi067.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T06:23:11.621 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T06:23:11.621 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T06:23:11.621 INFO:teuthology.orchestra.run.smithi067.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T06:23:11.621 INFO:teuthology.orchestra.run.smithi067.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T06:23:11.621 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T06:23:11.621 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T06:23:11.621 INFO:teuthology.orchestra.run.smithi067.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T06:23:11.621 INFO:teuthology.orchestra.run.smithi067.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T06:23:11.621 INFO:teuthology.orchestra.run.smithi067.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T06:23:11.622 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T06:23:11.622 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T06:23:11.622 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T06:23:11.622 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T06:23:11.622 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T06:23:11.622 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T06:23:11.622 INFO:teuthology.orchestra.run.smithi067.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T06:23:11.622 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T06:23:11.622 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T06:23:11.622 INFO:teuthology.orchestra.run.smithi067.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T06:23:11.622 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T06:23:11.622 INFO:teuthology.orchestra.run.smithi067.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T06:23:11.622 INFO:teuthology.orchestra.run.smithi067.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T06:23:11.622 INFO:teuthology.orchestra.run.smithi067.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T06:23:11.623 INFO:teuthology.orchestra.run.smithi067.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T06:23:11.623 INFO:teuthology.orchestra.run.smithi067.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T06:23:11.623 INFO:teuthology.orchestra.run.smithi067.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T06:23:11.623 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T06:23:11.623 INFO:teuthology.orchestra.run.smithi067.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T06:23:11.623 INFO:teuthology.orchestra.run.smithi067.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T06:23:11.623 INFO:teuthology.orchestra.run.smithi067.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T06:23:11.623 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T06:23:11.623 INFO:teuthology.orchestra.run.smithi067.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T06:23:11.623 INFO:teuthology.orchestra.run.smithi067.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T06:23:11.623 INFO:teuthology.orchestra.run.smithi067.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T06:23:11.623 INFO:teuthology.orchestra.run.smithi067.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T06:23:11.623 INFO:teuthology.orchestra.run.smithi067.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T06:23:11.623 INFO:teuthology.orchestra.run.smithi067.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T06:23:11.624 INFO:teuthology.orchestra.run.smithi067.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T06:23:11.624 INFO:teuthology.orchestra.run.smithi067.stdout:Installing weak dependencies: 2024-09-21T06:23:11.624 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T06:23:11.624 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:23:11.624 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-09-21T06:23:11.624 INFO:teuthology.orchestra.run.smithi067.stdout:======================================================================================= 2024-09-21T06:23:11.624 INFO:teuthology.orchestra.run.smithi067.stdout:Install 149 Packages 2024-09-21T06:23:11.624 INFO:teuthology.orchestra.run.smithi067.stdout:Upgrade 8 Packages 2024-09-21T06:23:11.624 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:23:11.624 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 228 M 2024-09-21T06:23:11.625 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-09-21T06:23:12.228 INFO:teuthology.orchestra.run.smithi067.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 54 kB/s | 6.6 kB 00:00 2024-09-21T06:23:12.428 INFO:teuthology.orchestra.run.smithi067.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 4.3 MB/s | 873 kB 00:00 2024-09-21T06:23:12.520 INFO:teuthology.orchestra.run.smithi067.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 1.6 MB/s | 148 kB 00:00 2024-09-21T06:23:12.604 INFO:teuthology.orchestra.run.smithi067.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 11 MB/s | 5.5 MB 00:00 2024-09-21T06:23:12.704 INFO:teuthology.orchestra.run.smithi067.stdout:(5/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 15 MB/s | 1.5 MB 00:00 2024-09-21T06:23:12.788 INFO:teuthology.orchestra.run.smithi067.stdout:(6/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 8.6 MB/s | 2.3 MB 00:00 2024-09-21T06:23:12.938 INFO:teuthology.orchestra.run.smithi067.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 21 MB/s | 5.0 MB 00:00 2024-09-21T06:23:13.196 INFO:teuthology.orchestra.run.smithi067.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 18 MB/s | 20 MB 00:01 2024-09-21T06:23:13.305 INFO:teuthology.orchestra.run.smithi067.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 232 kB/s | 25 kB 00:00 2024-09-21T06:23:13.359 INFO:teuthology.orchestra.run.smithi203.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-21T06:23:13.630 INFO:teuthology.orchestra.run.smithi067.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 15 MB/s | 10 MB 00:00 2024-09-21T06:23:13.706 INFO:teuthology.orchestra.run.smithi067.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 2.9 MB/s | 226 kB 00:00 2024-09-21T06:23:13.764 INFO:teuthology.orchestra.run.smithi067.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 465 kB/s | 27 kB 00:00 2024-09-21T06:23:13.965 INFO:teuthology.orchestra.run.smithi067.stdout:(13/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 15 MB/s | 18 MB 00:01 2024-09-21T06:23:13.990 INFO:teuthology.orchestra.run.smithi067.stdout:(14/157): libcephfs-devel-19.3.0-5120.g18432218 141 kB/s | 32 kB 00:00 2024-09-21T06:23:14.032 INFO:teuthology.orchestra.run.smithi067.stdout:(15/157): libcephfs2-19.3.0-5120.g18432218.el9. 11 MB/s | 766 kB 00:00 2024-09-21T06:23:14.057 INFO:teuthology.orchestra.run.smithi067.stdout:(16/157): libcephsqlite-19.3.0-5120.g18432218.e 2.3 MB/s | 158 kB 00:00 2024-09-21T06:23:14.082 INFO:teuthology.orchestra.run.smithi067.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 2.5 MB/s | 127 kB 00:00 2024-09-21T06:23:14.116 INFO:teuthology.orchestra.run.smithi067.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 8.5 MB/s | 507 kB 00:00 2024-09-21T06:23:14.175 INFO:teuthology.orchestra.run.smithi067.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 770 kB/s | 45 kB 00:00 2024-09-21T06:23:14.250 INFO:teuthology.orchestra.run.smithi067.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.9 MB/s | 148 kB 00:00 2024-09-21T06:23:14.334 INFO:teuthology.orchestra.run.smithi067.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 1.9 MB/s | 161 kB 00:00 2024-09-21T06:23:14.401 INFO:teuthology.orchestra.run.smithi067.stdout:(22/157): librgw2-19.3.0-5120.g18432218.el9.x86 17 MB/s | 5.3 MB 00:00 2024-09-21T06:23:14.426 INFO:teuthology.orchestra.run.smithi067.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 3.4 MB/s | 321 kB 00:00 2024-09-21T06:23:14.451 INFO:teuthology.orchestra.run.smithi067.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 6.0 MB/s | 307 kB 00:00 2024-09-21T06:23:14.477 INFO:teuthology.orchestra.run.smithi067.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.9 MB/s | 99 kB 00:00 2024-09-21T06:23:14.502 INFO:teuthology.orchestra.run.smithi067.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T06:23:14.552 INFO:teuthology.orchestra.run.smithi067.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 3.3 MB/s | 170 kB 00:00 2024-09-21T06:23:14.619 INFO:teuthology.orchestra.run.smithi067.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 514 kB/s | 34 kB 00:00 2024-09-21T06:23:14.678 INFO:teuthology.orchestra.run.smithi067.stdout:(29/157): rbd-mirror-19.3.0-5120.g18432218.el9. 15 MB/s | 3.1 MB 00:00 2024-09-21T06:23:14.706 INFO:teuthology.orchestra.run.smithi067.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 1.8 MB/s | 154 kB 00:00 2024-09-21T06:23:15.128 INFO:teuthology.orchestra.run.smithi067.stdout:(31/157): ceph-test-19.3.0-5120.g18432218.el9.x 22 MB/s | 40 MB 00:01 2024-09-21T06:23:15.179 INFO:teuthology.orchestra.run.smithi067.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 5.7 MB/s | 292 kB 00:00 2024-09-21T06:23:15.245 INFO:teuthology.orchestra.run.smithi067.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 744 kB/s | 49 kB 00:00 2024-09-21T06:23:15.337 INFO:teuthology.orchestra.run.smithi067.stdout:(34/157): ceph-mgr-diskprediction-local-19.3.0- 12 MB/s | 7.4 MB 00:00 2024-09-21T06:23:15.345 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-21T06:23:15.362 INFO:teuthology.orchestra.run.smithi067.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 141 kB/s | 16 kB 00:00 2024-09-21T06:23:15.396 INFO:teuthology.orchestra.run.smithi067.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 4.8 MB/s | 285 kB 00:00 2024-09-21T06:23:15.438 INFO:teuthology.orchestra.run.smithi067.stdout:(37/157): cephadm-19.3.0-5120.g18432218.el9.noa 10 MB/s | 784 kB 00:00 2024-09-21T06:23:15.463 INFO:teuthology.orchestra.run.smithi067.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 276 kB/s | 18 kB 00:00 2024-09-21T06:23:15.545 INFO:teuthology.orchestra.run.smithi203.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T06:23:15.545 INFO:teuthology.orchestra.run.smithi203.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T06:23:15.557 INFO:teuthology.orchestra.run.smithi203.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T06:23:15.558 INFO:teuthology.orchestra.run.smithi203.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T06:23:15.558 INFO:teuthology.orchestra.run.smithi203.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T06:23:15.559 INFO:teuthology.orchestra.run.smithi203.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T06:23:15.561 INFO:teuthology.orchestra.run.smithi203.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T06:23:15.563 INFO:teuthology.orchestra.run.smithi203.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T06:23:15.564 INFO:teuthology.orchestra.run.smithi203.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T06:23:15.627 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-09-21T06:23:15.630 INFO:teuthology.orchestra.run.smithi067.stdout:(39/157): ceph-mgr-dashboard-19.3.0-5120.g18432 15 MB/s | 14 MB 00:00 2024-09-21T06:23:15.651 INFO:teuthology.orchestra.run.smithi203.stdout:======================================================================================= 2024-09-21T06:23:15.651 INFO:teuthology.orchestra.run.smithi203.stdout: Package Arch Version Repository Size 2024-09-21T06:23:15.651 INFO:teuthology.orchestra.run.smithi203.stdout:======================================================================================= 2024-09-21T06:23:15.651 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2024-09-21T06:23:15.651 INFO:teuthology.orchestra.run.smithi203.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T06:23:15.651 INFO:teuthology.orchestra.run.smithi203.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T06:23:15.651 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T06:23:15.651 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T06:23:15.651 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T06:23:15.651 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T06:23:15.652 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T06:23:15.652 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T06:23:15.652 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T06:23:15.652 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T06:23:15.652 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T06:23:15.652 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T06:23:15.652 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T06:23:15.652 INFO:teuthology.orchestra.run.smithi203.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T06:23:15.652 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T06:23:15.652 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T06:23:15.652 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T06:23:15.652 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T06:23:15.652 INFO:teuthology.orchestra.run.smithi203.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T06:23:15.652 INFO:teuthology.orchestra.run.smithi203.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T06:23:15.653 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T06:23:15.653 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T06:23:15.653 INFO:teuthology.orchestra.run.smithi203.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T06:23:15.653 INFO:teuthology.orchestra.run.smithi203.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T06:23:15.653 INFO:teuthology.orchestra.run.smithi203.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T06:23:15.653 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T06:23:15.653 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T06:23:15.653 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T06:23:15.653 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T06:23:15.653 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T06:23:15.653 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T06:23:15.653 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T06:23:15.653 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T06:23:15.653 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T06:23:15.653 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T06:23:15.654 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T06:23:15.654 INFO:teuthology.orchestra.run.smithi203.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T06:23:15.654 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrading: 2024-09-21T06:23:15.654 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T06:23:15.654 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T06:23:15.654 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T06:23:15.654 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T06:23:15.654 INFO:teuthology.orchestra.run.smithi203.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T06:23:15.654 INFO:teuthology.orchestra.run.smithi203.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T06:23:15.654 INFO:teuthology.orchestra.run.smithi203.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T06:23:15.654 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T06:23:15.654 INFO:teuthology.orchestra.run.smithi203.stdout:Installing dependencies: 2024-09-21T06:23:15.654 INFO:teuthology.orchestra.run.smithi203.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T06:23:15.654 INFO:teuthology.orchestra.run.smithi203.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T06:23:15.655 INFO:teuthology.orchestra.run.smithi203.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T06:23:15.655 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T06:23:15.655 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T06:23:15.655 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T06:23:15.655 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T06:23:15.655 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T06:23:15.655 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T06:23:15.655 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T06:23:15.655 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T06:23:15.655 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T06:23:15.655 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T06:23:15.655 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T06:23:15.655 INFO:teuthology.orchestra.run.smithi203.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T06:23:15.655 INFO:teuthology.orchestra.run.smithi203.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T06:23:15.656 INFO:teuthology.orchestra.run.smithi203.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T06:23:15.656 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T06:23:15.656 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T06:23:15.656 INFO:teuthology.orchestra.run.smithi203.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T06:23:15.656 INFO:teuthology.orchestra.run.smithi203.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T06:23:15.656 INFO:teuthology.orchestra.run.smithi203.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T06:23:15.656 INFO:teuthology.orchestra.run.smithi203.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T06:23:15.656 INFO:teuthology.orchestra.run.smithi203.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T06:23:15.656 INFO:teuthology.orchestra.run.smithi203.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T06:23:15.656 INFO:teuthology.orchestra.run.smithi203.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T06:23:15.656 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T06:23:15.656 INFO:teuthology.orchestra.run.smithi203.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T06:23:15.656 INFO:teuthology.orchestra.run.smithi203.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T06:23:15.656 INFO:teuthology.orchestra.run.smithi203.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T06:23:15.656 INFO:teuthology.orchestra.run.smithi203.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T06:23:15.657 INFO:teuthology.orchestra.run.smithi203.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T06:23:15.657 INFO:teuthology.orchestra.run.smithi203.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T06:23:15.657 INFO:teuthology.orchestra.run.smithi203.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T06:23:15.657 INFO:teuthology.orchestra.run.smithi203.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T06:23:15.657 INFO:teuthology.orchestra.run.smithi203.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T06:23:15.657 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T06:23:15.657 INFO:teuthology.orchestra.run.smithi203.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T06:23:15.657 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T06:23:15.657 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T06:23:15.657 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T06:23:15.657 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T06:23:15.657 INFO:teuthology.orchestra.run.smithi203.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T06:23:15.657 INFO:teuthology.orchestra.run.smithi203.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T06:23:15.657 INFO:teuthology.orchestra.run.smithi203.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T06:23:15.657 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T06:23:15.658 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T06:23:15.658 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T06:23:15.658 INFO:teuthology.orchestra.run.smithi203.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T06:23:15.658 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T06:23:15.658 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T06:23:15.658 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T06:23:15.658 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T06:23:15.658 INFO:teuthology.orchestra.run.smithi203.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T06:23:15.658 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T06:23:15.658 INFO:teuthology.orchestra.run.smithi203.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T06:23:15.658 INFO:teuthology.orchestra.run.smithi203.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T06:23:15.658 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T06:23:15.658 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T06:23:15.658 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T06:23:15.658 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T06:23:15.659 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T06:23:15.659 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T06:23:15.659 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T06:23:15.659 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T06:23:15.659 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T06:23:15.659 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T06:23:15.659 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T06:23:15.659 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T06:23:15.659 INFO:teuthology.orchestra.run.smithi203.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T06:23:15.659 INFO:teuthology.orchestra.run.smithi203.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T06:23:15.659 INFO:teuthology.orchestra.run.smithi203.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T06:23:15.659 INFO:teuthology.orchestra.run.smithi203.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T06:23:15.659 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T06:23:15.659 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T06:23:15.660 INFO:teuthology.orchestra.run.smithi203.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T06:23:15.660 INFO:teuthology.orchestra.run.smithi203.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T06:23:15.660 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T06:23:15.660 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T06:23:15.660 INFO:teuthology.orchestra.run.smithi203.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T06:23:15.660 INFO:teuthology.orchestra.run.smithi203.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T06:23:15.660 INFO:teuthology.orchestra.run.smithi203.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T06:23:15.660 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T06:23:15.660 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T06:23:15.660 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T06:23:15.660 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T06:23:15.660 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T06:23:15.660 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T06:23:15.660 INFO:teuthology.orchestra.run.smithi203.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T06:23:15.661 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T06:23:15.661 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T06:23:15.661 INFO:teuthology.orchestra.run.smithi203.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T06:23:15.661 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T06:23:15.661 INFO:teuthology.orchestra.run.smithi203.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T06:23:15.661 INFO:teuthology.orchestra.run.smithi203.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T06:23:15.661 INFO:teuthology.orchestra.run.smithi203.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T06:23:15.661 INFO:teuthology.orchestra.run.smithi203.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T06:23:15.661 INFO:teuthology.orchestra.run.smithi203.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T06:23:15.661 INFO:teuthology.orchestra.run.smithi203.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T06:23:15.661 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T06:23:15.661 INFO:teuthology.orchestra.run.smithi203.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T06:23:15.661 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T06:23:15.661 INFO:teuthology.orchestra.run.smithi203.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T06:23:15.662 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T06:23:15.662 INFO:teuthology.orchestra.run.smithi203.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T06:23:15.662 INFO:teuthology.orchestra.run.smithi203.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T06:23:15.662 INFO:teuthology.orchestra.run.smithi203.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T06:23:15.662 INFO:teuthology.orchestra.run.smithi203.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T06:23:15.662 INFO:teuthology.orchestra.run.smithi203.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T06:23:15.662 INFO:teuthology.orchestra.run.smithi203.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T06:23:15.662 INFO:teuthology.orchestra.run.smithi203.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T06:23:15.662 INFO:teuthology.orchestra.run.smithi203.stdout:Installing weak dependencies: 2024-09-21T06:23:15.662 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T06:23:15.662 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T06:23:15.662 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-09-21T06:23:15.662 INFO:teuthology.orchestra.run.smithi203.stdout:======================================================================================= 2024-09-21T06:23:15.662 INFO:teuthology.orchestra.run.smithi203.stdout:Install 149 Packages 2024-09-21T06:23:15.663 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrade 8 Packages 2024-09-21T06:23:15.663 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T06:23:15.663 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 228 M 2024-09-21T06:23:15.663 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-09-21T06:23:15.664 INFO:teuthology.orchestra.run.smithi067.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 354 kB/s | 80 kB 00:00 2024-09-21T06:23:15.764 INFO:teuthology.orchestra.run.smithi067.stdout:(41/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-09-21T06:23:15.789 INFO:teuthology.orchestra.run.smithi067.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 2.4 MB/s | 801 kB 00:00 2024-09-21T06:23:15.840 INFO:teuthology.orchestra.run.smithi067.stdout:(43/157): libquadmath-11.5.0-2.el9.x86_64.rpm 915 kB/s | 191 kB 00:00 2024-09-21T06:23:15.865 INFO:teuthology.orchestra.run.smithi067.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-21T06:23:15.924 INFO:teuthology.orchestra.run.smithi067.stdout:(45/157): python3-cryptography-36.0.1-4.el9.x86 7.7 MB/s | 1.2 MB 00:00 2024-09-21T06:23:15.949 INFO:teuthology.orchestra.run.smithi067.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-09-21T06:23:15.974 INFO:teuthology.orchestra.run.smithi067.stdout:(47/157): python3-pycparser-2.20-6.el9.noarch.r 1.0 MB/s | 135 kB 00:00 2024-09-21T06:23:15.999 INFO:teuthology.orchestra.run.smithi067.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-09-21T06:23:16.141 INFO:teuthology.orchestra.run.smithi067.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 754 kB/s | 107 kB 00:00 2024-09-21T06:23:16.160 INFO:teuthology.orchestra.run.smithi203.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 48 kB/s | 6.6 kB 00:00 2024-09-21T06:23:16.175 INFO:teuthology.orchestra.run.smithi067.stdout:(50/157): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-09-21T06:23:16.200 INFO:teuthology.orchestra.run.smithi067.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 370 kB/s | 22 kB 00:00 2024-09-21T06:23:16.234 INFO:teuthology.orchestra.run.smithi067.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 3.5 MB/s | 943 kB 00:00 2024-09-21T06:23:16.259 INFO:teuthology.orchestra.run.smithi067.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 560 kB/s | 33 kB 00:00 2024-09-21T06:23:16.284 INFO:teuthology.orchestra.run.smithi067.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-09-21T06:23:16.310 INFO:teuthology.orchestra.run.smithi067.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-09-21T06:23:16.335 INFO:teuthology.orchestra.run.smithi067.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-21T06:23:16.360 INFO:teuthology.orchestra.run.smithi067.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 3.3 MB/s | 171 kB 00:00 2024-09-21T06:23:16.419 INFO:teuthology.orchestra.run.smithi067.stdout:(58/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-09-21T06:23:16.444 INFO:teuthology.orchestra.run.smithi067.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 542 kB/s | 45 kB 00:00 2024-09-21T06:23:16.469 INFO:teuthology.orchestra.run.smithi067.stdout:(60/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-09-21T06:23:16.503 INFO:teuthology.orchestra.run.smithi067.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 662 kB 00:00 2024-09-21T06:23:16.511 INFO:teuthology.orchestra.run.smithi203.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 2.4 MB/s | 873 kB 00:00 2024-09-21T06:23:16.520 INFO:teuthology.orchestra.run.smithi067.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 487 kB/s | 24 kB 00:00 2024-09-21T06:23:16.553 INFO:teuthology.orchestra.run.smithi067.stdout:(63/157): libtool-2.4.6-46.el9.x86_64.rpm 5.2 MB/s | 581 kB 00:00 2024-09-21T06:23:16.570 INFO:teuthology.orchestra.run.smithi067.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 709 kB/s | 47 kB 00:00 2024-09-21T06:23:16.594 INFO:teuthology.orchestra.run.smithi203.stdout:(3/157): ceph-base-19.3.0-5120.g18432218.el9.x8 9.7 MB/s | 5.5 MB 00:00 2024-09-21T06:23:16.628 INFO:teuthology.orchestra.run.smithi203.stdout:(4/157): ceph-immutable-object-cache-19.3.0-512 1.2 MB/s | 148 kB 00:00 2024-09-21T06:23:16.629 INFO:teuthology.orchestra.run.smithi067.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-21T06:23:16.646 INFO:teuthology.orchestra.run.smithi067.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 511 kB/s | 38 kB 00:00 2024-09-21T06:23:16.671 INFO:teuthology.orchestra.run.smithi067.stdout:(67/157): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-21T06:23:16.712 INFO:teuthology.orchestra.run.smithi203.stdout:(5/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 20 MB/s | 2.3 MB 00:00 2024-09-21T06:23:16.763 INFO:teuthology.orchestra.run.smithi067.stdout:(68/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-09-21T06:23:16.805 INFO:teuthology.orchestra.run.smithi067.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-21T06:23:16.838 INFO:teuthology.orchestra.run.smithi067.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 12 MB/s | 910 kB 00:00 2024-09-21T06:23:16.864 INFO:teuthology.orchestra.run.smithi067.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-21T06:23:16.939 INFO:teuthology.orchestra.run.smithi067.stdout:(72/157): openblas-openmp-0.3.26-2.el9.x86_64.r 16 MB/s | 4.9 MB 00:00 2024-09-21T06:23:16.981 INFO:teuthology.orchestra.run.smithi067.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 8.6 MB/s | 1.0 MB 00:00 2024-09-21T06:23:17.006 INFO:teuthology.orchestra.run.smithi067.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-21T06:23:17.079 INFO:teuthology.orchestra.run.smithi203.stdout:(6/157): ceph-common-19.3.0-5120.g18432218.el9. 19 MB/s | 20 MB 00:01 2024-09-21T06:23:17.098 INFO:teuthology.orchestra.run.smithi067.stdout:(75/157): postgresql-server-13.16-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-09-21T06:23:17.121 INFO:teuthology.orchestra.run.smithi203.stdout:(7/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 3.1 MB/s | 1.5 MB 00:00 2024-09-21T06:23:17.123 INFO:teuthology.orchestra.run.smithi067.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 524 kB/s | 61 kB 00:00 2024-09-21T06:23:17.149 INFO:teuthology.orchestra.run.smithi067.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-09-21T06:23:17.174 INFO:teuthology.orchestra.run.smithi067.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-21T06:23:17.196 INFO:teuthology.orchestra.run.smithi203.stdout:(8/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 10 MB/s | 5.0 MB 00:00 2024-09-21T06:23:17.199 INFO:teuthology.orchestra.run.smithi067.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-09-21T06:23:17.224 INFO:teuthology.orchestra.run.smithi067.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-21T06:23:17.242 INFO:teuthology.orchestra.run.smithi067.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-09-21T06:23:17.246 INFO:teuthology.orchestra.run.smithi203.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 504 kB/s | 25 kB 00:00 2024-09-21T06:23:17.267 INFO:teuthology.orchestra.run.smithi067.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 826 kB/s | 35 kB 00:00 2024-09-21T06:23:17.367 INFO:teuthology.orchestra.run.smithi067.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-21T06:23:17.392 INFO:teuthology.orchestra.run.smithi067.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 250 kB 00:00 2024-09-21T06:23:17.418 INFO:teuthology.orchestra.run.smithi067.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-21T06:23:17.451 INFO:teuthology.orchestra.run.smithi067.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-21T06:23:17.477 INFO:teuthology.orchestra.run.smithi067.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 723 kB/s | 42 kB 00:00 2024-09-21T06:23:17.502 INFO:teuthology.orchestra.run.smithi067.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-21T06:23:17.507 INFO:teuthology.orchestra.run.smithi100.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T06:23:17.508 INFO:teuthology.orchestra.run.smithi100.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T06:23:17.521 INFO:teuthology.orchestra.run.smithi100.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T06:23:17.522 INFO:teuthology.orchestra.run.smithi100.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T06:23:17.523 INFO:teuthology.orchestra.run.smithi100.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T06:23:17.524 INFO:teuthology.orchestra.run.smithi100.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T06:23:17.526 INFO:teuthology.orchestra.run.smithi100.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T06:23:17.527 INFO:teuthology.orchestra.run.smithi067.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-21T06:23:17.529 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T06:23:17.530 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T06:23:17.552 INFO:teuthology.orchestra.run.smithi067.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-21T06:23:17.578 INFO:teuthology.orchestra.run.smithi067.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-21T06:23:17.592 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-09-21T06:23:17.603 INFO:teuthology.orchestra.run.smithi067.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-21T06:23:17.609 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================= 2024-09-21T06:23:17.610 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-09-21T06:23:17.610 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================= 2024-09-21T06:23:17.610 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-09-21T06:23:17.610 INFO:teuthology.orchestra.run.smithi100.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T06:23:17.610 INFO:teuthology.orchestra.run.smithi100.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T06:23:17.610 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T06:23:17.610 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T06:23:17.610 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T06:23:17.610 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T06:23:17.610 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T06:23:17.610 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T06:23:17.610 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T06:23:17.611 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T06:23:17.611 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T06:23:17.611 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T06:23:17.611 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T06:23:17.611 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T06:23:17.611 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T06:23:17.611 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T06:23:17.611 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T06:23:17.611 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T06:23:17.611 INFO:teuthology.orchestra.run.smithi100.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T06:23:17.611 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T06:23:17.611 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T06:23:17.611 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T06:23:17.611 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T06:23:17.612 INFO:teuthology.orchestra.run.smithi100.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T06:23:17.612 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T06:23:17.612 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T06:23:17.612 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T06:23:17.612 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T06:23:17.612 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T06:23:17.612 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T06:23:17.612 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T06:23:17.612 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T06:23:17.612 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T06:23:17.612 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T06:23:17.612 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T06:23:17.612 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T06:23:17.612 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T06:23:17.612 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrading: 2024-09-21T06:23:17.613 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T06:23:17.613 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T06:23:17.613 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T06:23:17.613 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T06:23:17.613 INFO:teuthology.orchestra.run.smithi100.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T06:23:17.613 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T06:23:17.613 INFO:teuthology.orchestra.run.smithi100.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T06:23:17.613 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T06:23:17.613 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-09-21T06:23:17.613 INFO:teuthology.orchestra.run.smithi100.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T06:23:17.613 INFO:teuthology.orchestra.run.smithi100.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T06:23:17.613 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T06:23:17.613 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T06:23:17.613 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T06:23:17.613 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T06:23:17.614 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T06:23:17.614 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T06:23:17.614 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T06:23:17.614 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T06:23:17.614 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T06:23:17.614 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T06:23:17.614 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T06:23:17.614 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T06:23:17.614 INFO:teuthology.orchestra.run.smithi100.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T06:23:17.614 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T06:23:17.614 INFO:teuthology.orchestra.run.smithi100.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T06:23:17.614 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T06:23:17.614 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T06:23:17.614 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T06:23:17.615 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T06:23:17.615 INFO:teuthology.orchestra.run.smithi100.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T06:23:17.615 INFO:teuthology.orchestra.run.smithi100.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T06:23:17.615 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T06:23:17.615 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T06:23:17.615 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T06:23:17.615 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T06:23:17.615 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T06:23:17.615 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T06:23:17.615 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T06:23:17.615 INFO:teuthology.orchestra.run.smithi100.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T06:23:17.615 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T06:23:17.615 INFO:teuthology.orchestra.run.smithi100.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T06:23:17.615 INFO:teuthology.orchestra.run.smithi100.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T06:23:17.615 INFO:teuthology.orchestra.run.smithi100.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T06:23:17.616 INFO:teuthology.orchestra.run.smithi100.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T06:23:17.616 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T06:23:17.616 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T06:23:17.616 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T06:23:17.616 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T06:23:17.616 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T06:23:17.616 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T06:23:17.616 INFO:teuthology.orchestra.run.smithi100.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T06:23:17.616 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T06:23:17.616 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T06:23:17.616 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T06:23:17.616 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T06:23:17.616 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T06:23:17.616 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T06:23:17.617 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T06:23:17.617 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T06:23:17.617 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T06:23:17.617 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T06:23:17.617 INFO:teuthology.orchestra.run.smithi100.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T06:23:17.617 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T06:23:17.617 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T06:23:17.617 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T06:23:17.617 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T06:23:17.617 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T06:23:17.617 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T06:23:17.617 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T06:23:17.617 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T06:23:17.617 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T06:23:17.617 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T06:23:17.618 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T06:23:17.618 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T06:23:17.618 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T06:23:17.618 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T06:23:17.618 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T06:23:17.618 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T06:23:17.618 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T06:23:17.618 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T06:23:17.618 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T06:23:17.618 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T06:23:17.618 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T06:23:17.618 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T06:23:17.618 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T06:23:17.618 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T06:23:17.619 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T06:23:17.619 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T06:23:17.619 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T06:23:17.619 INFO:teuthology.orchestra.run.smithi100.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T06:23:17.619 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T06:23:17.619 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T06:23:17.619 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T06:23:17.619 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T06:23:17.619 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T06:23:17.619 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T06:23:17.619 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T06:23:17.619 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T06:23:17.619 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T06:23:17.619 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T06:23:17.620 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T06:23:17.620 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T06:23:17.620 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T06:23:17.620 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T06:23:17.620 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T06:23:17.620 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T06:23:17.620 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T06:23:17.620 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T06:23:17.620 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T06:23:17.620 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T06:23:17.621 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T06:23:17.621 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T06:23:17.621 INFO:teuthology.orchestra.run.smithi100.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T06:23:17.621 INFO:teuthology.orchestra.run.smithi100.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T06:23:17.621 INFO:teuthology.orchestra.run.smithi100.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T06:23:17.621 INFO:teuthology.orchestra.run.smithi100.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T06:23:17.621 INFO:teuthology.orchestra.run.smithi100.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T06:23:17.621 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T06:23:17.621 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T06:23:17.621 INFO:teuthology.orchestra.run.smithi100.stdout:Installing weak dependencies: 2024-09-21T06:23:17.621 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T06:23:17.622 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-21T06:23:17.622 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-09-21T06:23:17.622 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================= 2024-09-21T06:23:17.622 INFO:teuthology.orchestra.run.smithi100.stdout:Install 149 Packages 2024-09-21T06:23:17.622 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrade 8 Packages 2024-09-21T06:23:17.622 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-21T06:23:17.623 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 228 M 2024-09-21T06:23:17.623 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-09-21T06:23:17.678 INFO:teuthology.orchestra.run.smithi067.stdout:(93/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-21T06:23:17.703 INFO:teuthology.orchestra.run.smithi067.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-09-21T06:23:17.728 INFO:teuthology.orchestra.run.smithi067.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 885 kB/s | 44 kB 00:00 2024-09-21T06:23:17.738 INFO:teuthology.orchestra.run.smithi203.stdout:(10/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 27 MB/s | 18 MB 00:00 2024-09-21T06:23:17.754 INFO:teuthology.orchestra.run.smithi067.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-21T06:23:17.771 INFO:teuthology.orchestra.run.smithi067.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 858 kB/s | 36 kB 00:00 2024-09-21T06:23:17.788 INFO:teuthology.orchestra.run.smithi067.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-21T06:23:17.788 INFO:teuthology.orchestra.run.smithi203.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 4.4 MB/s | 226 kB 00:00 2024-09-21T06:23:17.821 INFO:teuthology.orchestra.run.smithi067.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-21T06:23:17.838 INFO:teuthology.orchestra.run.smithi067.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T06:23:17.855 INFO:teuthology.orchestra.run.smithi203.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 405 kB/s | 27 kB 00:00 2024-09-21T06:23:17.863 INFO:teuthology.orchestra.run.smithi067.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 2.3 MB/s | 96 kB 00:00 2024-09-21T06:23:17.880 INFO:teuthology.orchestra.run.smithi067.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-21T06:23:17.897 INFO:teuthology.orchestra.run.smithi067.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-09-21T06:23:17.914 INFO:teuthology.orchestra.run.smithi203.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 546 kB/s | 32 kB 00:00 2024-09-21T06:23:17.939 INFO:teuthology.orchestra.run.smithi067.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 383 kB/s | 22 kB 00:00 2024-09-21T06:23:17.989 INFO:teuthology.orchestra.run.smithi067.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-21T06:23:18.014 INFO:teuthology.orchestra.run.smithi203.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 7.5 MB/s | 766 kB 00:00 2024-09-21T06:23:18.023 INFO:teuthology.orchestra.run.smithi067.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-21T06:23:18.048 INFO:teuthology.orchestra.run.smithi067.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 783 kB/s | 19 kB 00:00 2024-09-21T06:23:18.081 INFO:teuthology.orchestra.run.smithi203.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 2.3 MB/s | 158 kB 00:00 2024-09-21T06:23:18.149 INFO:teuthology.orchestra.run.smithi203.stdout:(16/157): librados-devel-19.3.0-5120.g18432218. 1.9 MB/s | 127 kB 00:00 2024-09-21T06:23:18.224 INFO:teuthology.orchestra.run.smithi067.stdout:(108/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-09-21T06:23:18.232 INFO:teuthology.orchestra.run.smithi203.stdout:(17/157): libradosstriper1-19.3.0-5120.g1843221 6.0 MB/s | 507 kB 00:00 2024-09-21T06:23:18.265 INFO:teuthology.orchestra.run.smithi067.stdout:(109/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.3 MB/s | 863 kB 00:00 2024-09-21T06:23:18.299 INFO:teuthology.orchestra.run.smithi067.stdout:(110/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 347 kB/s | 26 kB 00:00 2024-09-21T06:23:18.324 INFO:teuthology.orchestra.run.smithi067.stdout:(111/157): liboath-2.6.7-2.el9.x86_64.rpm 859 kB/s | 49 kB 00:00 2024-09-21T06:23:18.349 INFO:teuthology.orchestra.run.smithi203.stdout:(18/157): ceph-radosgw-19.3.0-5120.g18432218.el 8.5 MB/s | 10 MB 00:01 2024-09-21T06:23:18.349 INFO:teuthology.orchestra.run.smithi067.stdout:(112/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.4 MB/s | 67 kB 00:00 2024-09-21T06:23:18.355 INFO:teuthology.orchestra.run.smithi100.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-09-21T06:23:18.399 INFO:teuthology.orchestra.run.smithi203.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 907 kB/s | 45 kB 00:00 2024-09-21T06:23:18.450 INFO:teuthology.orchestra.run.smithi067.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-09-21T06:23:18.492 INFO:teuthology.orchestra.run.smithi067.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.8 MB/s | 839 kB 00:00 2024-09-21T06:23:18.500 INFO:teuthology.orchestra.run.smithi203.stdout:(20/157): librgw2-19.3.0-5120.g18432218.el9.x86 20 MB/s | 5.3 MB 00:00 2024-09-21T06:23:18.517 INFO:teuthology.orchestra.run.smithi067.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T06:23:18.525 INFO:teuthology.orchestra.run.smithi203.stdout:(21/157): python3-ceph-common-19.3.0-5120.g1843 1.2 MB/s | 148 kB 00:00 2024-09-21T06:23:18.542 INFO:teuthology.orchestra.run.smithi067.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T06:23:18.550 INFO:teuthology.orchestra.run.smithi203.stdout:(22/157): python3-cephfs-19.3.0-5120.g18432218. 3.2 MB/s | 161 kB 00:00 2024-09-21T06:23:18.567 INFO:teuthology.orchestra.run.smithi067.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 564 kB/s | 14 kB 00:00 2024-09-21T06:23:18.576 INFO:teuthology.orchestra.run.smithi203.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 6.3 MB/s | 321 kB 00:00 2024-09-21T06:23:18.601 INFO:teuthology.orchestra.run.smithi203.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 6.0 MB/s | 307 kB 00:00 2024-09-21T06:23:18.601 INFO:teuthology.orchestra.run.smithi067.stdout:(118/157): python3-asyncssh-2.13.2-5.el9.noarch 3.5 MB/s | 548 kB 00:00 2024-09-21T06:23:18.627 INFO:teuthology.orchestra.run.smithi203.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.9 MB/s | 99 kB 00:00 2024-09-21T06:23:18.627 INFO:teuthology.orchestra.run.smithi067.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-21T06:23:18.652 INFO:teuthology.orchestra.run.smithi203.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T06:23:18.668 INFO:teuthology.orchestra.run.smithi067.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-21T06:23:18.693 INFO:teuthology.orchestra.run.smithi067.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-21T06:23:18.719 INFO:teuthology.orchestra.run.smithi067.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-21T06:23:18.719 INFO:teuthology.orchestra.run.smithi203.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.5 MB/s | 170 kB 00:00 2024-09-21T06:23:18.746 INFO:teuthology.orchestra.run.smithi100.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 2.2 MB/s | 873 kB 00:00 2024-09-21T06:23:18.752 INFO:teuthology.orchestra.run.smithi067.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-21T06:23:18.778 INFO:teuthology.orchestra.run.smithi067.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-21T06:23:18.822 INFO:teuthology.orchestra.run.smithi100.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 1.9 MB/s | 148 kB 00:00 2024-09-21T06:23:18.845 INFO:teuthology.orchestra.run.smithi067.stdout:(125/157): libarrow-9.0.0-11.el9.x86_64.rpm 5.5 MB/s | 4.4 MB 00:00 2024-09-21T06:23:18.870 INFO:teuthology.orchestra.run.smithi067.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-09-21T06:23:18.895 INFO:teuthology.orchestra.run.smithi067.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-09-21T06:23:18.921 INFO:teuthology.orchestra.run.smithi067.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-09-21T06:23:18.946 INFO:teuthology.orchestra.run.smithi067.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-09-21T06:23:18.971 INFO:teuthology.orchestra.run.smithi067.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T06:23:18.996 INFO:teuthology.orchestra.run.smithi067.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T06:23:19.021 INFO:teuthology.orchestra.run.smithi067.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T06:23:19.055 INFO:teuthology.orchestra.run.smithi067.stdout:(133/157): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-21T06:23:19.081 INFO:teuthology.orchestra.run.smithi067.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 663 kB/s | 16 kB 00:00 2024-09-21T06:23:19.097 INFO:teuthology.orchestra.run.smithi100.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 5.6 MB/s | 5.5 MB 00:00 2024-09-21T06:23:19.106 INFO:teuthology.orchestra.run.smithi067.stdout:(135/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.6 MB/s | 90 kB 00:00 2024-09-21T06:23:19.136 INFO:teuthology.orchestra.run.smithi203.stdout:(28/157): ceph-test-19.3.0-5120.g18432218.el9.x 21 MB/s | 40 MB 00:01 2024-09-21T06:23:19.139 INFO:teuthology.orchestra.run.smithi067.stdout:(136/157): python3-kubernetes-26.1.0-2.el9.noar 4.7 MB/s | 1.0 MB 00:00 2024-09-21T06:23:19.161 INFO:teuthology.orchestra.run.smithi203.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 77 kB/s | 34 kB 00:00 2024-09-21T06:23:19.164 INFO:teuthology.orchestra.run.smithi067.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 484 kB/s | 28 kB 00:00 2024-09-21T06:23:19.186 INFO:teuthology.orchestra.run.smithi203.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T06:23:19.206 INFO:teuthology.orchestra.run.smithi067.stdout:(138/157): python3-grpcio-1.46.7-10.el9.x86_64. 4.0 MB/s | 2.0 MB 00:00 2024-09-21T06:23:19.230 INFO:teuthology.orchestra.run.smithi100.stdout:(5/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 5.6 MB/s | 2.3 MB 00:00 2024-09-21T06:23:19.232 INFO:teuthology.orchestra.run.smithi067.stdout:(139/157): python3-repoze-lru-0.7-16.el9.noarch 334 kB/s | 31 kB 00:00 2024-09-21T06:23:19.257 INFO:teuthology.orchestra.run.smithi067.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-21T06:23:19.264 INFO:teuthology.orchestra.run.smithi100.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 9.1 MB/s | 1.5 MB 00:00 2024-09-21T06:23:19.270 INFO:teuthology.orchestra.run.smithi203.stdout:(31/157): rbd-mirror-19.3.0-5120.g18432218.el9. 4.8 MB/s | 3.1 MB 00:00 2024-09-21T06:23:19.282 INFO:teuthology.orchestra.run.smithi067.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-21T06:23:19.308 INFO:teuthology.orchestra.run.smithi067.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-21T06:23:19.333 INFO:teuthology.orchestra.run.smithi067.stdout:(143/157): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2024-09-21T06:23:19.358 INFO:teuthology.orchestra.run.smithi067.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-21T06:23:19.384 INFO:teuthology.orchestra.run.smithi067.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-21T06:23:19.401 INFO:teuthology.orchestra.run.smithi067.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 476 kB/s | 20 kB 00:00 2024-09-21T06:23:19.426 INFO:teuthology.orchestra.run.smithi067.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-21T06:23:19.446 INFO:teuthology.orchestra.run.smithi203.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 1.6 MB/s | 292 kB 00:00 2024-09-21T06:23:19.451 INFO:teuthology.orchestra.run.smithi067.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-09-21T06:23:19.501 INFO:teuthology.orchestra.run.smithi067.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-21T06:23:19.529 INFO:teuthology.orchestra.run.smithi203.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 592 kB/s | 49 kB 00:00 2024-09-21T06:23:19.621 INFO:teuthology.orchestra.run.smithi203.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 180 kB/s | 16 kB 00:00 2024-09-21T06:23:19.685 INFO:teuthology.orchestra.run.smithi067.stdout:(150/157): NetworkManager-1.51.0-1.el9.x86_64.r 13 MB/s | 2.3 MB 00:00 2024-09-21T06:23:19.738 INFO:teuthology.orchestra.run.smithi203.stdout:(35/157): ceph-mgr-diskprediction-local-19.3.0- 13 MB/s | 7.4 MB 00:00 2024-09-21T06:23:19.772 INFO:teuthology.orchestra.run.smithi203.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 1.9 MB/s | 285 kB 00:00 2024-09-21T06:23:19.814 INFO:teuthology.orchestra.run.smithi203.stdout:(37/157): cephadm-19.3.0-5120.g18432218.el9.noa 10 MB/s | 784 kB 00:00 2024-09-21T06:23:19.818 INFO:teuthology.orchestra.run.smithi067.stdout:(151/157): NetworkManager-libnm-1.51.0-1.el9.x8 14 MB/s | 1.8 MB 00:00 2024-09-21T06:23:19.839 INFO:teuthology.orchestra.run.smithi203.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 275 kB/s | 18 kB 00:00 2024-09-21T06:23:19.885 INFO:teuthology.orchestra.run.smithi067.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 616 kB/s | 41 kB 00:00 2024-09-21T06:23:19.944 INFO:teuthology.orchestra.run.smithi067.stdout:(153/157): librados2-19.3.0-5120.g18432218.el9. 6.5 MB/s | 3.4 MB 00:00 2024-09-21T06:23:19.969 INFO:teuthology.orchestra.run.smithi067.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 2.9 MB/s | 249 kB 00:00 2024-09-21T06:23:19.998 INFO:teuthology.orchestra.run.smithi203.stdout:(39/157): ceph-mgr-dashboard-19.3.0-5120.g18432 17 MB/s | 14 MB 00:00 2024-09-21T06:23:20.039 INFO:teuthology.orchestra.run.smithi100.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 6.1 MB/s | 5.0 MB 00:00 2024-09-21T06:23:20.086 INFO:teuthology.orchestra.run.smithi067.stdout:(155/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 21 MB/s | 2.4 MB 00:00 2024-09-21T06:23:20.178 INFO:teuthology.orchestra.run.smithi067.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 5.8 MB/s | 1.4 MB 00:00 2024-09-21T06:23:20.328 INFO:teuthology.orchestra.run.smithi067.stdout:(157/157): librbd1-19.3.0-5120.g18432218.el9.x8 3.6 MB/s | 3.2 MB 00:00 2024-09-21T06:23:20.331 INFO:teuthology.orchestra.run.smithi100.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 9.0 MB/s | 20 MB 00:02 2024-09-21T06:23:20.334 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-09-21T06:23:20.334 INFO:teuthology.orchestra.run.smithi067.stdout:Total 26 MB/s | 228 MB 00:08 2024-09-21T06:23:20.373 INFO:teuthology.orchestra.run.smithi100.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 607 kB/s | 25 kB 00:00 2024-09-21T06:23:20.374 INFO:teuthology.orchestra.run.smithi203.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 143 kB/s | 80 kB 00:00 2024-09-21T06:23:20.549 INFO:teuthology.orchestra.run.smithi203.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 347 kB/s | 191 kB 00:00 2024-09-21T06:23:20.657 INFO:teuthology.orchestra.run.smithi203.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 980 kB/s | 801 kB 00:00 2024-09-21T06:23:20.690 INFO:teuthology.orchestra.run.smithi100.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 16 MB/s | 10 MB 00:00 2024-09-21T06:23:20.740 INFO:teuthology.orchestra.run.smithi100.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 4.4 MB/s | 226 kB 00:00 2024-09-21T06:23:20.749 INFO:teuthology.orchestra.run.smithi203.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 674 kB/s | 253 kB 00:00 2024-09-21T06:23:20.775 INFO:teuthology.orchestra.run.smithi203.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 910 kB/s | 106 kB 00:00 2024-09-21T06:23:20.782 INFO:teuthology.orchestra.run.smithi100.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 651 kB/s | 27 kB 00:00 2024-09-21T06:23:20.841 INFO:teuthology.orchestra.run.smithi100.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 543 kB/s | 32 kB 00:00 2024-09-21T06:23:20.866 INFO:teuthology.orchestra.run.smithi203.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-09-21T06:23:20.892 INFO:teuthology.orchestra.run.smithi203.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-09-21T06:23:20.908 INFO:teuthology.orchestra.run.smithi100.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 11 MB/s | 766 kB 00:00 2024-09-21T06:23:20.958 INFO:teuthology.orchestra.run.smithi100.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 3.1 MB/s | 158 kB 00:00 2024-09-21T06:23:20.992 INFO:teuthology.orchestra.run.smithi203.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 1.7 MB/s | 215 kB 00:00 2024-09-21T06:23:21.008 INFO:teuthology.orchestra.run.smithi100.stdout:(16/157): librados-devel-19.3.0-5120.g18432218. 2.5 MB/s | 127 kB 00:00 2024-09-21T06:23:21.109 INFO:teuthology.orchestra.run.smithi203.stdout:(48/157): python3-cryptography-36.0.1-4.el9.x86 2.2 MB/s | 1.2 MB 00:00 2024-09-21T06:23:21.176 INFO:teuthology.orchestra.run.smithi100.stdout:(17/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 9.3 MB/s | 18 MB 00:01 2024-09-21T06:23:21.176 INFO:teuthology.orchestra.run.smithi203.stdout:(49/157): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-09-21T06:23:21.209 INFO:teuthology.orchestra.run.smithi100.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 2.5 MB/s | 507 kB 00:00 2024-09-21T06:23:21.226 INFO:teuthology.orchestra.run.smithi203.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 434 kB/s | 22 kB 00:00 2024-09-21T06:23:21.259 INFO:teuthology.orchestra.run.smithi100.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 900 kB/s | 45 kB 00:00 2024-09-21T06:23:21.277 INFO:teuthology.orchestra.run.smithi203.stdout:(51/157): boost-program-options-1.75.0-8.el9.x8 639 kB/s | 107 kB 00:00 2024-09-21T06:23:21.310 INFO:teuthology.orchestra.run.smithi100.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 2.9 MB/s | 148 kB 00:00 2024-09-21T06:23:21.319 INFO:teuthology.orchestra.run.smithi203.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 943 kB 00:00 2024-09-21T06:23:21.344 INFO:teuthology.orchestra.run.smithi203.stdout:(53/157): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-21T06:23:21.360 INFO:teuthology.orchestra.run.smithi100.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 3.2 MB/s | 161 kB 00:00 2024-09-21T06:23:21.361 INFO:teuthology.orchestra.run.smithi203.stdout:(54/157): flexiblas-3.0.4-8.el9.x86_64.rpm 390 kB/s | 33 kB 00:00 2024-09-21T06:23:21.403 INFO:teuthology.orchestra.run.smithi203.stdout:(55/157): libaio-devel-0.3.111-13.el9.x86_64.rp 269 kB/s | 11 kB 00:00 2024-09-21T06:23:21.420 INFO:teuthology.orchestra.run.smithi203.stdout:(56/157): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-09-21T06:23:21.427 INFO:teuthology.orchestra.run.smithi100.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 4.7 MB/s | 321 kB 00:00 2024-09-21T06:23:21.462 INFO:teuthology.orchestra.run.smithi203.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.8 MB/s | 171 kB 00:00 2024-09-21T06:23:21.486 INFO:teuthology.orchestra.run.smithi100.stdout:(23/157): python3-rbd-19.3.0-5120.g18432218.el9 5.1 MB/s | 307 kB 00:00 2024-09-21T06:23:21.504 INFO:teuthology.orchestra.run.smithi203.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-09-21T06:23:21.529 INFO:teuthology.orchestra.run.smithi203.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 675 kB/s | 45 kB 00:00 2024-09-21T06:23:21.536 INFO:teuthology.orchestra.run.smithi100.stdout:(24/157): python3-rgw-19.3.0-5120.g18432218.el9 2.0 MB/s | 99 kB 00:00 2024-09-21T06:23:21.611 INFO:teuthology.orchestra.run.smithi100.stdout:(25/157): librgw2-19.3.0-5120.g18432218.el9.x86 12 MB/s | 5.3 MB 00:00 2024-09-21T06:23:21.625 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-09-21T06:23:21.629 INFO:teuthology.orchestra.run.smithi203.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-09-21T06:23:21.636 INFO:teuthology.orchestra.run.smithi100.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 863 kB/s | 86 kB 00:00 2024-09-21T06:23:21.655 INFO:teuthology.orchestra.run.smithi203.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 4.5 MB/s | 581 kB 00:00 2024-09-21T06:23:21.695 INFO:teuthology.orchestra.run.smithi100.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.9 MB/s | 170 kB 00:00 2024-09-21T06:23:21.705 INFO:teuthology.orchestra.run.smithi203.stdout:(62/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.7 MB/s | 3.0 MB 00:00 2024-09-21T06:23:21.730 INFO:teuthology.orchestra.run.smithi203.stdout:(63/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 243 kB/s | 24 kB 00:00 2024-09-21T06:23:21.745 INFO:teuthology.orchestra.run.smithi100.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 685 kB/s | 34 kB 00:00 2024-09-21T06:23:21.756 INFO:teuthology.orchestra.run.smithi203.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 472 kB/s | 47 kB 00:00 2024-09-21T06:23:21.790 INFO:teuthology.orchestra.run.smithi203.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-21T06:23:21.795 INFO:teuthology.orchestra.run.smithi100.stdout:(29/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T06:23:21.797 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-09-21T06:23:21.797 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-09-21T06:23:21.815 INFO:teuthology.orchestra.run.smithi203.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 655 kB/s | 38 kB 00:00 2024-09-21T06:23:21.849 INFO:teuthology.orchestra.run.smithi203.stdout:(67/157): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-21T06:23:21.888 INFO:teuthology.orchestra.run.smithi100.stdout:(30/157): rbd-mirror-19.3.0-5120.g18432218.el9. 11 MB/s | 3.1 MB 00:00 2024-09-21T06:23:22.065 INFO:teuthology.orchestra.run.smithi203.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-09-21T06:23:22.216 INFO:teuthology.orchestra.run.smithi203.stdout:(69/157): openblas-openmp-0.3.26-2.el9.x86_64.r 12 MB/s | 4.9 MB 00:00 2024-09-21T06:23:22.258 INFO:teuthology.orchestra.run.smithi203.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 4.6 MB/s | 910 kB 00:00 2024-09-21T06:23:22.283 INFO:teuthology.orchestra.run.smithi203.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-21T06:23:22.342 INFO:teuthology.orchestra.run.smithi203.stdout:(72/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 8.4 MB/s | 4.4 MB 00:00 2024-09-21T06:23:22.400 INFO:teuthology.orchestra.run.smithi203.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-21T06:23:22.421 INFO:teuthology.orchestra.run.smithi100.stdout:(31/157): ceph-mgr-dashboard-19.3.0-5120.g18432 23 MB/s | 14 MB 00:00 2024-09-21T06:23:22.442 INFO:teuthology.orchestra.run.smithi203.stdout:(74/157): protobuf-3.14.0-13.el9.x86_64.rpm 6.3 MB/s | 1.0 MB 00:00 2024-09-21T06:23:22.480 INFO:teuthology.orchestra.run.smithi100.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 4.9 MB/s | 292 kB 00:00 2024-09-21T06:23:22.493 INFO:teuthology.orchestra.run.smithi203.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-21T06:23:22.530 INFO:teuthology.orchestra.run.smithi100.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 989 kB/s | 49 kB 00:00 2024-09-21T06:23:22.551 INFO:teuthology.orchestra.run.smithi203.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 4.1 MB/s | 245 kB 00:00 2024-09-21T06:23:22.580 INFO:teuthology.orchestra.run.smithi100.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 328 kB/s | 16 kB 00:00 2024-09-21T06:23:22.610 INFO:teuthology.orchestra.run.smithi203.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-09-21T06:23:22.668 INFO:teuthology.orchestra.run.smithi203.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 438 kB/s | 26 kB 00:00 2024-09-21T06:23:22.672 INFO:teuthology.orchestra.run.smithi100.stdout:(35/157): ceph-mgr-diskprediction-local-19.3.0- 9.4 MB/s | 7.4 MB 00:00 2024-09-21T06:23:22.697 INFO:teuthology.orchestra.run.smithi100.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 2.4 MB/s | 285 kB 00:00 2024-09-21T06:23:22.752 INFO:teuthology.orchestra.run.smithi203.stdout:(79/157): postgresql-server-13.16-1.el9.x86_64. 12 MB/s | 5.8 MB 00:00 2024-09-21T06:23:22.764 INFO:teuthology.orchestra.run.smithi100.stdout:(37/157): cephfs-top-19.3.0-5120.g18432218.el9. 275 kB/s | 18 kB 00:00 2024-09-21T06:23:22.777 INFO:teuthology.orchestra.run.smithi203.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 176 kB/s | 19 kB 00:00 2024-09-21T06:23:22.803 INFO:teuthology.orchestra.run.smithi203.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-21T06:23:22.831 INFO:teuthology.orchestra.run.smithi100.stdout:(38/157): cephadm-19.3.0-5120.g18432218.el9.noa 4.8 MB/s | 784 kB 00:00 2024-09-21T06:23:22.873 INFO:teuthology.orchestra.run.smithi100.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 735 kB/s | 80 kB 00:00 2024-09-21T06:23:22.886 INFO:teuthology.orchestra.run.smithi203.stdout:(82/157): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-09-21T06:23:22.912 INFO:teuthology.orchestra.run.smithi203.stdout:(83/157): python3-markupsafe-1.1.1-12.el9.x86_6 258 kB/s | 35 kB 00:00 2024-09-21T06:23:22.940 INFO:teuthology.orchestra.run.smithi100.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-21T06:23:22.945 INFO:teuthology.orchestra.run.smithi203.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-21T06:23:22.971 INFO:teuthology.orchestra.run.smithi203.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-21T06:23:22.982 INFO:teuthology.orchestra.run.smithi100.stdout:(41/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.9 MB/s | 253 kB 00:00 2024-09-21T06:23:22.996 INFO:teuthology.orchestra.run.smithi203.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-21T06:23:23.030 INFO:teuthology.orchestra.run.smithi203.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 723 kB/s | 42 kB 00:00 2024-09-21T06:23:23.055 INFO:teuthology.orchestra.run.smithi203.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-21T06:23:23.131 INFO:teuthology.orchestra.run.smithi203.stdout:(89/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-09-21T06:23:23.156 INFO:teuthology.orchestra.run.smithi203.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-21T06:23:23.181 INFO:teuthology.orchestra.run.smithi203.stdout:(91/157): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-09-21T06:23:23.198 INFO:teuthology.orchestra.run.smithi203.stdout:(92/157): python3-pytz-2021.1-5.el9.noarch.rpm 760 kB/s | 51 kB 00:00 2024-09-21T06:23:23.215 INFO:teuthology.orchestra.run.smithi203.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 912 kB/s | 54 kB 00:00 2024-09-21T06:23:23.240 INFO:teuthology.orchestra.run.smithi203.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 991 kB/s | 42 kB 00:00 2024-09-21T06:23:23.257 INFO:teuthology.orchestra.run.smithi203.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-21T06:23:23.299 INFO:teuthology.orchestra.run.smithi203.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-21T06:23:23.316 INFO:teuthology.orchestra.run.smithi203.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 610 kB/s | 36 kB 00:00 2024-09-21T06:23:23.341 INFO:teuthology.orchestra.run.smithi100.stdout:(42/157): ceph-test-19.3.0-5120.g18432218.el9.x 13 MB/s | 40 MB 00:02 2024-09-21T06:23:23.350 INFO:teuthology.orchestra.run.smithi203.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 941 kB/s | 47 kB 00:00 2024-09-21T06:23:23.374 INFO:teuthology.orchestra.run.smithi100.stdout:(43/157): libgfortran-11.5.0-2.el9.x86_64.rpm 1.4 MB/s | 801 kB 00:00 2024-09-21T06:23:23.384 INFO:teuthology.orchestra.run.smithi203.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 4.4 MB/s | 304 kB 00:00 2024-09-21T06:23:23.409 INFO:teuthology.orchestra.run.smithi203.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 985 kB/s | 58 kB 00:00 2024-09-21T06:23:23.416 INFO:teuthology.orchestra.run.smithi100.stdout:(44/157): python3-cryptography-36.0.1-4.el9.x86 2.8 MB/s | 1.2 MB 00:00 2024-09-21T06:23:23.442 INFO:teuthology.orchestra.run.smithi100.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-21T06:23:23.443 INFO:teuthology.orchestra.run.smithi203.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-21T06:23:23.467 INFO:teuthology.orchestra.run.smithi100.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-21T06:23:23.468 INFO:teuthology.orchestra.run.smithi203.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-21T06:23:23.493 INFO:teuthology.orchestra.run.smithi203.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 889 kB/s | 45 kB 00:00 2024-09-21T06:23:23.501 INFO:teuthology.orchestra.run.smithi100.stdout:(47/157): python3-ply-3.11-14.el9.noarch.rpm 669 kB/s | 106 kB 00:00 2024-09-21T06:23:23.526 INFO:teuthology.orchestra.run.smithi100.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-21T06:23:23.651 INFO:teuthology.orchestra.run.smithi100.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 856 kB/s | 107 kB 00:00 2024-09-21T06:23:23.685 INFO:teuthology.orchestra.run.smithi100.stdout:(50/157): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 689 kB 00:00 2024-09-21T06:23:23.710 INFO:teuthology.orchestra.run.smithi100.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 371 kB/s | 22 kB 00:00 2024-09-21T06:23:23.744 INFO:teuthology.orchestra.run.smithi100.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 3.8 MB/s | 943 kB 00:00 2024-09-21T06:23:23.769 INFO:teuthology.orchestra.run.smithi100.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 559 kB/s | 33 kB 00:00 2024-09-21T06:23:23.803 INFO:teuthology.orchestra.run.smithi100.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-21T06:23:23.828 INFO:teuthology.orchestra.run.smithi100.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-09-21T06:23:23.853 INFO:teuthology.orchestra.run.smithi100.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-09-21T06:23:23.879 INFO:teuthology.orchestra.run.smithi100.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 3.3 MB/s | 171 kB 00:00 2024-09-21T06:23:23.910 INFO:teuthology.orchestra.run.smithi203.stdout:(104/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-09-21T06:23:23.969 INFO:teuthology.orchestra.run.smithi203.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 9.4 MB/s | 548 kB 00:00 2024-09-21T06:23:23.994 INFO:teuthology.orchestra.run.smithi203.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 12 MB/s | 308 kB 00:00 2024-09-21T06:23:24.020 INFO:teuthology.orchestra.run.smithi203.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 778 kB/s | 19 kB 00:00 2024-09-21T06:23:24.027 INFO:teuthology.orchestra.run.smithi100.stdout:(58/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-09-21T06:23:24.054 INFO:teuthology.orchestra.run.smithi100.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 258 kB/s | 45 kB 00:00 2024-09-21T06:23:24.088 INFO:teuthology.orchestra.run.smithi100.stdout:(60/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 684 kB/s | 160 kB 00:00 2024-09-21T06:23:24.121 INFO:teuthology.orchestra.run.smithi100.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 8.5 MB/s | 581 kB 00:00 2024-09-21T06:23:24.136 INFO:teuthology.orchestra.run.smithi203.stdout:(108/157): libarrow-9.0.0-11.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-09-21T06:23:24.146 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-09-21T06:23:24.146 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-09-21T06:23:24.147 INFO:teuthology.orchestra.run.smithi100.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 419 kB/s | 24 kB 00:00 2024-09-21T06:23:24.162 INFO:teuthology.orchestra.run.smithi203.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-21T06:23:24.172 INFO:teuthology.orchestra.run.smithi100.stdout:(63/157): librdkafka-1.6.1-102.el9.x86_64.rpm 4.5 MB/s | 662 kB 00:00 2024-09-21T06:23:24.187 INFO:teuthology.orchestra.run.smithi203.stdout:(110/157): lua-devel-5.4.4-4.el9.x86_64.rpm 31 kB/s | 22 kB 00:00 2024-09-21T06:23:24.197 INFO:teuthology.orchestra.run.smithi100.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 627 kB/s | 47 kB 00:00 2024-09-21T06:23:24.212 INFO:teuthology.orchestra.run.smithi203.stdout:(111/157): liboath-2.6.7-2.el9.x86_64.rpm 982 kB/s | 49 kB 00:00 2024-09-21T06:23:24.223 INFO:teuthology.orchestra.run.smithi100.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-21T06:23:24.246 INFO:teuthology.orchestra.run.smithi203.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 5.6 MB/s | 151 kB 00:00 2024-09-21T06:23:24.248 INFO:teuthology.orchestra.run.smithi100.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 768 kB/s | 38 kB 00:00 2024-09-21T06:23:24.273 INFO:teuthology.orchestra.run.smithi100.stdout:(67/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-21T06:23:24.280 INFO:teuthology.orchestra.run.smithi203.stdout:(113/157): libunwind-1.6.2-1.el9.x86_64.rpm 731 kB/s | 67 kB 00:00 2024-09-21T06:23:24.314 INFO:teuthology.orchestra.run.smithi203.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 13 MB/s | 839 kB 00:00 2024-09-21T06:23:24.339 INFO:teuthology.orchestra.run.smithi203.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T06:23:24.364 INFO:teuthology.orchestra.run.smithi203.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T06:23:24.365 INFO:teuthology.orchestra.run.smithi100.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 35 MB/s | 4.9 MB 00:00 2024-09-21T06:23:24.390 INFO:teuthology.orchestra.run.smithi203.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 565 kB/s | 14 kB 00:00 2024-09-21T06:23:24.415 INFO:teuthology.orchestra.run.smithi203.stdout:(118/157): python3-asyncssh-2.13.2-5.el9.noarch 4.0 MB/s | 548 kB 00:00 2024-09-21T06:23:24.423 INFO:teuthology.orchestra.run.smithi100.stdout:(69/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-09-21T06:23:24.440 INFO:teuthology.orchestra.run.smithi203.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-21T06:23:24.465 INFO:teuthology.orchestra.run.smithi100.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 8.9 MB/s | 910 kB 00:00 2024-09-21T06:23:24.466 INFO:teuthology.orchestra.run.smithi203.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 6.9 MB/s | 358 kB 00:00 2024-09-21T06:23:24.491 INFO:teuthology.orchestra.run.smithi203.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-21T06:23:24.507 INFO:teuthology.orchestra.run.smithi100.stdout:(71/157): postgresql-13.16-1.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-09-21T06:23:24.516 INFO:teuthology.orchestra.run.smithi203.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-21T06:23:24.533 INFO:teuthology.orchestra.run.smithi100.stdout:(72/157): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-21T06:23:24.550 INFO:teuthology.orchestra.run.smithi203.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-21T06:23:24.574 INFO:teuthology.orchestra.run.smithi100.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-09-21T06:23:24.575 INFO:teuthology.orchestra.run.smithi203.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-21T06:23:24.600 INFO:teuthology.orchestra.run.smithi100.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-21T06:23:24.600 INFO:teuthology.orchestra.run.smithi203.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 705 kB/s | 18 kB 00:00 2024-09-21T06:23:24.626 INFO:teuthology.orchestra.run.smithi203.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 927 kB/s | 23 kB 00:00 2024-09-21T06:23:24.642 INFO:teuthology.orchestra.run.smithi100.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-21T06:23:24.668 INFO:teuthology.orchestra.run.smithi203.stdout:(127/157): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-09-21T06:23:24.685 INFO:teuthology.orchestra.run.smithi203.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 332 kB/s | 19 kB 00:00 2024-09-21T06:23:24.702 INFO:teuthology.orchestra.run.smithi203.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 583 kB/s | 20 kB 00:00 2024-09-21T06:23:24.717 INFO:teuthology.orchestra.run.smithi100.stdout:(76/157): postgresql-server-13.16-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-09-21T06:23:24.719 INFO:teuthology.orchestra.run.smithi203.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 2.7 MB/s | 46 kB 00:00 2024-09-21T06:23:24.742 INFO:teuthology.orchestra.run.smithi100.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-21T06:23:24.744 INFO:teuthology.orchestra.run.smithi203.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T06:23:24.761 INFO:teuthology.orchestra.run.smithi203.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-09-21T06:23:24.776 INFO:teuthology.orchestra.run.smithi100.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-09-21T06:23:24.794 INFO:teuthology.orchestra.run.smithi203.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 9.3 MB/s | 1.0 MB 00:00 2024-09-21T06:23:24.801 INFO:teuthology.orchestra.run.smithi100.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 436 kB/s | 26 kB 00:00 2024-09-21T06:23:24.827 INFO:teuthology.orchestra.run.smithi100.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-21T06:23:24.828 INFO:teuthology.orchestra.run.smithi203.stdout:(134/157): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-09-21T06:23:24.852 INFO:teuthology.orchestra.run.smithi100.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-21T06:23:24.862 INFO:teuthology.orchestra.run.smithi203.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 243 kB/s | 16 kB 00:00 2024-09-21T06:23:24.877 INFO:teuthology.orchestra.run.smithi100.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-09-21T06:23:24.887 INFO:teuthology.orchestra.run.smithi203.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-21T06:23:24.913 INFO:teuthology.orchestra.run.smithi203.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-09-21T06:23:24.938 INFO:teuthology.orchestra.run.smithi203.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 610 kB/s | 31 kB 00:00 2024-09-21T06:23:24.961 INFO:teuthology.orchestra.run.smithi100.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-21T06:23:24.963 INFO:teuthology.orchestra.run.smithi203.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T06:23:24.986 INFO:teuthology.orchestra.run.smithi100.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-09-21T06:23:24.989 INFO:teuthology.orchestra.run.smithi203.stdout:(140/157): protobuf-compiler-3.14.0-13.el9.x86_ 577 kB/s | 863 kB 00:01 2024-09-21T06:23:25.012 INFO:teuthology.orchestra.run.smithi100.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-21T06:23:25.014 INFO:teuthology.orchestra.run.smithi203.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 710 kB/s | 36 kB 00:00 2024-09-21T06:23:25.037 INFO:teuthology.orchestra.run.smithi100.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-21T06:23:25.048 INFO:teuthology.orchestra.run.smithi203.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-09-21T06:23:25.054 INFO:teuthology.orchestra.run.smithi100.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-21T06:23:25.073 INFO:teuthology.orchestra.run.smithi203.stdout:(143/157): python3-websocket-client-1.2.3-2.el9 3.7 MB/s | 90 kB 00:00 2024-09-21T06:23:25.087 INFO:teuthology.orchestra.run.smithi100.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-21T06:23:25.098 INFO:teuthology.orchestra.run.smithi203.stdout:(144/157): python3-typing-extensions-4.12.2-2.e 681 kB/s | 74 kB 00:00 2024-09-21T06:23:25.104 INFO:teuthology.orchestra.run.smithi100.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-21T06:23:25.124 INFO:teuthology.orchestra.run.smithi203.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-21T06:23:25.130 INFO:teuthology.orchestra.run.smithi100.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 6.5 MB/s | 279 kB 00:00 2024-09-21T06:23:25.149 INFO:teuthology.orchestra.run.smithi203.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 396 kB/s | 20 kB 00:00 2024-09-21T06:23:25.155 INFO:teuthology.orchestra.run.smithi100.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-21T06:23:25.175 INFO:teuthology.orchestra.run.smithi203.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T06:23:25.180 INFO:teuthology.orchestra.run.smithi100.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-21T06:23:25.200 INFO:teuthology.orchestra.run.smithi203.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-21T06:23:25.242 INFO:teuthology.orchestra.run.smithi203.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-21T06:23:25.264 INFO:teuthology.orchestra.run.smithi100.stdout:(93/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-21T06:23:25.289 INFO:teuthology.orchestra.run.smithi100.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2024-09-21T06:23:25.314 INFO:teuthology.orchestra.run.smithi100.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 883 kB/s | 44 kB 00:00 2024-09-21T06:23:25.340 INFO:teuthology.orchestra.run.smithi100.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-21T06:23:25.365 INFO:teuthology.orchestra.run.smithi100.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-09-21T06:23:25.390 INFO:teuthology.orchestra.run.smithi100.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-09-21T06:23:25.415 INFO:teuthology.orchestra.run.smithi100.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-21T06:23:25.441 INFO:teuthology.orchestra.run.smithi100.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T06:23:25.466 INFO:teuthology.orchestra.run.smithi100.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-21T06:23:25.491 INFO:teuthology.orchestra.run.smithi100.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-21T06:23:25.516 INFO:teuthology.orchestra.run.smithi100.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-09-21T06:23:25.542 INFO:teuthology.orchestra.run.smithi100.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 444 kB/s | 22 kB 00:00 2024-09-21T06:23:25.601 INFO:teuthology.orchestra.run.smithi100.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-21T06:23:25.634 INFO:teuthology.orchestra.run.smithi100.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 9.1 MB/s | 308 kB 00:00 2024-09-21T06:23:25.659 INFO:teuthology.orchestra.run.smithi100.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 784 kB/s | 19 kB 00:00 2024-09-21T06:23:25.684 INFO:teuthology.orchestra.run.smithi203.stdout:(150/157): librbd1-19.3.0-5120.g18432218.el9.x8 6.5 MB/s | 3.2 MB 00:00 2024-09-21T06:23:25.693 INFO:teuthology.orchestra.run.smithi100.stdout:(108/157): protobuf-compiler-3.14.0-13.el9.x86_ 4.8 MB/s | 863 kB 00:00 2024-09-21T06:23:25.734 INFO:teuthology.orchestra.run.smithi203.stdout:(151/157): librados2-19.3.0-5120.g18432218.el9. 6.1 MB/s | 3.4 MB 00:00 2024-09-21T06:23:25.743 INFO:teuthology.orchestra.run.smithi100.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 521 kB/s | 26 kB 00:00 2024-09-21T06:23:25.769 INFO:teuthology.orchestra.run.smithi100.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T06:23:25.794 INFO:teuthology.orchestra.run.smithi100.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-21T06:23:25.953 INFO:teuthology.orchestra.run.smithi100.stdout:(112/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-09-21T06:23:25.978 INFO:teuthology.orchestra.run.smithi100.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 824 kB/s | 151 kB 00:00 2024-09-21T06:23:26.018 INFO:teuthology.orchestra.run.smithi203.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 144 kB/s | 41 kB 00:00 2024-09-21T06:23:26.028 INFO:teuthology.orchestra.run.smithi100.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-21T06:23:26.103 INFO:teuthology.orchestra.run.smithi100.stdout:(115/157): libarrow-9.0.0-11.el9.x86_64.rpm 9.9 MB/s | 4.4 MB 00:00 2024-09-21T06:23:26.128 INFO:teuthology.orchestra.run.smithi100.stdout:(116/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 433 kB/s | 43 kB 00:00 2024-09-21T06:23:26.154 INFO:teuthology.orchestra.run.smithi100.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 571 kB/s | 14 kB 00:00 2024-09-21T06:23:26.168 INFO:teuthology.orchestra.run.smithi203.stdout:(153/157): NetworkManager-1.51.0-1.el9.x86_64.r 2.5 MB/s | 2.3 MB 00:00 2024-09-21T06:23:26.179 INFO:teuthology.orchestra.run.smithi100.stdout:(118/157): python3-cachetools-4.2.4-1.el9.noarc 642 kB/s | 32 kB 00:00 2024-09-21T06:23:26.204 INFO:teuthology.orchestra.run.smithi100.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-21T06:23:26.246 INFO:teuthology.orchestra.run.smithi100.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 5.3 MB/s | 358 kB 00:00 2024-09-21T06:23:26.271 INFO:teuthology.orchestra.run.smithi100.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-21T06:23:26.310 INFO:teuthology.orchestra.run.smithi203.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 852 kB/s | 249 kB 00:00 2024-09-21T06:23:26.313 INFO:teuthology.orchestra.run.smithi100.stdout:(122/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.3 MB/s | 839 kB 00:00 2024-09-21T06:23:26.339 INFO:teuthology.orchestra.run.smithi100.stdout:(123/157): python3-google-auth-2.34.0-1.el9.noa 2.3 MB/s | 220 kB 00:00 2024-09-21T06:23:26.364 INFO:teuthology.orchestra.run.smithi100.stdout:(124/157): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-21T06:23:26.385 INFO:teuthology.orchestra.run.smithi203.stdout:(155/157): openssl-3.2.2-6.el9.x86_64.rpm 6.2 MB/s | 1.4 MB 00:00 2024-09-21T06:23:26.389 INFO:teuthology.orchestra.run.smithi100.stdout:(125/157): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-09-21T06:23:26.414 INFO:teuthology.orchestra.run.smithi100.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-21T06:23:26.440 INFO:teuthology.orchestra.run.smithi100.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-21T06:23:26.457 INFO:teuthology.orchestra.run.smithi100.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 463 kB/s | 19 kB 00:00 2024-09-21T06:23:26.498 INFO:teuthology.orchestra.run.smithi100.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 336 kB/s | 20 kB 00:00 2024-09-21T06:23:26.515 INFO:teuthology.orchestra.run.smithi100.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 2.8 MB/s | 46 kB 00:00 2024-09-21T06:23:26.519 INFO:teuthology.orchestra.run.smithi203.stdout:(156/157): NetworkManager-libnm-1.51.0-1.el9.x8 2.2 MB/s | 1.8 MB 00:00 2024-09-21T06:23:26.541 INFO:teuthology.orchestra.run.smithi100.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T06:23:26.558 INFO:teuthology.orchestra.run.smithi100.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-09-21T06:23:26.591 INFO:teuthology.orchestra.run.smithi100.stdout:(133/157): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-21T06:23:26.608 INFO:teuthology.orchestra.run.smithi100.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 988 kB/s | 16 kB 00:00 2024-09-21T06:23:26.633 INFO:teuthology.orchestra.run.smithi100.stdout:(135/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-21T06:23:26.667 INFO:teuthology.orchestra.run.smithi100.stdout:(136/157): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2024-09-21T06:23:26.692 INFO:teuthology.orchestra.run.smithi100.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 481 kB/s | 28 kB 00:00 2024-09-21T06:23:26.718 INFO:teuthology.orchestra.run.smithi100.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 610 kB/s | 31 kB 00:00 2024-09-21T06:23:26.743 INFO:teuthology.orchestra.run.smithi100.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T06:23:26.769 INFO:teuthology.orchestra.run.smithi100.stdout:(140/157): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-21T06:23:26.785 INFO:teuthology.orchestra.run.smithi203.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 5.0 MB/s | 2.4 MB 00:00 2024-09-21T06:23:26.792 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-09-21T06:23:26.792 INFO:teuthology.orchestra.run.smithi203.stdout:Total 20 MB/s | 228 MB 00:11 2024-09-21T06:23:26.794 INFO:teuthology.orchestra.run.smithi100.stdout:(141/157): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-21T06:23:26.819 INFO:teuthology.orchestra.run.smithi100.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-21T06:23:26.836 INFO:teuthology.orchestra.run.smithi100.stdout:(143/157): python3-websocket-client-1.2.3-2.el9 5.3 MB/s | 90 kB 00:00 2024-09-21T06:23:26.862 INFO:teuthology.orchestra.run.smithi100.stdout:(144/157): python3-webob-1.8.8-2.el9.noarch.rpm 3.3 MB/s | 230 kB 00:00 2024-09-21T06:23:26.887 INFO:teuthology.orchestra.run.smithi100.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 806 kB/s | 20 kB 00:00 2024-09-21T06:23:26.929 INFO:teuthology.orchestra.run.smithi100.stdout:(146/157): python3-grpcio-1.46.7-10.el9.x86_64. 3.1 MB/s | 2.0 MB 00:00 2024-09-21T06:23:27.028 INFO:teuthology.orchestra.run.smithi100.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 1.3 MB/s | 191 kB 00:00 2024-09-21T06:23:27.054 INFO:teuthology.orchestra.run.smithi100.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 1.9 MB/s | 427 kB 00:00 2024-09-21T06:23:27.105 INFO:teuthology.orchestra.run.smithi100.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 9.0 MB/s | 1.6 MB 00:00 2024-09-21T06:23:27.280 INFO:teuthology.orchestra.run.smithi100.stdout:(150/157): NetworkManager-1.51.0-1.el9.x86_64.r 13 MB/s | 2.3 MB 00:00 2024-09-21T06:23:27.347 INFO:teuthology.orchestra.run.smithi100.stdout:(151/157): NetworkManager-libnm-1.51.0-1.el9.x8 27 MB/s | 1.8 MB 00:00 2024-09-21T06:23:27.389 INFO:teuthology.orchestra.run.smithi100.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 981 kB/s | 41 kB 00:00 2024-09-21T06:23:27.431 INFO:teuthology.orchestra.run.smithi100.stdout:(153/157): NetworkManager-tui-1.51.0-1.el9.x86_ 5.8 MB/s | 249 kB 00:00 2024-09-21T06:23:27.490 INFO:teuthology.orchestra.run.smithi100.stdout:(154/157): openssl-3.2.2-6.el9.x86_64.rpm 23 MB/s | 1.4 MB 00:00 2024-09-21T06:23:27.556 INFO:teuthology.orchestra.run.smithi100.stdout:(155/157): librados2-19.3.0-5120.g18432218.el9. 6.4 MB/s | 3.4 MB 00:00 2024-09-21T06:23:27.615 INFO:teuthology.orchestra.run.smithi100.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-09-21T06:23:27.690 INFO:teuthology.orchestra.run.smithi100.stdout:(157/157): librbd1-19.3.0-5120.g18432218.el9.x8 5.0 MB/s | 3.2 MB 00:00 2024-09-21T06:23:27.697 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-09-21T06:23:27.697 INFO:teuthology.orchestra.run.smithi100.stdout:Total 23 MB/s | 228 MB 00:10 2024-09-21T06:23:28.041 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-09-21T06:23:28.194 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-09-21T06:23:28.194 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-09-21T06:23:28.956 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-09-21T06:23:29.152 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-09-21T06:23:29.152 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-09-21T06:23:30.430 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-09-21T06:23:30.430 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-09-21T06:23:30.654 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-09-21T06:23:30.791 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T06:23:30.838 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T06:23:30.892 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T06:23:30.941 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T06:23:30.965 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T06:23:31.016 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T06:23:31.366 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-09-21T06:23:31.367 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-09-21T06:23:31.378 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T06:23:31.508 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:23:31.638 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:23:31.668 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:23:31.761 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:23:31.789 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T06:23:31.819 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T06:23:31.995 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T06:23:32.053 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T06:23:32.080 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:23:32.172 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:23:32.198 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T06:23:32.245 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T06:23:32.274 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T06:23:32.298 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T06:23:32.494 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T06:23:32.687 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T06:23:32.831 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T06:23:32.940 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:23:33.024 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:23:33.257 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:23:33.366 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T06:23:33.395 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T06:23:33.431 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T06:23:33.515 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T06:23:33.565 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T06:23:33.605 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T06:23:33.656 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T06:23:33.689 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T06:23:33.716 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T06:23:33.789 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T06:23:33.841 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T06:23:33.869 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T06:23:33.913 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T06:23:33.997 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T06:23:34.032 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T06:23:34.076 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T06:23:34.112 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T06:23:34.195 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T06:23:34.244 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T06:23:34.323 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T06:23:34.467 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T06:23:34.522 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T06:23:35.082 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T06:23:35.169 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T06:23:35.203 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T06:23:35.247 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T06:23:35.285 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T06:23:35.327 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T06:23:35.506 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T06:23:35.571 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T06:23:35.676 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T06:23:35.737 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T06:23:35.853 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T06:23:36.091 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T06:23:36.126 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T06:23:36.174 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T06:23:36.208 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T06:23:36.252 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T06:23:36.387 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T06:23:36.704 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T06:23:36.745 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T06:23:36.787 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T06:23:36.903 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T06:23:36.945 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T06:23:36.985 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T06:23:37.019 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T06:23:37.043 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T06:23:37.073 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-09-21T06:23:37.153 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T06:23:37.230 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T06:23:37.276 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T06:23:37.283 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T06:23:37.326 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T06:23:37.338 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T06:23:37.353 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T06:23:37.373 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T06:23:37.382 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T06:23:37.398 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T06:23:37.423 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T06:23:37.446 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T06:23:37.462 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T06:23:37.500 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T06:23:37.534 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T06:23:37.568 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T06:23:37.610 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T06:23:37.644 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T06:23:37.651 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-09-21T06:23:37.733 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T06:23:37.752 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T06:23:37.783 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T06:23:37.791 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T06:23:37.795 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T06:23:37.813 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T06:23:37.844 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T06:23:37.877 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T06:23:37.903 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T06:23:37.916 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:23:37.922 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T06:23:37.944 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T06:23:37.982 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T06:23:38.027 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:23:38.051 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:23:38.143 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:23:38.172 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T06:23:38.202 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T06:23:38.293 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T06:23:38.375 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T06:23:38.421 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:23:38.428 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T06:23:38.464 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:23:38.530 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:23:38.557 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:23:38.580 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:23:38.607 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T06:23:38.649 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:23:38.675 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T06:23:38.679 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T06:23:38.700 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T06:23:38.708 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T06:23:38.722 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T06:23:38.747 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T06:23:38.795 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T06:23:38.829 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T06:23:38.879 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T06:23:38.886 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T06:23:38.933 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T06:23:38.934 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T06:23:38.962 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:23:38.993 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T06:23:39.052 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:23:39.055 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T06:23:39.073 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T06:23:39.080 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T06:23:39.140 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T06:23:39.144 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T06:23:39.164 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T06:23:39.181 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T06:23:39.186 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T06:23:39.191 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T06:23:39.315 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:23:39.371 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T06:23:39.374 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T06:23:39.395 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T06:23:39.409 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:23:39.483 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T06:23:39.533 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T06:23:39.651 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:23:39.662 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T06:23:39.780 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T06:23:39.836 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:23:39.836 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T06:23:39.856 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:23:39.865 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T06:23:39.939 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T06:23:40.000 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T06:23:40.041 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T06:23:40.077 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:23:40.092 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T06:23:40.133 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T06:23:40.170 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T06:23:40.187 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T06:23:40.188 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T06:23:40.218 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T06:23:40.240 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T06:23:40.244 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T06:23:40.291 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T06:23:40.314 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T06:23:40.317 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T06:23:40.323 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T06:23:40.362 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T06:23:40.375 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T06:23:40.403 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T06:23:40.446 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T06:23:40.455 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T06:23:40.479 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T06:23:40.485 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T06:23:40.507 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T06:23:40.538 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T06:23:40.574 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T06:23:40.576 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T06:23:40.628 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T06:23:40.646 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T06:23:40.659 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T06:23:40.698 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T06:23:40.703 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T06:23:40.772 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T06:23:40.782 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T06:23:40.821 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T06:23:40.865 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T06:23:40.894 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T06:23:40.905 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T06:23:40.958 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T06:23:40.962 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T06:23:41.008 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T06:23:41.082 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T06:23:41.221 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T06:23:41.276 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T06:23:41.483 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T06:23:41.580 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T06:23:41.608 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T06:23:41.652 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T06:23:41.652 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T06:23:41.698 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T06:23:41.747 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T06:23:41.748 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T06:23:41.775 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T06:23:41.786 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T06:23:41.808 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T06:23:41.871 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T06:23:41.902 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T06:23:41.928 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T06:23:41.936 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T06:23:41.974 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T06:23:42.009 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T06:23:42.024 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T06:23:42.101 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T06:23:42.153 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T06:23:42.157 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T06:23:42.169 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T06:23:42.213 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T06:23:42.219 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T06:23:42.238 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T06:23:42.296 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T06:23:42.308 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T06:23:42.314 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T06:23:42.331 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T06:23:42.343 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T06:23:42.374 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T06:23:42.378 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T06:23:42.387 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T06:23:42.414 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T06:23:42.438 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T06:23:42.457 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T06:23:42.471 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T06:23:42.499 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T06:23:42.507 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T06:23:42.553 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T06:23:42.589 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T06:23:42.632 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T06:23:42.637 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T06:23:42.675 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T06:23:42.685 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T06:23:42.726 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T06:23:42.808 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T06:23:42.830 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T06:23:42.924 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T06:23:42.929 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T06:23:42.957 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T06:23:43.029 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T06:23:43.044 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T06:23:43.059 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T06:23:43.090 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T06:23:43.118 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T06:23:43.143 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T06:23:43.164 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T06:23:43.246 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T06:23:43.266 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T06:23:43.303 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T06:23:43.344 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T06:23:43.369 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T06:23:43.384 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T06:23:43.409 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T06:23:43.412 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T06:23:43.451 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T06:23:43.483 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T06:23:43.503 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T06:23:43.523 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T06:23:43.554 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T06:23:43.592 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T06:23:43.627 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T06:23:43.634 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T06:23:43.660 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T06:23:43.674 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T06:23:43.702 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T06:23:43.702 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T06:23:43.736 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T06:23:43.736 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T06:23:43.784 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T06:23:43.816 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T06:23:43.827 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T06:23:43.845 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T06:23:43.875 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T06:23:43.880 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T06:23:43.906 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T06:23:43.913 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T06:23:43.955 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T06:23:43.989 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T06:23:44.019 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T06:23:44.074 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T06:23:44.076 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T06:23:44.128 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T06:23:44.163 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T06:23:44.265 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T06:23:44.322 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T06:23:44.798 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T06:23:44.847 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T06:23:44.881 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T06:23:44.979 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T06:23:45.036 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T06:23:45.036 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T06:23:45.078 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T06:23:45.089 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T06:23:45.111 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T06:23:45.179 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T06:23:45.208 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T06:23:45.226 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T06:23:45.266 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T06:23:45.311 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T06:23:45.386 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:23:45.397 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T06:23:45.422 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:23:45.426 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T06:23:45.439 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T06:23:45.448 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T06:23:45.534 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T06:23:45.621 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T06:23:45.645 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T06:23:45.722 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T06:23:46.036 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:23:46.075 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T06:23:46.159 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T06:23:46.159 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T06:23:46.159 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T06:23:46.159 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:23:46.195 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T06:23:46.201 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T06:23:46.323 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T06:23:46.360 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T06:23:46.476 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T06:23:47.619 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T06:23:47.713 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T06:23:47.713 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T06:23:47.741 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T06:23:47.774 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T06:23:47.810 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T06:23:47.838 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T06:23:47.871 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T06:23:48.101 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T06:23:48.145 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T06:23:48.195 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T06:23:48.229 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T06:23:48.242 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T06:23:48.259 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T06:23:48.303 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T06:23:48.334 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T06:23:48.363 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T06:23:48.399 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T06:23:48.809 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T06:23:48.849 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T06:23:48.897 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T06:23:48.902 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T06:23:48.925 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T06:23:48.936 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T06:23:49.000 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T06:23:49.030 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T06:23:51.222 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:23:51.264 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:23:51.319 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:23:51.510 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:23:51.851 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:23:51.893 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T06:23:51.985 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T06:23:51.985 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T06:23:51.985 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T06:23:51.985 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T06:23:52.028 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T06:23:52.086 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:23:52.122 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T06:23:52.194 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T06:23:52.194 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T06:23:52.194 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T06:23:52.194 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-21T06:23:52.262 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T06:23:58.370 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T06:23:58.370 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /sys 2024-09-21T06:23:58.371 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /proc 2024-09-21T06:23:58.371 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /mnt 2024-09-21T06:23:58.371 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /var/tmp 2024-09-21T06:23:58.371 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /home 2024-09-21T06:23:58.371 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /root 2024-09-21T06:23:58.371 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /tmp 2024-09-21T06:23:58.371 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:23:58.556 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T06:23:58.573 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T06:23:58.573 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:23:58.573 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T06:23:58.573 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:24:00.242 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T06:24:00.370 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T06:24:00.370 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:24:00.370 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T06:24:00.370 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T06:24:00.370 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T06:24:00.370 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:24:00.509 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T06:24:00.534 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T06:24:00.534 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:24:00.534 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T06:24:00.534 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T06:24:00.534 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T06:24:00.534 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:24:01.026 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T06:24:01.051 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T06:24:01.052 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:24:01.052 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T06:24:01.052 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T06:24:01.052 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T06:24:01.052 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:24:01.157 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T06:24:01.216 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T06:24:01.279 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T06:24:01.484 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T06:24:01.536 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T06:24:01.626 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T06:24:01.714 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T06:24:01.874 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T06:24:01.919 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T06:24:01.959 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T06:24:01.986 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T06:24:02.414 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T06:24:02.444 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T06:24:02.476 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T06:24:02.545 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T06:24:02.575 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T06:24:03.903 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T06:24:03.903 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /sys 2024-09-21T06:24:03.903 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /proc 2024-09-21T06:24:03.903 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /mnt 2024-09-21T06:24:03.903 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /var/tmp 2024-09-21T06:24:03.903 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /home 2024-09-21T06:24:03.903 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /root 2024-09-21T06:24:03.903 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /tmp 2024-09-21T06:24:03.903 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-21T06:24:03.973 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T06:24:03.973 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /sys 2024-09-21T06:24:03.973 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /proc 2024-09-21T06:24:03.973 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /mnt 2024-09-21T06:24:03.973 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /var/tmp 2024-09-21T06:24:03.973 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /home 2024-09-21T06:24:03.973 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /root 2024-09-21T06:24:03.973 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /tmp 2024-09-21T06:24:03.973 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T06:24:04.070 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T06:24:04.087 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T06:24:04.087 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:24:04.087 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T06:24:04.087 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-21T06:24:04.153 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T06:24:04.171 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T06:24:04.171 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:24:04.171 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T06:24:04.172 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T06:24:05.690 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T06:24:05.717 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T06:24:05.717 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:24:05.717 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T06:24:05.718 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T06:24:05.718 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T06:24:05.718 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-21T06:24:05.773 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T06:24:05.800 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T06:24:05.800 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:24:05.800 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T06:24:05.800 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T06:24:05.800 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T06:24:05.800 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T06:24:05.941 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T06:24:05.961 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T06:24:05.965 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T06:24:05.965 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:24:05.965 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T06:24:05.965 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T06:24:05.965 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T06:24:05.965 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-21T06:24:06.023 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T06:24:06.032 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T06:24:06.056 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T06:24:06.056 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:24:06.056 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T06:24:06.056 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T06:24:06.056 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T06:24:06.056 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T06:24:06.434 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T06:24:06.458 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T06:24:06.458 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:24:06.458 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T06:24:06.458 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T06:24:06.458 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T06:24:06.459 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-21T06:24:06.524 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T06:24:06.548 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T06:24:06.548 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:24:06.548 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T06:24:06.549 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T06:24:06.549 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T06:24:06.549 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T06:24:06.563 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T06:24:06.612 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T06:24:06.654 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T06:24:06.667 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T06:24:06.712 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T06:24:06.775 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T06:24:06.859 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T06:24:06.909 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T06:24:06.962 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T06:24:06.986 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T06:24:07.017 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T06:24:07.041 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T06:24:07.074 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T06:24:07.081 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T06:24:07.102 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T06:24:07.201 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T06:24:07.202 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T06:24:07.256 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T06:24:07.293 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T06:24:07.324 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T06:24:07.341 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T06:24:07.352 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T06:24:07.381 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T06:24:07.384 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T06:24:07.408 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T06:24:07.416 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T06:24:07.433 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T06:24:07.433 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:24:07.434 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T06:24:07.434 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T06:24:07.434 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T06:24:07.434 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:24:07.453 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T06:24:07.484 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T06:24:07.496 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T06:24:07.599 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T06:24:07.656 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T06:24:07.821 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T06:24:07.851 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T06:24:07.883 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T06:24:07.887 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T06:24:07.918 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T06:24:07.949 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T06:24:07.950 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T06:24:07.974 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T06:24:08.016 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T06:24:08.050 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T06:24:08.680 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T06:24:08.711 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T06:24:08.736 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T06:24:08.736 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:24:08.736 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T06:24:08.736 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T06:24:08.737 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T06:24:08.737 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:24:08.787 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T06:24:08.811 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T06:24:08.811 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:24:08.811 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T06:24:08.811 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:24:08.870 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T06:24:08.895 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T06:24:08.895 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:24:08.895 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T06:24:08.895 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T06:24:08.895 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T06:24:08.895 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:24:09.230 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T06:24:09.255 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T06:24:09.255 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:24:09.255 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T06:24:09.255 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T06:24:09.255 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T06:24:09.255 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:24:11.138 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T06:24:11.197 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T06:24:11.231 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T06:24:11.299 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T06:24:12.153 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T06:24:12.181 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T06:24:12.265 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T06:24:12.295 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T06:24:12.298 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T06:24:12.413 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T06:24:12.464 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T06:24:12.491 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T06:24:12.515 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T06:24:12.515 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:24:12.515 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T06:24:12.515 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T06:24:12.516 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T06:24:12.516 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-21T06:24:12.567 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T06:24:12.579 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T06:24:12.582 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T06:24:12.610 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T06:24:12.633 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T06:24:12.633 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:24:12.634 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T06:24:12.634 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T06:24:12.634 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T06:24:12.634 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T06:24:12.676 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T06:24:12.685 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T06:24:12.698 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T06:24:12.740 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T06:24:12.797 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T06:24:12.850 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T06:24:13.523 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T06:24:13.567 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T06:24:13.608 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T06:24:13.639 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T06:24:13.699 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T06:24:13.717 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T06:24:13.727 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T06:24:13.751 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T06:24:13.752 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:24:13.752 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T06:24:13.752 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T06:24:13.752 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T06:24:13.752 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-21T06:24:13.795 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T06:24:13.818 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T06:24:13.818 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:24:13.818 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T06:24:13.818 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-21T06:24:13.826 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T06:24:13.863 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T06:24:13.870 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T06:24:13.887 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T06:24:13.887 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:24:13.887 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T06:24:13.887 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T06:24:13.887 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T06:24:13.887 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T06:24:13.894 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T06:24:13.894 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:24:13.894 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T06:24:13.894 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T06:24:13.894 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T06:24:13.894 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-21T06:24:13.932 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T06:24:13.939 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T06:24:13.962 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T06:24:13.962 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:24:13.962 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T06:24:13.962 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T06:24:13.980 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T06:24:14.010 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T06:24:14.015 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T06:24:14.038 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T06:24:14.039 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:24:14.039 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T06:24:14.039 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T06:24:14.039 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T06:24:14.039 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T06:24:14.039 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T06:24:14.086 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T06:24:14.222 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T06:24:14.245 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T06:24:14.245 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:24:14.245 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T06:24:14.246 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T06:24:14.246 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T06:24:14.246 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-21T06:24:14.382 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T06:24:14.406 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T06:24:14.406 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:24:14.406 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T06:24:14.406 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T06:24:14.406 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T06:24:14.406 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T06:24:14.689 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:24:14.733 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:24:14.755 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:24:15.670 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T06:24:15.762 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T06:24:15.814 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T06:24:15.906 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T06:24:15.949 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T06:24:15.974 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T06:24:16.000 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T06:24:16.000 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T06:24:16.092 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T06:24:16.092 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T06:24:16.176 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T06:24:16.201 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T06:24:16.207 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:24:16.207 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:24:16.395 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:24:16.395 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T06:24:16.471 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T06:24:16.539 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:24:18.273 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T06:24:18.325 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T06:24:18.365 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T06:24:18.398 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T06:24:18.425 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T06:24:18.433 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T06:24:18.478 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T06:24:18.518 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T06:24:18.551 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T06:24:18.578 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T06:24:18.628 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T06:24:18.680 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T06:24:18.710 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T06:24:18.740 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T06:24:18.778 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T06:24:18.798 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T06:24:18.862 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T06:24:18.889 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T06:24:18.919 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T06:24:18.965 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T06:24:19.406 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:24:19.451 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:24:19.471 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:24:19.601 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:24:19.668 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:24:19.689 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:24:20.312 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T06:24:20.397 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T06:24:20.450 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T06:24:20.548 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T06:24:20.550 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T06:24:20.601 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T06:24:20.626 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T06:24:20.637 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T06:24:20.660 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T06:24:20.660 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T06:24:20.684 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T06:24:20.736 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T06:24:20.736 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T06:24:20.781 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T06:24:20.816 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T06:24:20.820 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T06:24:20.841 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T06:24:20.845 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T06:24:20.851 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:24:20.851 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:24:20.875 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T06:24:20.875 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T06:24:20.968 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T06:24:20.968 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T06:24:21.030 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:24:21.030 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T06:24:21.060 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T06:24:21.085 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T06:24:21.091 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:24:21.091 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:24:21.114 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T06:24:21.187 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:24:21.270 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:24:21.271 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T06:24:21.346 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T06:24:21.412 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:24:22.866 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:24:22.866 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T06:24:22.866 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T06:24:22.866 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T06:24:22.867 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T06:24:22.867 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T06:24:22.867 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T06:24:22.867 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T06:24:22.867 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:24:22.867 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:24:22.867 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T06:24:22.867 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T06:24:22.868 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T06:24:22.868 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T06:24:22.868 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:24:22.868 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T06:24:22.868 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T06:24:22.868 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T06:24:22.868 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T06:24:22.869 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T06:24:22.869 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T06:24:22.869 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T06:24:22.869 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T06:24:22.869 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T06:24:22.869 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T06:24:22.869 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T06:24:22.870 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T06:24:22.870 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T06:24:22.870 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T06:24:22.870 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T06:24:22.870 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T06:24:22.870 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T06:24:22.871 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T06:24:22.871 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T06:24:22.871 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T06:24:22.871 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T06:24:22.871 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T06:24:22.871 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T06:24:22.871 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T06:24:22.871 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T06:24:22.872 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T06:24:22.872 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T06:24:22.872 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T06:24:22.872 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T06:24:22.872 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T06:24:22.872 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T06:24:22.872 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T06:24:22.873 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T06:24:22.873 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T06:24:22.873 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T06:24:22.873 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T06:24:22.873 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T06:24:22.873 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T06:24:22.873 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T06:24:22.874 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T06:24:22.874 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T06:24:22.874 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T06:24:22.874 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T06:24:22.874 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T06:24:22.874 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T06:24:22.874 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T06:24:22.874 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T06:24:22.875 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T06:24:22.875 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T06:24:22.875 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T06:24:22.875 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T06:24:22.875 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T06:24:22.875 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T06:24:22.875 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T06:24:22.876 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T06:24:22.876 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T06:24:22.876 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T06:24:22.876 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T06:24:22.876 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T06:24:22.876 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T06:24:22.877 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T06:24:22.877 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T06:24:22.877 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T06:24:22.877 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T06:24:22.877 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T06:24:22.877 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T06:24:22.877 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T06:24:22.878 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T06:24:22.878 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T06:24:22.878 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T06:24:22.878 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T06:24:22.878 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T06:24:22.878 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T06:24:22.879 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T06:24:22.879 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T06:24:22.879 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T06:24:22.879 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T06:24:22.879 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T06:24:22.879 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T06:24:22.879 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T06:24:22.880 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T06:24:22.880 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T06:24:22.880 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T06:24:22.880 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T06:24:22.880 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T06:24:22.880 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T06:24:22.880 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T06:24:22.881 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T06:24:22.881 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T06:24:22.881 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T06:24:22.881 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T06:24:22.881 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T06:24:22.881 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T06:24:22.882 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T06:24:22.882 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T06:24:22.882 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T06:24:22.882 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T06:24:22.882 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T06:24:22.882 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T06:24:22.882 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T06:24:22.883 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T06:24:22.883 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T06:24:22.883 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T06:24:22.883 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T06:24:22.883 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T06:24:22.883 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T06:24:22.884 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T06:24:22.884 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T06:24:22.884 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T06:24:22.884 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T06:24:22.884 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T06:24:22.884 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T06:24:22.884 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T06:24:22.885 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T06:24:22.885 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T06:24:22.885 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T06:24:22.885 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T06:24:22.885 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T06:24:22.885 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T06:24:22.885 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T06:24:22.886 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T06:24:22.886 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T06:24:22.886 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T06:24:22.886 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T06:24:22.886 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T06:24:22.886 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T06:24:22.887 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T06:24:22.887 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T06:24:22.887 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T06:24:22.887 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T06:24:22.887 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T06:24:22.887 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T06:24:22.887 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T06:24:22.888 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T06:24:22.888 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T06:24:22.888 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T06:24:22.888 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T06:24:22.888 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T06:24:22.888 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T06:24:22.888 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T06:24:22.889 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T06:24:22.889 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T06:24:22.889 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T06:24:22.889 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T06:24:22.889 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T06:24:22.889 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T06:24:22.889 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T06:24:22.890 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T06:24:22.890 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T06:24:22.890 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T06:24:25.814 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:24:25.814 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:24:25.814 INFO:teuthology.orchestra.run.smithi067.stdout:Upgraded: 2024-09-21T06:24:25.814 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T06:24:25.814 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T06:24:25.815 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T06:24:25.815 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T06:24:25.815 INFO:teuthology.orchestra.run.smithi067.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:25.815 INFO:teuthology.orchestra.run.smithi067.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:25.815 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T06:24:25.815 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T06:24:25.815 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-09-21T06:24:25.815 INFO:teuthology.orchestra.run.smithi067.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T06:24:25.815 INFO:teuthology.orchestra.run.smithi067.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T06:24:25.815 INFO:teuthology.orchestra.run.smithi067.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T06:24:25.815 INFO:teuthology.orchestra.run.smithi067.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T06:24:25.815 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:25.816 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:25.816 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:25.816 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:25.816 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:24:25.816 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:25.816 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:25.816 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:25.816 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:24:25.816 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:24:25.816 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:24:25.816 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:24:25.816 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:24:25.816 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:25.816 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:25.817 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:24:25.817 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:25.817 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:25.817 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:25.817 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:24:25.817 INFO:teuthology.orchestra.run.smithi067.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:24:25.817 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:25.817 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:25.817 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:24:25.818 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T06:24:25.818 INFO:teuthology.orchestra.run.smithi067.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T06:24:25.818 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T06:24:25.818 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T06:24:25.818 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T06:24:25.818 INFO:teuthology.orchestra.run.smithi067.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T06:24:25.818 INFO:teuthology.orchestra.run.smithi067.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T06:24:25.818 INFO:teuthology.orchestra.run.smithi067.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T06:24:25.818 INFO:teuthology.orchestra.run.smithi067.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T06:24:25.818 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T06:24:25.818 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T06:24:25.818 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:25.818 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:25.818 INFO:teuthology.orchestra.run.smithi067.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:25.819 INFO:teuthology.orchestra.run.smithi067.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T06:24:25.819 INFO:teuthology.orchestra.run.smithi067.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T06:24:25.819 INFO:teuthology.orchestra.run.smithi067.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T06:24:25.819 INFO:teuthology.orchestra.run.smithi067.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T06:24:25.819 INFO:teuthology.orchestra.run.smithi067.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T06:24:25.819 INFO:teuthology.orchestra.run.smithi067.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T06:24:25.819 INFO:teuthology.orchestra.run.smithi067.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:25.819 INFO:teuthology.orchestra.run.smithi067.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:25.819 INFO:teuthology.orchestra.run.smithi067.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T06:24:25.819 INFO:teuthology.orchestra.run.smithi067.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:25.819 INFO:teuthology.orchestra.run.smithi067.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T06:24:25.819 INFO:teuthology.orchestra.run.smithi067.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T06:24:25.819 INFO:teuthology.orchestra.run.smithi067.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T06:24:25.819 INFO:teuthology.orchestra.run.smithi067.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T06:24:25.820 INFO:teuthology.orchestra.run.smithi067.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T06:24:25.820 INFO:teuthology.orchestra.run.smithi067.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T06:24:25.820 INFO:teuthology.orchestra.run.smithi067.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T06:24:25.820 INFO:teuthology.orchestra.run.smithi067.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T06:24:25.820 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T06:24:25.820 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T06:24:25.820 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T06:24:25.820 INFO:teuthology.orchestra.run.smithi067.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T06:24:25.820 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T06:24:25.820 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T06:24:25.820 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T06:24:25.820 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T06:24:25.820 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T06:24:25.820 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T06:24:25.820 INFO:teuthology.orchestra.run.smithi067.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T06:24:25.821 INFO:teuthology.orchestra.run.smithi067.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T06:24:25.821 INFO:teuthology.orchestra.run.smithi067.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T06:24:25.821 INFO:teuthology.orchestra.run.smithi067.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T06:24:25.821 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T06:24:25.821 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:25.821 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:25.821 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:25.821 INFO:teuthology.orchestra.run.smithi067.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T06:24:25.821 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T06:24:25.821 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T06:24:25.821 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T06:24:25.822 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T06:24:25.822 INFO:teuthology.orchestra.run.smithi067.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T06:24:25.822 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T06:24:25.822 INFO:teuthology.orchestra.run.smithi067.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T06:24:25.822 INFO:teuthology.orchestra.run.smithi067.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T06:24:25.822 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T06:24:25.822 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T06:24:25.822 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T06:24:25.822 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T06:24:25.822 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T06:24:25.822 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T06:24:25.822 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T06:24:25.822 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T06:24:25.822 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T06:24:25.823 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T06:24:25.823 INFO:teuthology.orchestra.run.smithi067.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T06:24:25.823 INFO:teuthology.orchestra.run.smithi067.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T06:24:25.823 INFO:teuthology.orchestra.run.smithi067.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T06:24:25.823 INFO:teuthology.orchestra.run.smithi067.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T06:24:25.823 INFO:teuthology.orchestra.run.smithi067.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T06:24:25.823 INFO:teuthology.orchestra.run.smithi067.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T06:24:25.823 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T06:24:25.823 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T06:24:25.823 INFO:teuthology.orchestra.run.smithi067.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T06:24:25.823 INFO:teuthology.orchestra.run.smithi067.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T06:24:25.823 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T06:24:25.823 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T06:24:25.824 INFO:teuthology.orchestra.run.smithi067.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T06:24:25.824 INFO:teuthology.orchestra.run.smithi067.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T06:24:25.824 INFO:teuthology.orchestra.run.smithi067.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T06:24:25.824 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T06:24:25.824 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T06:24:25.824 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T06:24:25.824 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T06:24:25.824 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T06:24:25.824 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T06:24:25.824 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:25.824 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:25.824 INFO:teuthology.orchestra.run.smithi067.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T06:24:25.824 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T06:24:25.824 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T06:24:25.825 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:25.825 INFO:teuthology.orchestra.run.smithi067.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T06:24:25.825 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T06:24:25.825 INFO:teuthology.orchestra.run.smithi067.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T06:24:25.825 INFO:teuthology.orchestra.run.smithi067.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T06:24:25.825 INFO:teuthology.orchestra.run.smithi067.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T06:24:25.825 INFO:teuthology.orchestra.run.smithi067.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T06:24:25.825 INFO:teuthology.orchestra.run.smithi067.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T06:24:25.825 INFO:teuthology.orchestra.run.smithi067.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T06:24:25.825 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T06:24:25.825 INFO:teuthology.orchestra.run.smithi067.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T06:24:25.825 INFO:teuthology.orchestra.run.smithi067.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T06:24:25.826 INFO:teuthology.orchestra.run.smithi067.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T06:24:25.826 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T06:24:25.826 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T06:24:25.826 INFO:teuthology.orchestra.run.smithi067.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T06:24:25.826 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:25.826 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:25.826 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:25.826 INFO:teuthology.orchestra.run.smithi067.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T06:24:25.826 INFO:teuthology.orchestra.run.smithi067.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T06:24:25.826 INFO:teuthology.orchestra.run.smithi067.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T06:24:25.826 INFO:teuthology.orchestra.run.smithi067.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T06:24:25.826 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T06:24:25.826 INFO:teuthology.orchestra.run.smithi067.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T06:24:25.827 INFO:teuthology.orchestra.run.smithi067.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T06:24:25.827 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:24:25.827 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-21T06:24:26.225 DEBUG:teuthology.parallel:result is None 2024-09-21T06:24:27.594 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:24:27.595 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T06:24:27.595 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T06:24:27.595 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T06:24:27.595 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T06:24:27.595 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T06:24:27.595 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T06:24:27.595 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T06:24:27.595 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:24:27.595 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:24:27.595 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T06:24:27.595 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T06:24:27.596 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T06:24:27.596 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T06:24:27.596 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:24:27.596 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T06:24:27.596 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T06:24:27.596 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T06:24:27.596 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T06:24:27.596 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T06:24:27.596 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T06:24:27.596 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T06:24:27.596 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T06:24:27.596 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T06:24:27.596 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T06:24:27.596 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T06:24:27.597 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T06:24:27.597 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T06:24:27.597 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T06:24:27.608 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T06:24:27.608 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T06:24:27.608 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T06:24:27.608 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T06:24:27.608 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T06:24:27.608 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T06:24:27.608 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T06:24:27.608 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T06:24:27.608 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T06:24:27.609 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T06:24:27.609 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T06:24:27.609 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T06:24:27.609 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T06:24:27.609 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T06:24:27.609 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T06:24:27.609 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T06:24:27.609 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T06:24:27.609 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T06:24:27.609 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T06:24:27.609 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T06:24:27.609 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T06:24:27.609 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T06:24:27.609 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T06:24:27.610 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T06:24:27.610 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T06:24:27.610 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T06:24:27.610 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T06:24:27.610 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T06:24:27.610 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T06:24:27.610 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T06:24:27.610 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T06:24:27.610 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T06:24:27.610 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T06:24:27.610 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T06:24:27.610 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T06:24:27.610 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T06:24:27.611 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T06:24:27.611 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T06:24:27.611 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T06:24:27.611 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T06:24:27.611 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T06:24:27.611 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T06:24:27.611 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T06:24:27.611 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T06:24:27.611 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T06:24:27.611 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T06:24:27.611 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T06:24:27.611 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T06:24:27.611 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T06:24:27.612 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T06:24:27.612 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T06:24:27.612 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T06:24:27.612 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T06:24:27.612 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T06:24:27.612 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T06:24:27.612 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T06:24:27.612 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T06:24:27.612 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T06:24:27.612 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T06:24:27.612 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T06:24:27.612 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T06:24:27.612 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T06:24:27.612 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T06:24:27.613 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T06:24:27.613 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T06:24:27.613 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T06:24:27.613 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T06:24:27.613 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T06:24:27.613 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T06:24:27.613 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T06:24:27.613 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T06:24:27.613 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T06:24:27.613 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T06:24:27.613 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T06:24:27.613 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T06:24:27.614 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T06:24:27.614 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T06:24:27.614 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T06:24:27.614 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T06:24:27.614 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T06:24:27.614 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T06:24:27.614 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T06:24:27.614 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T06:24:27.614 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T06:24:27.614 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T06:24:27.614 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T06:24:27.614 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T06:24:27.614 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T06:24:27.614 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T06:24:27.615 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T06:24:27.615 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T06:24:27.615 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T06:24:27.615 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T06:24:27.615 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T06:24:27.615 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T06:24:27.615 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T06:24:27.615 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T06:24:27.615 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T06:24:27.615 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T06:24:27.615 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T06:24:27.615 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T06:24:27.615 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T06:24:27.616 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T06:24:27.616 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T06:24:27.616 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T06:24:27.616 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T06:24:27.616 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T06:24:27.616 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T06:24:27.616 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T06:24:27.616 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T06:24:27.616 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T06:24:27.616 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T06:24:27.616 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T06:24:27.616 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T06:24:27.616 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T06:24:27.616 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T06:24:27.617 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T06:24:27.617 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T06:24:27.617 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T06:24:27.617 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T06:24:27.617 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T06:24:27.617 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T06:24:27.617 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T06:24:27.617 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T06:24:27.617 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T06:24:27.617 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T06:24:27.617 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T06:24:27.618 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T06:24:27.618 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T06:24:27.618 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T06:24:27.618 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T06:24:27.618 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T06:24:27.618 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T06:24:27.618 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T06:24:27.618 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T06:24:27.618 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T06:24:28.311 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:24:28.311 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T06:24:28.311 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T06:24:28.311 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T06:24:28.311 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T06:24:28.311 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T06:24:28.312 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T06:24:28.312 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T06:24:28.312 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:24:28.312 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:24:28.312 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T06:24:28.312 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T06:24:28.312 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T06:24:28.312 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T06:24:28.312 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:24:28.312 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T06:24:28.312 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T06:24:28.313 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T06:24:28.313 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T06:24:28.313 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T06:24:28.313 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T06:24:28.313 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T06:24:28.313 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T06:24:28.313 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T06:24:28.313 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T06:24:28.313 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T06:24:28.313 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T06:24:28.313 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T06:24:28.313 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T06:24:28.313 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T06:24:28.314 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T06:24:28.314 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T06:24:28.314 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T06:24:28.314 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T06:24:28.314 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T06:24:28.314 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T06:24:28.314 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T06:24:28.314 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T06:24:28.314 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T06:24:28.314 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T06:24:28.314 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T06:24:28.314 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T06:24:28.314 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T06:24:28.315 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T06:24:28.315 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T06:24:28.315 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T06:24:28.315 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T06:24:28.315 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T06:24:28.315 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T06:24:28.315 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T06:24:28.315 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T06:24:28.315 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T06:24:28.315 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T06:24:28.315 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T06:24:28.315 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T06:24:28.315 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T06:24:28.315 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T06:24:28.316 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T06:24:28.316 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T06:24:28.316 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T06:24:28.316 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T06:24:28.316 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T06:24:28.316 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T06:24:28.316 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T06:24:28.316 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T06:24:28.316 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T06:24:28.316 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T06:24:28.316 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T06:24:28.316 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T06:24:28.316 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T06:24:28.316 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T06:24:28.317 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T06:24:28.317 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T06:24:28.317 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T06:24:28.317 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T06:24:28.317 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T06:24:28.317 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T06:24:28.317 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T06:24:28.317 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T06:24:28.317 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T06:24:28.317 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T06:24:28.317 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T06:24:28.317 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T06:24:28.317 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T06:24:28.317 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T06:24:28.318 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T06:24:28.318 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T06:24:28.318 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T06:24:28.318 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T06:24:28.318 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T06:24:28.318 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T06:24:28.318 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T06:24:28.318 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T06:24:28.318 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T06:24:28.318 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T06:24:28.318 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T06:24:28.318 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T06:24:28.318 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T06:24:28.318 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T06:24:28.319 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T06:24:28.319 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T06:24:28.319 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T06:24:28.319 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T06:24:28.319 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T06:24:28.319 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T06:24:28.319 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T06:24:28.319 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T06:24:28.319 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T06:24:28.319 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T06:24:28.319 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T06:24:28.319 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T06:24:28.320 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T06:24:28.320 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T06:24:28.320 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T06:24:28.320 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T06:24:28.320 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T06:24:28.320 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T06:24:28.320 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T06:24:28.320 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T06:24:28.320 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T06:24:28.320 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T06:24:28.320 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T06:24:28.320 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T06:24:28.320 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T06:24:28.320 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T06:24:28.321 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T06:24:28.321 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T06:24:28.321 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T06:24:28.321 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T06:24:28.321 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T06:24:28.321 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T06:24:28.321 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T06:24:28.321 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T06:24:28.321 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T06:24:28.321 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T06:24:28.321 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T06:24:28.321 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T06:24:28.321 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T06:24:28.321 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T06:24:28.321 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T06:24:28.321 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T06:24:28.322 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T06:24:28.322 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T06:24:28.322 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T06:24:28.322 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T06:24:28.322 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T06:24:28.322 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T06:24:28.322 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T06:24:28.322 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T06:24:28.322 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T06:24:28.322 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T06:24:28.322 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T06:24:28.322 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T06:24:28.322 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T06:24:28.323 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T06:24:28.323 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T06:24:28.323 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T06:24:28.323 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T06:24:28.323 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T06:24:28.323 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T06:24:28.323 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T06:24:28.323 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T06:24:28.323 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T06:24:28.323 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T06:24:28.323 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T06:24:30.423 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:24:30.424 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-21T06:24:30.424 INFO:teuthology.orchestra.run.smithi100.stdout:Upgraded: 2024-09-21T06:24:30.424 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T06:24:30.424 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T06:24:30.424 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T06:24:30.424 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T06:24:30.424 INFO:teuthology.orchestra.run.smithi100.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:30.424 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:30.424 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T06:24:30.424 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T06:24:30.424 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-09-21T06:24:30.424 INFO:teuthology.orchestra.run.smithi100.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T06:24:30.425 INFO:teuthology.orchestra.run.smithi100.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T06:24:30.425 INFO:teuthology.orchestra.run.smithi100.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T06:24:30.425 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T06:24:30.425 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:30.425 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:30.425 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:30.425 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:30.425 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:24:30.425 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:30.425 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:30.425 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:30.425 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:24:30.425 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:24:30.425 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:24:30.425 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:24:30.426 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:24:30.426 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:30.426 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:30.426 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:24:30.426 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:30.426 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:30.426 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:30.426 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:24:30.426 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:24:30.426 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:30.426 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:30.426 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:24:30.426 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T06:24:30.427 INFO:teuthology.orchestra.run.smithi100.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T06:24:30.427 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T06:24:30.427 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T06:24:30.427 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T06:24:30.427 INFO:teuthology.orchestra.run.smithi100.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T06:24:30.427 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T06:24:30.427 INFO:teuthology.orchestra.run.smithi100.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T06:24:30.427 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T06:24:30.427 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T06:24:30.427 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T06:24:30.427 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:30.427 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:30.427 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:30.427 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T06:24:30.428 INFO:teuthology.orchestra.run.smithi100.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T06:24:30.428 INFO:teuthology.orchestra.run.smithi100.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T06:24:30.428 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T06:24:30.428 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T06:24:30.428 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T06:24:30.428 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:30.428 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:30.428 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T06:24:30.428 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:30.428 INFO:teuthology.orchestra.run.smithi100.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T06:24:30.428 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T06:24:30.428 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T06:24:30.428 INFO:teuthology.orchestra.run.smithi100.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T06:24:30.428 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T06:24:30.429 INFO:teuthology.orchestra.run.smithi100.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T06:24:30.429 INFO:teuthology.orchestra.run.smithi100.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T06:24:30.429 INFO:teuthology.orchestra.run.smithi100.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T06:24:30.429 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T06:24:30.429 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T06:24:30.429 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T06:24:30.429 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T06:24:30.429 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T06:24:30.429 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T06:24:30.429 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T06:24:30.429 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T06:24:30.429 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T06:24:30.429 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T06:24:30.429 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T06:24:30.430 INFO:teuthology.orchestra.run.smithi100.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T06:24:30.430 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T06:24:30.430 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T06:24:30.430 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T06:24:30.430 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:30.430 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:30.430 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:30.430 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T06:24:30.430 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T06:24:30.430 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T06:24:30.430 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T06:24:30.430 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T06:24:30.430 INFO:teuthology.orchestra.run.smithi100.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T06:24:30.430 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T06:24:30.430 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T06:24:30.431 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T06:24:30.431 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T06:24:30.431 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T06:24:30.431 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T06:24:30.431 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T06:24:30.431 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T06:24:30.431 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T06:24:30.431 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T06:24:30.431 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T06:24:30.431 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T06:24:30.431 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T06:24:30.431 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T06:24:30.431 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T06:24:30.432 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T06:24:30.432 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T06:24:30.432 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T06:24:30.432 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T06:24:30.432 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T06:24:30.432 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T06:24:30.432 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T06:24:30.432 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T06:24:30.432 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T06:24:30.432 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T06:24:30.432 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T06:24:30.432 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T06:24:30.432 INFO:teuthology.orchestra.run.smithi100.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T06:24:30.432 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T06:24:30.433 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T06:24:30.433 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T06:24:30.433 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T06:24:30.433 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T06:24:30.433 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T06:24:30.433 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:30.433 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:30.433 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T06:24:30.433 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T06:24:30.433 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T06:24:30.433 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:30.433 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T06:24:30.433 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T06:24:30.433 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T06:24:30.433 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T06:24:30.434 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T06:24:30.434 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T06:24:30.434 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T06:24:30.434 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T06:24:30.434 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T06:24:30.434 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T06:24:30.434 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T06:24:30.434 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T06:24:30.434 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T06:24:30.434 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T06:24:30.434 INFO:teuthology.orchestra.run.smithi100.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T06:24:30.434 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:30.434 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:30.434 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:30.435 INFO:teuthology.orchestra.run.smithi100.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T06:24:30.435 INFO:teuthology.orchestra.run.smithi100.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T06:24:30.435 INFO:teuthology.orchestra.run.smithi100.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T06:24:30.435 INFO:teuthology.orchestra.run.smithi100.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T06:24:30.435 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T06:24:30.435 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T06:24:30.435 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T06:24:30.435 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-21T06:24:30.436 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-09-21T06:24:30.861 DEBUG:teuthology.parallel:result is None 2024-09-21T06:24:31.053 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:24:31.053 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T06:24:31.053 INFO:teuthology.orchestra.run.smithi203.stdout:Upgraded: 2024-09-21T06:24:31.053 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T06:24:31.053 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T06:24:31.053 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T06:24:31.053 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T06:24:31.053 INFO:teuthology.orchestra.run.smithi203.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:31.053 INFO:teuthology.orchestra.run.smithi203.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:31.054 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T06:24:31.054 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T06:24:31.054 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2024-09-21T06:24:31.054 INFO:teuthology.orchestra.run.smithi203.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T06:24:31.054 INFO:teuthology.orchestra.run.smithi203.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T06:24:31.054 INFO:teuthology.orchestra.run.smithi203.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T06:24:31.054 INFO:teuthology.orchestra.run.smithi203.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T06:24:31.054 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:31.054 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:31.054 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:31.054 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:31.054 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:24:31.054 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:31.054 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:31.055 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:31.055 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:24:31.055 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:24:31.055 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:24:31.055 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:24:31.055 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:24:31.055 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:31.055 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:31.055 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:24:31.055 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:31.055 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:31.055 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:31.055 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:24:31.055 INFO:teuthology.orchestra.run.smithi203.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:24:31.056 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:31.056 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:31.056 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:24:31.056 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T06:24:31.056 INFO:teuthology.orchestra.run.smithi203.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T06:24:31.056 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T06:24:31.056 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T06:24:31.056 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T06:24:31.056 INFO:teuthology.orchestra.run.smithi203.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T06:24:31.056 INFO:teuthology.orchestra.run.smithi203.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T06:24:31.056 INFO:teuthology.orchestra.run.smithi203.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T06:24:31.056 INFO:teuthology.orchestra.run.smithi203.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T06:24:31.056 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T06:24:31.057 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T06:24:31.057 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:31.057 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:31.057 INFO:teuthology.orchestra.run.smithi203.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:31.057 INFO:teuthology.orchestra.run.smithi203.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T06:24:31.057 INFO:teuthology.orchestra.run.smithi203.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T06:24:31.057 INFO:teuthology.orchestra.run.smithi203.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T06:24:31.057 INFO:teuthology.orchestra.run.smithi203.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T06:24:31.057 INFO:teuthology.orchestra.run.smithi203.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T06:24:31.057 INFO:teuthology.orchestra.run.smithi203.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T06:24:31.057 INFO:teuthology.orchestra.run.smithi203.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:31.057 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:31.057 INFO:teuthology.orchestra.run.smithi203.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T06:24:31.057 INFO:teuthology.orchestra.run.smithi203.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:31.058 INFO:teuthology.orchestra.run.smithi203.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T06:24:31.058 INFO:teuthology.orchestra.run.smithi203.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T06:24:31.058 INFO:teuthology.orchestra.run.smithi203.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T06:24:31.058 INFO:teuthology.orchestra.run.smithi203.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T06:24:31.058 INFO:teuthology.orchestra.run.smithi203.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T06:24:31.058 INFO:teuthology.orchestra.run.smithi203.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T06:24:31.058 INFO:teuthology.orchestra.run.smithi203.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T06:24:31.058 INFO:teuthology.orchestra.run.smithi203.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T06:24:31.058 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T06:24:31.058 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T06:24:31.058 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T06:24:31.058 INFO:teuthology.orchestra.run.smithi203.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T06:24:31.058 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T06:24:31.058 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T06:24:31.059 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T06:24:31.059 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T06:24:31.059 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T06:24:31.059 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T06:24:31.059 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T06:24:31.059 INFO:teuthology.orchestra.run.smithi203.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T06:24:31.059 INFO:teuthology.orchestra.run.smithi203.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T06:24:31.059 INFO:teuthology.orchestra.run.smithi203.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T06:24:31.059 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T06:24:31.059 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:31.059 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:31.059 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:31.059 INFO:teuthology.orchestra.run.smithi203.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T06:24:31.060 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T06:24:31.060 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T06:24:31.060 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T06:24:31.060 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T06:24:31.060 INFO:teuthology.orchestra.run.smithi203.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T06:24:31.060 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T06:24:31.060 INFO:teuthology.orchestra.run.smithi203.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T06:24:31.060 INFO:teuthology.orchestra.run.smithi203.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T06:24:31.060 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T06:24:31.060 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T06:24:31.060 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T06:24:31.060 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T06:24:31.060 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T06:24:31.060 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T06:24:31.060 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T06:24:31.061 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T06:24:31.061 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T06:24:31.061 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T06:24:31.061 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T06:24:31.061 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T06:24:31.061 INFO:teuthology.orchestra.run.smithi203.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T06:24:31.061 INFO:teuthology.orchestra.run.smithi203.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T06:24:31.061 INFO:teuthology.orchestra.run.smithi203.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T06:24:31.061 INFO:teuthology.orchestra.run.smithi203.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T06:24:31.061 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T06:24:31.061 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T06:24:31.061 INFO:teuthology.orchestra.run.smithi203.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T06:24:31.061 INFO:teuthology.orchestra.run.smithi203.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T06:24:31.061 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T06:24:31.062 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T06:24:31.062 INFO:teuthology.orchestra.run.smithi203.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T06:24:31.062 INFO:teuthology.orchestra.run.smithi203.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T06:24:31.062 INFO:teuthology.orchestra.run.smithi203.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T06:24:31.062 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T06:24:31.062 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T06:24:31.062 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T06:24:31.062 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T06:24:31.062 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T06:24:31.062 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T06:24:31.062 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:31.062 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:31.062 INFO:teuthology.orchestra.run.smithi203.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T06:24:31.062 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T06:24:31.063 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T06:24:31.063 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:31.063 INFO:teuthology.orchestra.run.smithi203.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T06:24:31.063 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T06:24:31.063 INFO:teuthology.orchestra.run.smithi203.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T06:24:31.063 INFO:teuthology.orchestra.run.smithi203.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T06:24:31.063 INFO:teuthology.orchestra.run.smithi203.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T06:24:31.063 INFO:teuthology.orchestra.run.smithi203.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T06:24:31.063 INFO:teuthology.orchestra.run.smithi203.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T06:24:31.063 INFO:teuthology.orchestra.run.smithi203.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T06:24:31.063 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T06:24:31.063 INFO:teuthology.orchestra.run.smithi203.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T06:24:31.063 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T06:24:31.063 INFO:teuthology.orchestra.run.smithi203.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T06:24:31.063 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T06:24:31.064 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T06:24:31.064 INFO:teuthology.orchestra.run.smithi203.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T06:24:31.064 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:31.064 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:31.064 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:24:31.064 INFO:teuthology.orchestra.run.smithi203.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T06:24:31.064 INFO:teuthology.orchestra.run.smithi203.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T06:24:31.064 INFO:teuthology.orchestra.run.smithi203.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T06:24:31.064 INFO:teuthology.orchestra.run.smithi203.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T06:24:31.064 INFO:teuthology.orchestra.run.smithi203.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T06:24:31.064 INFO:teuthology.orchestra.run.smithi203.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T06:24:31.065 INFO:teuthology.orchestra.run.smithi203.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T06:24:31.065 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T06:24:31.065 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-09-21T06:24:31.514 DEBUG:teuthology.parallel:result is None 2024-09-21T06:24:31.514 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:24:31.658 DEBUG:teuthology.orchestra.run.smithi067:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T06:24:31.686 INFO:teuthology.orchestra.run.smithi067.stdout:19.3.0-5120.g18432218.el9 2024-09-21T06:24:31.686 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T06:24:31.686 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T06:24:31.688 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:24:31.833 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T06:24:31.857 INFO:teuthology.orchestra.run.smithi100.stdout:19.3.0-5120.g18432218.el9 2024-09-21T06:24:31.858 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T06:24:31.858 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T06:24:31.860 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:24:32.001 DEBUG:teuthology.orchestra.run.smithi203:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T06:24:32.024 INFO:teuthology.orchestra.run.smithi203.stdout:19.3.0-5120.g18432218.el9 2024-09-21T06:24:32.025 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T06:24:32.025 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T06:24:32.026 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T06:24:32.026 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-21T06:24:32.027 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T06:24:32.060 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-21T06:24:32.060 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T06:24:32.090 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-21T06:24:32.091 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T06:24:32.119 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T06:24:32.120 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-21T06:24:32.120 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T06:24:32.150 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T06:24:32.220 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-21T06:24:32.220 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T06:24:32.247 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T06:24:32.315 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-21T06:24:32.315 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T06:24:32.342 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T06:24:32.408 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T06:24:32.409 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-21T06:24:32.409 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T06:24:32.441 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T06:24:32.510 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-21T06:24:32.510 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T06:24:32.538 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T06:24:32.604 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-21T06:24:32.604 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T06:24:32.632 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T06:24:32.698 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T06:24:32.699 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-21T06:24:32.699 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T06:24:32.725 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T06:24:32.792 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-21T06:24:32.792 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T06:24:32.819 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T06:24:32.885 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-21T06:24:32.885 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T06:24:32.913 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T06:24:32.979 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-21T06:24:33.066 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 1, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'subvols': {'create': 2}} 2024-09-21T06:24:33.066 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:24:33.066 INFO:tasks.cephadm:Cluster fsid is 2a791768-77e2-11ef-baf5-efdc52797490 2024-09-21T06:24:33.066 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-21T06:24:33.067 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.67', 'mon.b': '172.21.15.100', 'mon.c': '172.21.15.203'} 2024-09-21T06:24:33.067 INFO:tasks.cephadm:First mon is mon.a on smithi067 2024-09-21T06:24:33.067 INFO:tasks.cephadm:First mgr is x 2024-09-21T06:24:33.067 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-21T06:24:33.067 DEBUG:teuthology.orchestra.run.smithi067:> sudo hostname $(hostname -s) 2024-09-21T06:24:33.094 DEBUG:teuthology.orchestra.run.smithi100:> sudo hostname $(hostname -s) 2024-09-21T06:24:33.120 DEBUG:teuthology.orchestra.run.smithi203:> sudo hostname $(hostname -s) 2024-09-21T06:24:33.146 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-21T06:24:33.147 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:24:33.293 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.4.75+confusa13', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-21 00:04:17.484514', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['x86_64', 'source', 'arm64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.2.22+braggi22', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-20 22:54:16.499090', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-21T06:24:33.433 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-pdonnell-testing-20240920.212106-debug, sha1 1843221895adccfb3b692060251dd6a0183bc410 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:24:33.435 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T06:24:33.435 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T06:24:33.435 DEBUG:teuthology.orchestra.run.smithi067:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T06:24:33.977 INFO:teuthology.orchestra.run.smithi067.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 06:24 /home/ubuntu/cephtest/cephadm 2024-09-21T06:24:33.977 DEBUG:teuthology.orchestra.run.smithi100:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T06:24:34.880 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 06:24 /home/ubuntu/cephtest/cephadm 2024-09-21T06:24:34.880 DEBUG:teuthology.orchestra.run.smithi203:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T06:24:35.436 INFO:teuthology.orchestra.run.smithi203.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 06:24 /home/ubuntu/cephtest/cephadm 2024-09-21T06:24:35.436 DEBUG:teuthology.orchestra.run.smithi067:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T06:24:35.457 DEBUG:teuthology.orchestra.run.smithi100:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T06:24:35.478 DEBUG:teuthology.orchestra.run.smithi203:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T06:24:35.512 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 on all hosts... 2024-09-21T06:24:35.513 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T06:24:35.515 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T06:24:35.520 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T06:24:37.116 INFO:teuthology.orchestra.run.smithi203.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T06:24:37.144 INFO:teuthology.orchestra.run.smithi067.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T06:24:37.163 INFO:teuthology.orchestra.run.smithi100.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T06:25:30.156 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-21T06:25:30.156 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T06:25:30.156 INFO:teuthology.orchestra.run.smithi100.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T06:25:30.156 INFO:teuthology.orchestra.run.smithi100.stdout: "repo_digests": [ 2024-09-21T06:25:30.156 INFO:teuthology.orchestra.run.smithi100.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T06:25:30.156 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-21T06:25:30.156 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-21T06:25:36.637 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-21T06:25:36.637 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T06:25:36.637 INFO:teuthology.orchestra.run.smithi067.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T06:25:36.637 INFO:teuthology.orchestra.run.smithi067.stdout: "repo_digests": [ 2024-09-21T06:25:36.637 INFO:teuthology.orchestra.run.smithi067.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T06:25:36.637 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-21T06:25:36.637 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-21T06:26:00.660 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2024-09-21T06:26:00.661 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T06:26:00.661 INFO:teuthology.orchestra.run.smithi203.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T06:26:00.661 INFO:teuthology.orchestra.run.smithi203.stdout: "repo_digests": [ 2024-09-21T06:26:00.661 INFO:teuthology.orchestra.run.smithi203.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T06:26:00.661 INFO:teuthology.orchestra.run.smithi203.stdout: ] 2024-09-21T06:26:00.661 INFO:teuthology.orchestra.run.smithi203.stdout:} 2024-09-21T06:26:00.687 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /etc/ceph 2024-09-21T06:26:00.723 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /etc/ceph 2024-09-21T06:26:00.757 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /etc/ceph 2024-09-21T06:26:00.784 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 777 /etc/ceph 2024-09-21T06:26:00.814 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 777 /etc/ceph 2024-09-21T06:26:00.842 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 777 /etc/ceph 2024-09-21T06:26:00.867 INFO:tasks.cephadm:Writing seed config... 2024-09-21T06:26:00.868 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-21T06:26:00.868 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-21T06:26:00.868 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-21T06:26:00.868 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-21T06:26:00.868 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-21T06:26:00.868 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-21T06:26:00.868 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-21T06:26:00.868 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-21T06:26:00.868 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-21T06:26:00.868 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-21T06:26:00.868 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-21T06:26:00.868 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-21T06:26:00.869 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-21T06:26:00.869 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-21T06:26:00.869 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-21T06:26:00.869 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-21T06:26:00.869 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-21T06:26:00.869 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-21T06:26:00.869 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-21T06:26:00.869 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-21T06:26:00.869 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-21T06:26:00.869 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-21T06:26:00.870 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-21T06:26:00.870 DEBUG:teuthology.orchestra.run.smithi067:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-21T06:26:00.888 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 2a791768-77e2-11ef-baf5-efdc52797490 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-21T06:26:00.888 DEBUG:teuthology.orchestra.run.smithi067:mon.a> sudo journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mon.a.service 2024-09-21T06:26:00.931 DEBUG:teuthology.orchestra.run.smithi067:mgr.x> sudo journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mgr.x.service 2024-09-21T06:26:00.974 INFO:tasks.cephadm:Bootstrapping... 2024-09-21T06:26:00.974 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 -v bootstrap --fsid 2a791768-77e2-11ef-baf5-efdc52797490 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.67 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:26:01.192 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-09-21T06:26:01.193 INFO:teuthology.orchestra.run.smithi067.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', '-v', 'bootstrap', '--fsid', '2a791768-77e2-11ef-baf5-efdc52797490', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.67', '--skip-admin-label'] 2024-09-21T06:26:01.193 INFO:teuthology.orchestra.run.smithi067.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-21T06:26:01.193 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying podman|docker is present... 2024-09-21T06:26:01.218 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stdout 5.2.2 2024-09-21T06:26:01.218 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying lvm2 is present... 2024-09-21T06:26:01.218 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying time synchronization is in place... 2024-09-21T06:26:01.226 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T06:26:01.226 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T06:26:01.232 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T06:26:01.232 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-09-21T06:26:01.239 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout enabled 2024-09-21T06:26:01.246 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout active 2024-09-21T06:26:01.246 INFO:teuthology.orchestra.run.smithi067.stdout:Unit chronyd.service is enabled and running 2024-09-21T06:26:01.246 INFO:teuthology.orchestra.run.smithi067.stdout:Repeating the final host check... 2024-09-21T06:26:01.269 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stdout 5.2.2 2024-09-21T06:26:01.269 INFO:teuthology.orchestra.run.smithi067.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-21T06:26:01.269 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl is present 2024-09-21T06:26:01.269 INFO:teuthology.orchestra.run.smithi067.stdout:lvcreate is present 2024-09-21T06:26:01.276 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T06:26:01.276 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T06:26:01.283 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T06:26:01.283 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-09-21T06:26:01.289 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout enabled 2024-09-21T06:26:01.297 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout active 2024-09-21T06:26:01.297 INFO:teuthology.orchestra.run.smithi067.stdout:Unit chronyd.service is enabled and running 2024-09-21T06:26:01.297 INFO:teuthology.orchestra.run.smithi067.stdout:Host looks OK 2024-09-21T06:26:01.297 INFO:teuthology.orchestra.run.smithi067.stdout:Cluster fsid: 2a791768-77e2-11ef-baf5-efdc52797490 2024-09-21T06:26:01.298 INFO:teuthology.orchestra.run.smithi067.stdout:Acquiring lock 140282296879472 on /run/cephadm/2a791768-77e2-11ef-baf5-efdc52797490.lock 2024-09-21T06:26:01.298 INFO:teuthology.orchestra.run.smithi067.stdout:Lock 140282296879472 acquired on /run/cephadm/2a791768-77e2-11ef-baf5-efdc52797490.lock 2024-09-21T06:26:01.298 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying IP 172.21.15.67 port 3300 ... 2024-09-21T06:26:01.299 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying IP 172.21.15.67 port 6789 ... 2024-09-21T06:26:01.299 INFO:teuthology.orchestra.run.smithi067.stdout:Base mon IP(s) is [172.21.15.67:3300, 172.21.15.67:6789], mon addrv is [v2:172.21.15.67:3300,v1:172.21.15.67:6789] 2024-09-21T06:26:01.301 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.67 metric 100 2024-09-21T06:26:01.301 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.67 metric 100 2024-09-21T06:26:01.304 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-21T06:26:01.304 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-21T06:26:01.304 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-21T06:26:01.306 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-21T06:26:01.306 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-21T06:26:01.306 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T06:26:01.306 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-21T06:26:01.306 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link noprefixroute 2024-09-21T06:26:01.306 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T06:26:01.307 INFO:teuthology.orchestra.run.smithi067.stdout:Mon IP `172.21.15.67` is in CIDR network `172.21.0.0/20` 2024-09-21T06:26:01.307 INFO:teuthology.orchestra.run.smithi067.stdout:Mon IP `172.21.15.67` is in CIDR network `172.21.0.0/20` 2024-09-21T06:26:01.307 INFO:teuthology.orchestra.run.smithi067.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-21T06:26:01.308 INFO:teuthology.orchestra.run.smithi067.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-21T06:26:01.308 INFO:teuthology.orchestra.run.smithi067.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T06:26:01.773 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stdout ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T06:26:01.773 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T06:26:01.774 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Getting image source signatures 2024-09-21T06:26:01.774 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Copying blob sha256:8425bc7408d91b180e7fdc62f23e7ae6603757f4f36e9969a4eaf9fd9416b764 2024-09-21T06:26:01.774 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-21T06:26:01.774 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Copying config sha256:ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T06:26:01.774 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-21T06:26:02.662 INFO:teuthology.orchestra.run.smithi067.stdout:ceph: stdout ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T06:26:02.663 INFO:teuthology.orchestra.run.smithi067.stdout:Ceph version: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T06:26:02.663 INFO:teuthology.orchestra.run.smithi067.stdout:Extracting ceph user uid/gid from container image... 2024-09-21T06:26:03.802 INFO:teuthology.orchestra.run.smithi067.stdout:stat: stdout 167 167 2024-09-21T06:26:03.802 INFO:teuthology.orchestra.run.smithi067.stdout:Creating initial keys... 2024-09-21T06:26:09.119 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph-authtool: stdout AQD8Zu5mpkBVBxAAK1ey6AkIOoR2UtyD1fUD0Q== 2024-09-21T06:26:10.916 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph-authtool: stdout AQACZ+5meDK7EhAARHhGvps9lnDTmH+dtOtvGQ== 2024-09-21T06:26:11.822 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph-authtool: stdout AQADZ+5mrCmhDxAAULkgcTZpGv8DJKIakxGPNA== 2024-09-21T06:26:11.823 INFO:teuthology.orchestra.run.smithi067.stdout:Creating initial monmap... 2024-09-21T06:26:12.740 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T06:26:12.740 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-21T06:26:12.740 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 2a791768-77e2-11ef-baf5-efdc52797490 2024-09-21T06:26:12.740 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T06:26:12.740 INFO:teuthology.orchestra.run.smithi067.stdout:monmaptool for a [v2:172.21.15.67:3300,v1:172.21.15.67:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T06:26:12.740 INFO:teuthology.orchestra.run.smithi067.stdout:setting min_mon_release = quincy 2024-09-21T06:26:12.740 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: set fsid to 2a791768-77e2-11ef-baf5-efdc52797490 2024-09-21T06:26:12.740 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T06:26:12.741 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:26:12.741 INFO:teuthology.orchestra.run.smithi067.stdout:Creating mon... 2024-09-21T06:26:13.693 INFO:teuthology.orchestra.run.smithi067.stdout:create mon.a on 2024-09-21T06:26:13.904 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-21T06:26:14.132 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-21T06:26:14.340 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-2a791768-77e2-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-2a791768-77e2-11ef-baf5-efdc52797490.target. 2024-09-21T06:26:14.340 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-2a791768-77e2-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-2a791768-77e2-11ef-baf5-efdc52797490.target. 2024-09-21T06:26:14.577 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2a791768-77e2-11ef-baf5-efdc52797490@mon.a 2024-09-21T06:26:14.577 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2a791768-77e2-11ef-baf5-efdc52797490@mon.a.service: Unit ceph-2a791768-77e2-11ef-baf5-efdc52797490@mon.a.service not loaded. 2024-09-21T06:26:14.752 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2a791768-77e2-11ef-baf5-efdc52797490.target.wants/ceph-2a791768-77e2-11ef-baf5-efdc52797490@mon.a.service → /etc/systemd/system/ceph-2a791768-77e2-11ef-baf5-efdc52797490@.service. 2024-09-21T06:26:15.226 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:15 smithi067 podman[30844]: 2024-09-21 06:26:15.171473518 +0000 UTC m=+0.304246952 container start 1783d05ebb975813db507c51527206d02c3df913cd3735ac2e83ba779ec59c69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mon-a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-21T06:26:15.226 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:15 smithi067 ceph-mon[30863]: mkfs 2a791768-77e2-11ef-baf5-efdc52797490 2024-09-21T06:26:15.267 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T06:26:15.267 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout disabled 2024-09-21T06:26:15.274 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T06:26:15.275 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-09-21T06:26:15.275 INFO:teuthology.orchestra.run.smithi067.stdout:firewalld.service is not enabled 2024-09-21T06:26:15.275 INFO:teuthology.orchestra.run.smithi067.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T06:26:15.275 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mon to start... 2024-09-21T06:26:15.275 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mon... 2024-09-21T06:26:15.484 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:15 smithi067 ceph-mon[30863]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T06:26:15.485 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:15 smithi067 bash[30844]: 1783d05ebb975813db507c51527206d02c3df913cd3735ac2e83ba779ec59c69 2024-09-21T06:26:15.485 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:15 smithi067 systemd[1]: Started Ceph mon.a for 2a791768-77e2-11ef-baf5-efdc52797490. 2024-09-21T06:26:16.405 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout cluster: 2024-09-21T06:26:16.406 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout id: 2a791768-77e2-11ef-baf5-efdc52797490 2024-09-21T06:26:16.406 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-21T06:26:16.406 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-09-21T06:26:16.406 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout services: 2024-09-21T06:26:16.406 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.657719s) 2024-09-21T06:26:16.406 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-21T06:26:16.406 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-21T06:26:16.406 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-09-21T06:26:16.406 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout data: 2024-09-21T06:26:16.406 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-21T06:26:16.406 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-21T06:26:16.406 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-21T06:26:16.406 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout pgs: 2024-09-21T06:26:16.407 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-09-21T06:26:16.407 INFO:teuthology.orchestra.run.smithi067.stdout:mon is available 2024-09-21T06:26:16.407 INFO:teuthology.orchestra.run.smithi067.stdout:Assimilating anything we can from ceph.conf... 2024-09-21T06:26:16.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:16 smithi067 ceph-mon[30863]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T06:26:16.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:16 smithi067 ceph-mon[30863]: monmap epoch 1 2024-09-21T06:26:16.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:16 smithi067 ceph-mon[30863]: fsid 2a791768-77e2-11ef-baf5-efdc52797490 2024-09-21T06:26:16.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:16 smithi067 ceph-mon[30863]: last_changed 2024-09-21T06:26:12.167159+0000 2024-09-21T06:26:16.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:16 smithi067 ceph-mon[30863]: created 2024-09-21T06:26:12.167159+0000 2024-09-21T06:26:16.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:16 smithi067 ceph-mon[30863]: min_mon_release 19 (squid) 2024-09-21T06:26:16.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:16 smithi067 ceph-mon[30863]: election_strategy: 1 2024-09-21T06:26:16.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:16 smithi067 ceph-mon[30863]: 0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-09-21T06:26:16.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:16 smithi067 ceph-mon[30863]: fsmap 2024-09-21T06:26:16.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:16 smithi067 ceph-mon[30863]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T06:26:16.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:16 smithi067 ceph-mon[30863]: mgrmap e1: no daemons active 2024-09-21T06:26:16.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:16 smithi067 ceph-mon[30863]: from='client.? 172.21.15.67:0/3579407219' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-21T06:26:17.507 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:17 smithi067 ceph-mon[30863]: from='client.? 172.21.15.67:0/3197561547' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T06:26:17.507 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:17 smithi067 ceph-mon[30863]: from='client.? 172.21.15.67:0/3197561547' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-21T06:26:17.511 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-09-21T06:26:17.511 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T06:26:17.511 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout fsid = 2a791768-77e2-11ef-baf5-efdc52797490 2024-09-21T06:26:17.511 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T06:26:17.511 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.67:3300,v1:172.21.15.67:6789] 2024-09-21T06:26:17.511 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T06:26:17.511 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T06:26:17.512 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T06:26:17.512 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T06:26:17.512 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-09-21T06:26:17.512 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T06:26:17.512 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T06:26:17.512 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-09-21T06:26:17.512 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T06:26:17.512 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T06:26:17.512 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T06:26:17.512 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T06:26:17.512 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T06:26:17.512 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T06:26:17.512 INFO:teuthology.orchestra.run.smithi067.stdout:Generating new minimal ceph.conf... 2024-09-21T06:26:18.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:18 smithi067 ceph-mon[30863]: from='client.? 172.21.15.67:0/1889843708' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:26:18.600 INFO:teuthology.orchestra.run.smithi067.stdout:Restarting the monitor... 2024-09-21T06:26:18.876 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:18 smithi067 systemd[1]: Stopping Ceph mon.a for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:26:19.234 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:18 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mon-a[30859]: 2024-09-21T06:26:18.873+0000 7f246c2f9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:26:19.234 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:18 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mon-a[30859]: 2024-09-21T06:26:18.873+0000 7f246c2f9640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-21T06:26:19.234 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:18 smithi067 podman[31105]: 2024-09-21 06:26:18.997564145 +0000 UTC m=+0.223197895 container died 1783d05ebb975813db507c51527206d02c3df913cd3735ac2e83ba779ec59c69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mon-a, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-21T06:26:19.509 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:19 smithi067 podman[31105]: 2024-09-21 06:26:19.331943308 +0000 UTC m=+0.557577037 container cleanup 1783d05ebb975813db507c51527206d02c3df913cd3735ac2e83ba779ec59c69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mon-a, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T06:26:19.509 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:19 smithi067 bash[31105]: ceph-2a791768-77e2-11ef-baf5-efdc52797490-mon-a 2024-09-21T06:26:19.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:19 smithi067 podman[31117]: 2024-09-21 06:26:19.507678603 +0000 UTC m=+0.504125589 container remove 1783d05ebb975813db507c51527206d02c3df913cd3735ac2e83ba779ec59c69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-21T06:26:19.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:19 smithi067 systemd[1]: ceph-2a791768-77e2-11ef-baf5-efdc52797490@mon.a.service: Deactivated successfully. 2024-09-21T06:26:19.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:19 smithi067 systemd[1]: Stopped Ceph mon.a for 2a791768-77e2-11ef-baf5-efdc52797490. 2024-09-21T06:26:19.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:19 smithi067 systemd[1]: Starting Ceph mon.a for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:26:20.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:19 smithi067 podman[31213]: 2024-09-21 06:26:19.941889077 +0000 UTC m=+0.100970177 container create 853c94eacb0a03af1e73ff829802dde383540b0848e22c2197cbdf6e7c29de18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mon-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-21T06:26:20.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:19 smithi067 podman[31213]: 2024-09-21 06:26:19.857232288 +0000 UTC m=+0.016313406 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:26:20.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 podman[31213]: 2024-09-21 06:26:20.219542318 +0000 UTC m=+0.378623427 container init 853c94eacb0a03af1e73ff829802dde383540b0848e22c2197cbdf6e7c29de18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731) 2024-09-21T06:26:20.300 INFO:teuthology.orchestra.run.smithi067.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-21T06:26:20.515 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 podman[31213]: 2024-09-21 06:26:20.222921017 +0000 UTC m=+0.382002118 container start 853c94eacb0a03af1e73ff829802dde383540b0848e22c2197cbdf6e7c29de18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mon-a, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:26:20.515 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T06:26:20.515 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T06:26:20.515 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: pidfile_write: ignore empty --pid-file 2024-09-21T06:26:20.515 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: load: jerasure load: lrc 2024-09-21T06:26:20.515 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: RocksDB version: 7.9.2 2024-09-21T06:26:20.515 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Git sha 0 2024-09-21T06:26:20.515 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T06:26:20.515 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: DB SUMMARY 2024-09-21T06:26:20.515 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: DB Session ID: QEW40Z8S4QLZM4QNWY3O 2024-09-21T06:26:20.516 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: CURRENT file: CURRENT 2024-09-21T06:26:20.516 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T06:26:20.516 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-21T06:26:20.516 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-21T06:26:20.516 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 93734 ; 2024-09-21T06:26:20.516 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.error_if_exists: 0 2024-09-21T06:26:20.516 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.create_if_missing: 0 2024-09-21T06:26:20.516 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.paranoid_checks: 1 2024-09-21T06:26:20.516 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T06:26:20.516 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T06:26:20.516 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T06:26:20.516 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.env: 0x564e99d71d60 2024-09-21T06:26:20.516 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T06:26:20.516 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.info_log: 0x564e9fee3360 2024-09-21T06:26:20.517 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T06:26:20.517 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.statistics: (nil) 2024-09-21T06:26:20.517 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.use_fsync: 0 2024-09-21T06:26:20.517 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.max_log_file_size: 0 2024-09-21T06:26:20.517 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T06:26:20.517 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T06:26:20.517 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T06:26:20.517 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T06:26:20.517 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.allow_fallocate: 1 2024-09-21T06:26:20.518 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T06:26:20.518 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T06:26:20.518 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.use_direct_reads: 0 2024-09-21T06:26:20.518 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T06:26:20.519 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T06:26:20.519 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.db_log_dir: 2024-09-21T06:26:20.519 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.wal_dir: 2024-09-21T06:26:20.519 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T06:26:20.519 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T06:26:20.519 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T06:26:20.519 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T06:26:20.519 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T06:26:20.519 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T06:26:20.519 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T06:26:20.519 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T06:26:20.519 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.write_buffer_manager: 0x564e9fee7d60 2024-09-21T06:26:20.519 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T06:26:20.519 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T06:26:20.520 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T06:26:20.520 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T06:26:20.520 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T06:26:20.520 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T06:26:20.520 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T06:26:20.520 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T06:26:20.520 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.unordered_write: 0 2024-09-21T06:26:20.520 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T06:26:20.520 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T06:26:20.520 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T06:26:20.520 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T06:26:20.520 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.row_cache: None 2024-09-21T06:26:20.520 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.wal_filter: None 2024-09-21T06:26:20.520 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T06:26:20.521 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T06:26:20.521 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.two_write_queues: 0 2024-09-21T06:26:20.521 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T06:26:20.521 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.wal_compression: 0 2024-09-21T06:26:20.521 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.atomic_flush: 0 2024-09-21T06:26:20.521 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T06:26:20.521 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T06:26:20.521 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T06:26:20.521 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.log_readahead_size: 0 2024-09-21T06:26:20.521 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T06:26:20.521 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T06:26:20.521 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T06:26:20.521 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T06:26:20.522 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T06:26:20.522 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T06:26:20.522 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T06:26:20.522 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.max_background_jobs: 2 2024-09-21T06:26:20.522 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.max_background_compactions: -1 2024-09-21T06:26:20.522 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.max_subcompactions: 1 2024-09-21T06:26:20.522 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T06:26:20.522 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T06:26:20.522 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T06:26:20.522 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T06:26:20.522 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T06:26:20.522 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T06:26:20.522 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T06:26:20.522 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T06:26:20.523 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.max_open_files: -1 2024-09-21T06:26:20.523 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T06:26:20.523 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T06:26:20.523 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T06:26:20.523 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T06:26:20.523 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.max_background_flushes: -1 2024-09-21T06:26:20.523 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Compression algorithms supported: 2024-09-21T06:26:20.523 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: kZSTD supported: 0 2024-09-21T06:26:20.523 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: kXpressCompression supported: 0 2024-09-21T06:26:20.523 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: kBZip2Compression supported: 0 2024-09-21T06:26:20.523 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T06:26:20.523 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: kLZ4Compression supported: 1 2024-09-21T06:26:20.523 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: kZlibCompression supported: 1 2024-09-21T06:26:20.523 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T06:26:20.523 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: kSnappyCompression supported: 1 2024-09-21T06:26:20.524 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T06:26:20.524 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T06:26:20.524 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-21T06:26:20.524 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T06:26:20.524 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T06:26:20.524 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.merge_operator: 2024-09-21T06:26:20.524 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.compaction_filter: None 2024-09-21T06:26:20.524 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T06:26:20.524 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T06:26:20.524 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T06:26:20.524 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T06:26:20.524 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x564e9fee2fa0) 2024-09-21T06:26:20.524 INFO:journalctl@ceph.mon.a.smithi067.stdout: cache_index_and_filter_blocks: 1 2024-09-21T06:26:20.524 INFO:journalctl@ceph.mon.a.smithi067.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T06:26:20.524 INFO:journalctl@ceph.mon.a.smithi067.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T06:26:20.525 INFO:journalctl@ceph.mon.a.smithi067.stdout: pin_top_level_index_and_filter: 1 2024-09-21T06:26:20.525 INFO:journalctl@ceph.mon.a.smithi067.stdout: index_type: 0 2024-09-21T06:26:20.525 INFO:journalctl@ceph.mon.a.smithi067.stdout: data_block_index_type: 0 2024-09-21T06:26:20.525 INFO:journalctl@ceph.mon.a.smithi067.stdout: index_shortening: 1 2024-09-21T06:26:20.525 INFO:journalctl@ceph.mon.a.smithi067.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T06:26:20.525 INFO:journalctl@ceph.mon.a.smithi067.stdout: checksum: 4 2024-09-21T06:26:20.525 INFO:journalctl@ceph.mon.a.smithi067.stdout: no_block_cache: 0 2024-09-21T06:26:20.525 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_cache: 0x564e9bea7610 2024-09-21T06:26:20.525 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_cache_name: BinnedLRUCache 2024-09-21T06:26:20.525 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_cache_options: 2024-09-21T06:26:20.525 INFO:journalctl@ceph.mon.a.smithi067.stdout: capacity : 536870912 2024-09-21T06:26:20.525 INFO:journalctl@ceph.mon.a.smithi067.stdout: num_shard_bits : 4 2024-09-21T06:26:20.525 INFO:journalctl@ceph.mon.a.smithi067.stdout: strict_capacity_limit : 0 2024-09-21T06:26:20.525 INFO:journalctl@ceph.mon.a.smithi067.stdout: high_pri_pool_ratio: 0.000 2024-09-21T06:26:20.526 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_cache_compressed: (nil) 2024-09-21T06:26:20.526 INFO:journalctl@ceph.mon.a.smithi067.stdout: persistent_cache: (nil) 2024-09-21T06:26:20.526 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_size: 4096 2024-09-21T06:26:20.526 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_size_deviation: 10 2024-09-21T06:26:20.526 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_restart_interval: 16 2024-09-21T06:26:20.526 INFO:journalctl@ceph.mon.a.smithi067.stdout: index_block_restart_interval: 1 2024-09-21T06:26:20.526 INFO:journalctl@ceph.mon.a.smithi067.stdout: metadata_block_size: 4096 2024-09-21T06:26:20.526 INFO:journalctl@ceph.mon.a.smithi067.stdout: partition_filters: 0 2024-09-21T06:26:20.526 INFO:journalctl@ceph.mon.a.smithi067.stdout: use_delta_encoding: 1 2024-09-21T06:26:20.526 INFO:journalctl@ceph.mon.a.smithi067.stdout: filter_policy: bloomfilter 2024-09-21T06:26:20.526 INFO:journalctl@ceph.mon.a.smithi067.stdout: whole_key_filtering: 1 2024-09-21T06:26:20.526 INFO:journalctl@ceph.mon.a.smithi067.stdout: verify_compression: 0 2024-09-21T06:26:20.526 INFO:journalctl@ceph.mon.a.smithi067.stdout: read_amp_bytes_per_bit: 0 2024-09-21T06:26:20.526 INFO:journalctl@ceph.mon.a.smithi067.stdout: format_version: 5 2024-09-21T06:26:20.526 INFO:journalctl@ceph.mon.a.smithi067.stdout: enable_index_compression: 1 2024-09-21T06:26:20.527 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_align: 0 2024-09-21T06:26:20.527 INFO:journalctl@ceph.mon.a.smithi067.stdout: max_auto_readahead_size: 262144 2024-09-21T06:26:20.527 INFO:journalctl@ceph.mon.a.smithi067.stdout: prepopulate_block_cache: 0 2024-09-21T06:26:20.527 INFO:journalctl@ceph.mon.a.smithi067.stdout: initial_auto_readahead_size: 8192 2024-09-21T06:26:20.527 INFO:journalctl@ceph.mon.a.smithi067.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T06:26:20.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T06:26:20.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T06:26:20.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.compression: NoCompression 2024-09-21T06:26:20.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T06:26:20.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T06:26:20.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T06:26:20.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.num_levels: 7 2024-09-21T06:26:20.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T06:26:20.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T06:26:20.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T06:26:20.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T06:26:20.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T06:26:20.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T06:26:20.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T06:26:20.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T06:26:20.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T06:26:20.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T06:26:20.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T06:26:20.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T06:26:20.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T06:26:20.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T06:26:20.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T06:26:20.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T06:26:20.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T06:26:20.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T06:26:20.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T06:26:20.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T06:26:20.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T06:26:20.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T06:26:20.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T06:26:20.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T06:26:20.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T06:26:20.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T06:26:20.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T06:26:20.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T06:26:20.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T06:26:20.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T06:26:20.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T06:26:20.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T06:26:20.530 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T06:26:20.530 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T06:26:20.530 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T06:26:20.530 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T06:26:20.530 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T06:26:20.530 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T06:26:20.530 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T06:26:20.530 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T06:26:20.530 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T06:26:20.530 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T06:26:20.530 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T06:26:20.530 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T06:26:20.530 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T06:26:20.530 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T06:26:20.531 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T06:26:20.531 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T06:26:20.531 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T06:26:20.531 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T06:26:20.531 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T06:26:20.531 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T06:26:20.531 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T06:26:20.531 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T06:26:20.531 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.inplace_update_support: 0 2024-09-21T06:26:20.531 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T06:26:20.531 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T06:26:20.531 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T06:26:20.531 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T06:26:20.531 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.bloom_locality: 0 2024-09-21T06:26:20.531 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.max_successive_merges: 0 2024-09-21T06:26:20.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T06:26:20.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T06:26:20.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T06:26:20.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T06:26:20.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.ttl: 2592000 2024-09-21T06:26:20.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T06:26:20.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T06:26:20.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T06:26:20.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.enable_blob_files: false 2024-09-21T06:26:20.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.min_blob_size: 0 2024-09-21T06:26:20.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T06:26:20.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T06:26:20.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T06:26:20.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T06:26:20.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T06:26:20.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T06:26:20.533 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T06:26:20.533 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T06:26:20.533 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-21T06:26:20.533 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-21T06:26:20.533 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e65350c6-8fde-4cba-a453-a7be90b63cc5 2024-09-21T06:26:20.533 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726899980261103, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-21T06:26:20.533 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-21T06:26:20.533 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726899980262343, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 90020, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 279, "table_properties": {"data_size": 88090, "index_size": 250, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11329, "raw_average_key_size": 47, "raw_value_size": 81682, "raw_average_value_size": 343, "num_data_blocks": 11, "num_entries": 238, "num_filter_entries": 238, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726899980, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e65350c6-8fde-4cba-a453-a7be90b63cc5", "db_session_id": "QEW40Z8S4QLZM4QNWY3O", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-21T06:26:20.533 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726899980262462, "job": 1, "event": "recovery_finished"} 2024-09-21T06:26:20.533 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-21T06:26:20.533 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T06:26:20.533 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x564e9ff06a80 2024-09-21T06:26:20.533 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: DB pointer 0x564ea0cac000 2024-09-21T06:26:20.533 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T06:26:20.534 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T06:26:20.534 INFO:journalctl@ceph.mon.a.smithi067.stdout: ** DB Stats ** 2024-09-21T06:26:20.534 INFO:journalctl@ceph.mon.a.smithi067.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T06:26:20.534 INFO:journalctl@ceph.mon.a.smithi067.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T06:26:20.534 INFO:journalctl@ceph.mon.a.smithi067.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T06:26:20.534 INFO:journalctl@ceph.mon.a.smithi067.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T06:26:20.534 INFO:journalctl@ceph.mon.a.smithi067.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T06:26:20.534 INFO:journalctl@ceph.mon.a.smithi067.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T06:26:20.534 INFO:journalctl@ceph.mon.a.smithi067.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T06:26:20.534 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-09-21T06:26:20.534 INFO:journalctl@ceph.mon.a.smithi067.stdout: ** Compaction Stats [default] ** 2024-09-21T06:26:20.534 INFO:journalctl@ceph.mon.a.smithi067.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T06:26:20.534 INFO:journalctl@ceph.mon.a.smithi067.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T06:26:20.534 INFO:journalctl@ceph.mon.a.smithi067.stdout: L0 2/0 89.77 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 117.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T06:26:20.535 INFO:journalctl@ceph.mon.a.smithi067.stdout: Sum 2/0 89.77 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 117.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T06:26:20.535 INFO:journalctl@ceph.mon.a.smithi067.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 117.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T06:26:20.535 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-09-21T06:26:20.535 INFO:journalctl@ceph.mon.a.smithi067.stdout: ** Compaction Stats [default] ** 2024-09-21T06:26:20.535 INFO:journalctl@ceph.mon.a.smithi067.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T06:26:20.535 INFO:journalctl@ceph.mon.a.smithi067.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T06:26:20.535 INFO:journalctl@ceph.mon.a.smithi067.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 117.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T06:26:20.535 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-09-21T06:26:20.535 INFO:journalctl@ceph.mon.a.smithi067.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T06:26:20.535 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-09-21T06:26:20.535 INFO:journalctl@ceph.mon.a.smithi067.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T06:26:20.535 INFO:journalctl@ceph.mon.a.smithi067.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T06:26:20.535 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T06:26:20.535 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T06:26:20.535 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T06:26:20.536 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T06:26:20.536 INFO:journalctl@ceph.mon.a.smithi067.stdout: Cumulative compaction: 0.00 GB write, 24.27 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T06:26:20.536 INFO:journalctl@ceph.mon.a.smithi067.stdout: Interval compaction: 0.00 GB write, 24.27 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T06:26:20.536 INFO:journalctl@ceph.mon.a.smithi067.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T06:26:20.536 INFO:journalctl@ceph.mon.a.smithi067.stdout: Block cache BinnedLRUCache@0x564e9bea7610#2 capacity: 512.00 MB usage: 1.28 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-21T06:26:20.536 INFO:journalctl@ceph.mon.a.smithi067.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.84 KB,0.000160933%) IndexBlock(2,0.44 KB,8.34465e-05%) Misc(1,0.00 KB,0%) 2024-09-21T06:26:20.536 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-09-21T06:26:20.536 INFO:journalctl@ceph.mon.a.smithi067.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T06:26:20.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: starting mon.a rank 0 at public addrs [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] at bind addrs [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 2a791768-77e2-11ef-baf5-efdc52797490 2024-09-21T06:26:20.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: mon.a@-1(???) e1 preinit fsid 2a791768-77e2-11ef-baf5-efdc52797490 2024-09-21T06:26:20.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: mon.a@-1(???).mds e1 new map 2024-09-21T06:26:20.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: mon.a@-1(???).mds e1 print_map 2024-09-21T06:26:20.536 INFO:journalctl@ceph.mon.a.smithi067.stdout: e1 2024-09-21T06:26:20.536 INFO:journalctl@ceph.mon.a.smithi067.stdout: btime 2024-09-21T06:26:15:222052+0000 2024-09-21T06:26:20.536 INFO:journalctl@ceph.mon.a.smithi067.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T06:26:20.536 INFO:journalctl@ceph.mon.a.smithi067.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T06:26:20.537 INFO:journalctl@ceph.mon.a.smithi067.stdout: legacy client fscid: -1 2024-09-21T06:26:20.537 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-09-21T06:26:20.537 INFO:journalctl@ceph.mon.a.smithi067.stdout: No filesystems configured 2024-09-21T06:26:20.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T06:26:20.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T06:26:20.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T06:26:20.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T06:26:20.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-21T06:26:20.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 bash[31213]: 853c94eacb0a03af1e73ff829802dde383540b0848e22c2197cbdf6e7c29de18 2024-09-21T06:26:20.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 systemd[1]: Started Ceph mon.a for 2a791768-77e2-11ef-baf5-efdc52797490. 2024-09-21T06:26:20.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T06:26:20.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: monmap epoch 1 2024-09-21T06:26:20.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: fsid 2a791768-77e2-11ef-baf5-efdc52797490 2024-09-21T06:26:20.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: last_changed 2024-09-21T06:26:12.167159+0000 2024-09-21T06:26:20.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: created 2024-09-21T06:26:12.167159+0000 2024-09-21T06:26:20.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: min_mon_release 19 (squid) 2024-09-21T06:26:20.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: election_strategy: 1 2024-09-21T06:26:20.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: 0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-09-21T06:26:20.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: fsmap 2024-09-21T06:26:20.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T06:26:20.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:20 smithi067 ceph-mon[31232]: mgrmap e1: no daemons active 2024-09-21T06:26:21.329 INFO:teuthology.orchestra.run.smithi067.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-21T06:26:21.351 INFO:teuthology.orchestra.run.smithi067.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:26:21.351 INFO:teuthology.orchestra.run.smithi067.stdout:Creating mgr... 2024-09-21T06:26:21.351 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-21T06:26:21.352 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-21T06:26:21.609 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2a791768-77e2-11ef-baf5-efdc52797490@mgr.x 2024-09-21T06:26:21.609 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2a791768-77e2-11ef-baf5-efdc52797490@mgr.x.service: Unit ceph-2a791768-77e2-11ef-baf5-efdc52797490@mgr.x.service not loaded. 2024-09-21T06:26:21.776 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2a791768-77e2-11ef-baf5-efdc52797490.target.wants/ceph-2a791768-77e2-11ef-baf5-efdc52797490@mgr.x.service → /etc/systemd/system/ceph-2a791768-77e2-11ef-baf5-efdc52797490@.service. 2024-09-21T06:26:22.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:21 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2431751219' entity='client.admin' 2024-09-21T06:26:22.042 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:22 smithi067 podman[31443]: 2024-09-21 06:26:22.019319802 +0000 UTC m=+0.119658575 container create 5f64d4ee2103e6d32e6dd163eeae1e1419ecdebddd8f127444218d71ebf57a55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-09-21T06:26:22.338 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T06:26:22.338 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout disabled 2024-09-21T06:26:22.347 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T06:26:22.347 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-09-21T06:26:22.347 INFO:teuthology.orchestra.run.smithi067.stdout:firewalld.service is not enabled 2024-09-21T06:26:22.347 INFO:teuthology.orchestra.run.smithi067.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T06:26:22.358 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T06:26:22.358 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout disabled 2024-09-21T06:26:22.367 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T06:26:22.367 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-09-21T06:26:22.367 INFO:teuthology.orchestra.run.smithi067.stdout:firewalld.service is not enabled 2024-09-21T06:26:22.367 INFO:teuthology.orchestra.run.smithi067.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-21T06:26:22.367 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mgr to start... 2024-09-21T06:26:22.367 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mgr... 2024-09-21T06:26:22.491 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:22 smithi067 podman[31443]: 2024-09-21 06:26:22.238276025 +0000 UTC m=+0.338614766 container init 5f64d4ee2103e6d32e6dd163eeae1e1419ecdebddd8f127444218d71ebf57a55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-21T06:26:22.491 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:22 smithi067 podman[31443]: 2024-09-21 06:26:22.241186108 +0000 UTC m=+0.341524842 container start 5f64d4ee2103e6d32e6dd163eeae1e1419ecdebddd8f127444218d71ebf57a55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:26:22.491 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:22 smithi067 bash[31443]: 5f64d4ee2103e6d32e6dd163eeae1e1419ecdebddd8f127444218d71ebf57a55 2024-09-21T06:26:22.491 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:22 smithi067 systemd[1]: Started Ceph mgr.x for 2a791768-77e2-11ef-baf5-efdc52797490. 2024-09-21T06:26:23.041 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:22 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3227674726' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T06:26:23.477 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-09-21T06:26:23.477 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-09-21T06:26:23.477 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsid": "2a791768-77e2-11ef-baf5-efdc52797490", 2024-09-21T06:26:23.477 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T06:26:23.477 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T06:26:23.477 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T06:26:23.477 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T06:26:23.477 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:26:23.477 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T06:26:23.477 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T06:26:23.477 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 0 2024-09-21T06:26:23.477 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:26:23.477 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T06:26:23.478 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T06:26:23.478 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:26:23.478 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-21T06:26:23.478 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T06:26:23.478 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:26:23.478 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T06:26:23.478 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T06:26:23.478 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:26:23.478 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T06:26:23.478 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:26:23.478 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T06:26:23.478 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T06:26:23.478 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T06:26:23.478 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T06:26:23.479 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T06:26:23.479 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T06:26:23.482 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:26:23.482 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T06:26:23.482 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T06:26:23.482 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T06:26:23.482 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T06:26:23.482 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T06:26:23.482 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T06:26:23.482 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T06:26:23.482 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T06:26:23.483 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T06:26:23.483 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:26:23.483 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T06:26:23.483 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:26:23.483 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T06:26:15:222052+0000", 2024-09-21T06:26:23.483 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T06:26:23.483 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T06:26:23.483 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:26:23.483 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T06:26:23.483 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T06:26:23.483 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T06:26:23.483 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T06:26:23.484 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T06:26:23.484 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T06:26:23.484 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T06:26:23.484 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:26:23.484 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T06:26:23.484 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:26:23.484 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T06:26:23.484 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:26:23.484 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T06:26:15.222889+0000", 2024-09-21T06:26:23.484 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T06:26:23.484 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:26:23.484 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T06:26:23.484 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-09-21T06:26:23.484 INFO:teuthology.orchestra.run.smithi067.stdout:mgr not available, waiting (1/15)... 2024-09-21T06:26:26.286 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:26 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2345836846' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T06:26:26.582 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-09-21T06:26:26.582 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-09-21T06:26:26.582 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsid": "2a791768-77e2-11ef-baf5-efdc52797490", 2024-09-21T06:26:26.582 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T06:26:26.582 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T06:26:26.582 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T06:26:26.582 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T06:26:26.583 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:26:26.583 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T06:26:26.583 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T06:26:26.583 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 0 2024-09-21T06:26:26.583 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:26:26.583 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T06:26:26.583 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T06:26:26.583 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:26:26.583 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-21T06:26:26.583 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T06:26:26.583 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:26:26.583 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T06:26:26.583 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T06:26:26.583 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:26:26.583 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T06:26:26.584 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:26:26.584 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T06:26:26.584 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T06:26:26.584 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T06:26:26.584 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T06:26:26.584 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T06:26:26.584 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T06:26:26.587 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:26:26.587 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T06:26:26.587 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T06:26:26.587 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T06:26:26.587 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T06:26:26.587 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T06:26:26.587 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T06:26:26.587 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T06:26:26.587 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T06:26:26.587 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T06:26:26.587 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:26:26.587 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T06:26:26.587 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:26:26.588 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T06:26:15:222052+0000", 2024-09-21T06:26:26.588 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T06:26:26.588 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T06:26:26.588 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:26:26.588 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T06:26:26.588 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T06:26:26.588 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T06:26:26.588 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T06:26:26.588 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T06:26:26.588 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T06:26:26.588 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T06:26:26.588 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:26:26.588 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T06:26:26.588 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:26:26.588 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T06:26:26.588 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:26:26.588 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T06:26:15.222889+0000", 2024-09-21T06:26:26.588 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T06:26:26.589 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:26:26.589 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T06:26:26.589 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-09-21T06:26:26.589 INFO:teuthology.orchestra.run.smithi067.stdout:mgr not available, waiting (2/15)... 2024-09-21T06:26:28.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:27 smithi067 ceph-mon[31232]: Activating manager daemon x 2024-09-21T06:26:28.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:27 smithi067 ceph-mon[31232]: mgrmap e2: x(active, starting, since 0.00358912s) 2024-09-21T06:26:28.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:27 smithi067 ceph-mon[31232]: from='mgr.14100 172.21.15.67:0/2804104130' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T06:26:28.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:27 smithi067 ceph-mon[31232]: from='mgr.14100 172.21.15.67:0/2804104130' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T06:26:28.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:27 smithi067 ceph-mon[31232]: from='mgr.14100 172.21.15.67:0/2804104130' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T06:26:28.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:27 smithi067 ceph-mon[31232]: from='mgr.14100 172.21.15.67:0/2804104130' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:26:28.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:27 smithi067 ceph-mon[31232]: from='mgr.14100 172.21.15.67:0/2804104130' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T06:26:28.043 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:27 smithi067 ceph-mon[31232]: Manager daemon x is now available 2024-09-21T06:26:28.043 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:27 smithi067 ceph-mon[31232]: from='mgr.14100 172.21.15.67:0/2804104130' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T06:26:28.043 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:27 smithi067 ceph-mon[31232]: from='mgr.14100 172.21.15.67:0/2804104130' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T06:26:28.043 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:27 smithi067 ceph-mon[31232]: from='mgr.14100 172.21.15.67:0/2804104130' entity='mgr.x' 2024-09-21T06:26:28.043 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:27 smithi067 ceph-mon[31232]: from='mgr.14100 172.21.15.67:0/2804104130' entity='mgr.x' 2024-09-21T06:26:28.043 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:27 smithi067 ceph-mon[31232]: from='mgr.14100 172.21.15.67:0/2804104130' entity='mgr.x' 2024-09-21T06:26:29.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:29 smithi067 ceph-mon[31232]: mgrmap e3: x(active, since 1.00874s) 2024-09-21T06:26:29.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:29 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1416403943' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T06:26:29.838 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-09-21T06:26:29.838 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-09-21T06:26:29.838 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsid": "2a791768-77e2-11ef-baf5-efdc52797490", 2024-09-21T06:26:29.838 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T06:26:29.838 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T06:26:29.838 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T06:26:29.838 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T06:26:29.838 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:26:29.838 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T06:26:29.839 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T06:26:29.839 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 0 2024-09-21T06:26:29.839 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:26:29.839 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T06:26:29.839 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T06:26:29.839 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:26:29.839 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-21T06:26:29.839 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T06:26:29.839 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:26:29.839 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T06:26:29.839 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T06:26:29.839 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:26:29.839 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T06:26:29.839 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:26:29.839 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T06:26:29.840 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T06:26:29.840 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T06:26:29.843 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T06:26:29.843 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T06:26:29.843 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T06:26:29.843 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:26:29.843 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T06:26:29.843 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T06:26:29.843 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T06:26:29.844 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T06:26:29.844 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T06:26:29.844 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T06:26:29.844 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T06:26:29.844 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T06:26:29.844 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T06:26:29.844 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:26:29.844 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T06:26:29.844 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:26:29.844 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T06:26:15:222052+0000", 2024-09-21T06:26:29.844 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T06:26:29.844 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T06:26:29.844 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:26:29.844 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T06:26:29.845 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T06:26:29.845 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T06:26:29.845 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T06:26:29.845 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T06:26:29.845 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T06:26:29.845 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T06:26:29.845 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:26:29.845 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T06:26:29.845 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:26:29.845 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T06:26:29.845 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:26:29.845 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T06:26:15.222889+0000", 2024-09-21T06:26:29.845 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T06:26:29.845 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:26:29.846 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T06:26:29.846 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-09-21T06:26:29.846 INFO:teuthology.orchestra.run.smithi067.stdout:mgr is available 2024-09-21T06:26:30.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:30 smithi067 ceph-mon[31232]: mgrmap e4: x(active, since 2s) 2024-09-21T06:26:30.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:30 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/887794759' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T06:26:31.060 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-09-21T06:26:31.060 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T06:26:31.060 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout fsid = 2a791768-77e2-11ef-baf5-efdc52797490 2024-09-21T06:26:31.060 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T06:26:31.060 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.67:3300,v1:172.21.15.67:6789] 2024-09-21T06:26:31.060 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T06:26:31.061 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T06:26:31.061 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T06:26:31.061 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T06:26:31.061 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-09-21T06:26:31.061 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T06:26:31.061 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T06:26:31.061 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-09-21T06:26:31.061 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T06:26:31.061 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T06:26:31.061 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T06:26:31.061 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T06:26:31.061 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T06:26:31.061 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T06:26:31.061 INFO:teuthology.orchestra.run.smithi067.stdout:Enabling cephadm module... 2024-09-21T06:26:32.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:31 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2917837247' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-21T06:26:32.042 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:31 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: ignoring --setuser ceph since I am not root 2024-09-21T06:26:32.042 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:31 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: ignoring --setgroup ceph since I am not root 2024-09-21T06:26:32.988 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:32 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2917837247' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-21T06:26:32.988 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:32 smithi067 ceph-mon[31232]: mgrmap e5: x(active, since 4s) 2024-09-21T06:26:33.622 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-09-21T06:26:33.622 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-21T06:26:33.623 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T06:26:33.623 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T06:26:33.623 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T06:26:33.623 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-09-21T06:26:33.623 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for the mgr to restart... 2024-09-21T06:26:33.623 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mgr epoch 5... 2024-09-21T06:26:33.905 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:33 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1769638918' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T06:26:37.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:37 smithi067 ceph-mon[31232]: Active manager daemon x restarted 2024-09-21T06:26:37.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:37 smithi067 ceph-mon[31232]: Activating manager daemon x 2024-09-21T06:26:37.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:37 smithi067 ceph-mon[31232]: osdmap e2: 0 total, 0 up, 0 in 2024-09-21T06:26:37.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:37 smithi067 ceph-mon[31232]: mgrmap e6: x(active, starting, since 0.00380085s) 2024-09-21T06:26:37.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:37 smithi067 ceph-mon[31232]: from='mgr.14118 172.21.15.67:0/8313892' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:26:37.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:37 smithi067 ceph-mon[31232]: from='mgr.14118 172.21.15.67:0/8313892' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T06:26:37.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:37 smithi067 ceph-mon[31232]: from='mgr.14118 172.21.15.67:0/8313892' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T06:26:37.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:37 smithi067 ceph-mon[31232]: from='mgr.14118 172.21.15.67:0/8313892' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T06:26:37.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:37 smithi067 ceph-mon[31232]: from='mgr.14118 172.21.15.67:0/8313892' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T06:26:37.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:37 smithi067 ceph-mon[31232]: Manager daemon x is now available 2024-09-21T06:26:38.603 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-09-21T06:26:38.603 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-21T06:26:38.603 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T06:26:38.603 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-09-21T06:26:38.603 INFO:teuthology.orchestra.run.smithi067.stdout:mgr epoch 5 is available 2024-09-21T06:26:38.603 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying orchestrator module is enabled... 2024-09-21T06:26:39.035 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:38 smithi067 ceph-mon[31232]: from='mgr.14118 172.21.15.67:0/8313892' entity='mgr.x' 2024-09-21T06:26:39.036 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:38 smithi067 ceph-mon[31232]: from='mgr.14118 172.21.15.67:0/8313892' entity='mgr.x' 2024-09-21T06:26:39.036 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:38 smithi067 ceph-mon[31232]: Found migration_current of "None". Setting to last migration. 2024-09-21T06:26:39.036 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:38 smithi067 ceph-mon[31232]: from='mgr.14118 172.21.15.67:0/8313892' entity='mgr.x' 2024-09-21T06:26:39.036 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:38 smithi067 ceph-mon[31232]: from='mgr.14118 172.21.15.67:0/8313892' entity='mgr.x' 2024-09-21T06:26:39.036 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:38 smithi067 ceph-mon[31232]: from='mgr.14118 172.21.15.67:0/8313892' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:26:39.036 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:38 smithi067 ceph-mon[31232]: from='mgr.14118 172.21.15.67:0/8313892' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:26:39.036 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:38 smithi067 ceph-mon[31232]: from='mgr.14118 172.21.15.67:0/8313892' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T06:26:39.036 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:38 smithi067 ceph-mon[31232]: from='mgr.14118 172.21.15.67:0/8313892' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T06:26:39.036 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:38 smithi067 ceph-mon[31232]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T06:26:39.036 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:38 smithi067 ceph-mon[31232]: mgrmap e7: x(active, since 1.00747s) 2024-09-21T06:26:39.036 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:38 smithi067 ceph-mon[31232]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T06:26:40.288 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:39 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2995632571' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch 2024-09-21T06:26:40.491 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr module 'orchestrator' is already enabled (always-on) 2024-09-21T06:26:40.491 INFO:teuthology.orchestra.run.smithi067.stdout:Setting orchestrator backend to cephadm... 2024-09-21T06:26:41.150 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:40 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2995632571' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished 2024-09-21T06:26:41.150 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:40 smithi067 ceph-mon[31232]: mgrmap e8: x(active, since 2s) 2024-09-21T06:26:42.023 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:41 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: 2024-09-21T06:26:41.808+0000 7f3713752640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:06:26:41] ENGINE Error in HTTPServer.serve 2024-09-21T06:26:42.023 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:41 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: Traceback (most recent call last): 2024-09-21T06:26:42.023 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:41 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T06:26:42.023 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:41 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: self._connections.run(self.expiration_interval) 2024-09-21T06:26:42.023 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:41 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T06:26:42.023 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:41 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: self._run(expiration_interval) 2024-09-21T06:26:42.023 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:41 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T06:26:42.023 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:41 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T06:26:42.023 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:41 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T06:26:42.023 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:41 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T06:26:42.023 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:41 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T06:26:42.023 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:41 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: s = self.context.wrap_socket( 2024-09-21T06:26:42.024 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:41 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T06:26:42.024 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:41 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: return self.sslsocket_class._create( 2024-09-21T06:26:42.024 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:41 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T06:26:42.024 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:41 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: self.do_handshake() 2024-09-21T06:26:42.024 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:41 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T06:26:42.024 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:41 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: self._sslobj.do_handshake() 2024-09-21T06:26:42.024 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:41 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T06:26:42.024 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:41 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: 2024-09-21T06:26:42.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:42 smithi067 ceph-mon[31232]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:26:42.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:42 smithi067 ceph-mon[31232]: from='mgr.14118 172.21.15.67:0/8313892' entity='mgr.x' 2024-09-21T06:26:42.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:42 smithi067 ceph-mon[31232]: from='mgr.14118 172.21.15.67:0/8313892' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:26:42.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:42 smithi067 ceph-mon[31232]: [21/Sep/2024:06:26:41] ENGINE Bus STARTING 2024-09-21T06:26:42.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:42 smithi067 ceph-mon[31232]: [21/Sep/2024:06:26:41] ENGINE Serving on http://172.21.15.67:8765 2024-09-21T06:26:42.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:42 smithi067 ceph-mon[31232]: [21/Sep/2024:06:26:41] ENGINE Serving on https://172.21.15.67:7150 2024-09-21T06:26:42.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:42 smithi067 ceph-mon[31232]: [21/Sep/2024:06:26:41] ENGINE Bus STARTED 2024-09-21T06:26:42.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:42 smithi067 ceph-mon[31232]: from='mgr.14118 172.21.15.67:0/8313892' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:26:42.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:42 smithi067 ceph-mon[31232]: [21/Sep/2024:06:26:41] ENGINE Error in HTTPServer.serve 2024-09-21T06:26:42.293 INFO:journalctl@ceph.mon.a.smithi067.stdout: Traceback (most recent call last): 2024-09-21T06:26:42.293 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T06:26:42.293 INFO:journalctl@ceph.mon.a.smithi067.stdout: self._connections.run(self.expiration_interval) 2024-09-21T06:26:42.293 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T06:26:42.293 INFO:journalctl@ceph.mon.a.smithi067.stdout: self._run(expiration_interval) 2024-09-21T06:26:42.293 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T06:26:42.293 INFO:journalctl@ceph.mon.a.smithi067.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T06:26:42.293 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T06:26:42.293 INFO:journalctl@ceph.mon.a.smithi067.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T06:26:42.293 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T06:26:42.293 INFO:journalctl@ceph.mon.a.smithi067.stdout: s = self.context.wrap_socket( 2024-09-21T06:26:42.293 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T06:26:42.293 INFO:journalctl@ceph.mon.a.smithi067.stdout: return self.sslsocket_class._create( 2024-09-21T06:26:42.293 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T06:26:42.294 INFO:journalctl@ceph.mon.a.smithi067.stdout: self.do_handshake() 2024-09-21T06:26:42.294 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T06:26:42.294 INFO:journalctl@ceph.mon.a.smithi067.stdout: self._sslobj.do_handshake() 2024-09-21T06:26:42.294 INFO:journalctl@ceph.mon.a.smithi067.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T06:26:42.871 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-21T06:26:42.871 INFO:teuthology.orchestra.run.smithi067.stdout:Generating ssh key... 2024-09-21T06:26:43.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:43 smithi067 ceph-mon[31232]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:26:44.399 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:44 smithi067 ceph-mon[31232]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:26:44.400 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:44 smithi067 ceph-mon[31232]: Generating ssh key... 2024-09-21T06:26:44.400 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:44 smithi067 ceph-mon[31232]: from='mgr.14118 172.21.15.67:0/8313892' entity='mgr.x' 2024-09-21T06:26:44.400 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:44 smithi067 ceph-mon[31232]: from='mgr.14118 172.21.15.67:0/8313892' entity='mgr.x' 2024-09-21T06:26:44.400 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:44 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: Generating public/private rsa key pair. 2024-09-21T06:26:44.400 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:44 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: Your identification has been saved in /tmp/tmplg51mscj/key 2024-09-21T06:26:44.400 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:44 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: Your public key has been saved in /tmp/tmplg51mscj/key.pub 2024-09-21T06:26:44.400 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:44 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: The key fingerprint is: 2024-09-21T06:26:44.400 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:44 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: SHA256:z2Vvl4brc/J3851s93T5tQg/ckerN49voWeHVHSHlQg ceph-2a791768-77e2-11ef-baf5-efdc52797490 2024-09-21T06:26:44.401 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:44 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: The key's randomart image is: 2024-09-21T06:26:44.401 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:44 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: +---[RSA 3072]----+ 2024-09-21T06:26:44.401 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:44 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: | E. .oo| 2024-09-21T06:26:44.401 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:44 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: | ...+| 2024-09-21T06:26:44.401 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:44 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: | .o| 2024-09-21T06:26:44.401 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:44 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: | .| 2024-09-21T06:26:44.401 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:44 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: | S o . | 2024-09-21T06:26:44.401 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:44 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: | o o .ooo| 2024-09-21T06:26:44.401 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:44 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: | o. o==B| 2024-09-21T06:26:44.401 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:44 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: | .oB*#&| 2024-09-21T06:26:44.401 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:44 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: | +*#B^| 2024-09-21T06:26:44.401 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:44 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: +----[SHA256]-----+ 2024-09-21T06:26:45.767 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtVRZT+VINosykitNt+uL0Tfgvao4CsZcdgLaZlofN3JkFhRsHWLTb/eKSqWqezPUxNHZIWFiACqKiOy1k6Sw/zOHZsJ7csgsDTEMPVDnLiUXLXkYUwtuoAIOSCvPYUcQtiTdnQrKOb6jOipdyThQTmuc+KN2tFGtnxuYBULHvVxm0KPcs1ulLuO+15RLQ8x+Ex321w49nMc78T6EE1j16O691ArVssfQqSFHp44ICA3SMobCUxRrATrJ9bDHlZ+Q8b23Wr0p7uFSvuZ8J1FUJk2J8UNjUi3uwqOf8178IYwDeXb6ibQJTd+8jgtckWTdePFz60Ifvl9VVXrAklERZTt4vDfnhdRTx9Xlw2g1WvaR22/VW2yMkfVIaBRPlh7ei9YeEvjCPkuDGJX/TNWuoqRqvA81WCg4UrAvTlBJxDqw82XhWaCvBVSDbYJpivwkP9LYuPPaedjGD8fKJRW6l+zMTPK0vbnJPBchuKafd3ToeCjSthnKPEkPLWwoN0Pk= ceph-2a791768-77e2-11ef-baf5-efdc52797490 2024-09-21T06:26:45.767 INFO:teuthology.orchestra.run.smithi067.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-21T06:26:45.767 INFO:teuthology.orchestra.run.smithi067.stdout:Adding key to root@localhost authorized_keys... 2024-09-21T06:26:45.773 INFO:teuthology.orchestra.run.smithi067.stdout:Adding host smithi067... 2024-09-21T06:26:46.200 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:45 smithi067 ceph-mon[31232]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:26:47.111 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:46 smithi067 ceph-mon[31232]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi067", "addr": "172.21.15.67", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:26:47.987 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:47 smithi067 ceph-mon[31232]: Deploying cephadm binary to smithi067 2024-09-21T06:26:50.841 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout Added host 'smithi067' with addr '172.21.15.67' 2024-09-21T06:26:50.841 INFO:teuthology.orchestra.run.smithi067.stdout:Deploying unmanaged mon service... 2024-09-21T06:26:51.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:51 smithi067 ceph-mon[31232]: from='mgr.14118 172.21.15.67:0/8313892' entity='mgr.x' 2024-09-21T06:26:51.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:51 smithi067 ceph-mon[31232]: Added host smithi067 2024-09-21T06:26:51.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:51 smithi067 ceph-mon[31232]: from='mgr.14118 172.21.15.67:0/8313892' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:26:52.562 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-21T06:26:52.562 INFO:teuthology.orchestra.run.smithi067.stdout:Deploying unmanaged mgr service... 2024-09-21T06:26:52.695 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:52 smithi067 ceph-mon[31232]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:26:52.695 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:52 smithi067 ceph-mon[31232]: Saving service mon spec with placement count:5 2024-09-21T06:26:52.695 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:52 smithi067 ceph-mon[31232]: from='mgr.14118 172.21.15.67:0/8313892' entity='mgr.x' 2024-09-21T06:26:53.963 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-21T06:26:54.280 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:54 smithi067 ceph-mon[31232]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:26:54.280 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:54 smithi067 ceph-mon[31232]: Saving service mgr spec with placement count:2 2024-09-21T06:26:54.280 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:54 smithi067 ceph-mon[31232]: from='mgr.14118 172.21.15.67:0/8313892' entity='mgr.x' 2024-09-21T06:26:54.280 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:54 smithi067 ceph-mon[31232]: from='mgr.14118 172.21.15.67:0/8313892' entity='mgr.x' 2024-09-21T06:26:55.475 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:55 smithi067 ceph-mon[31232]: from='mgr.14118 172.21.15.67:0/8313892' entity='mgr.x' 2024-09-21T06:26:55.475 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:55 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1808664343' entity='client.admin' 2024-09-21T06:26:56.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:56 smithi067 ceph-mon[31232]: from='mgr.14118 172.21.15.67:0/8313892' entity='mgr.x' 2024-09-21T06:26:56.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:56 smithi067 ceph-mon[31232]: from='mgr.14118 172.21.15.67:0/8313892' entity='mgr.x' 2024-09-21T06:26:57.031 INFO:teuthology.orchestra.run.smithi067.stdout:Enabling the dashboard module... 2024-09-21T06:26:57.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:57 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/681410499' entity='client.admin' 2024-09-21T06:26:58.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:58 smithi067 ceph-mon[31232]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:26:58.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:58 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/174063220' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-21T06:26:58.792 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:58 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: ignoring --setuser ceph since I am not root 2024-09-21T06:26:58.792 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:26:58 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: ignoring --setgroup ceph since I am not root 2024-09-21T06:26:59.707 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:59 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/174063220' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-21T06:26:59.707 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:26:59 smithi067 ceph-mon[31232]: mgrmap e9: x(active, since 21s) 2024-09-21T06:27:00.390 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-09-21T06:27:00.390 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-21T06:27:00.390 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T06:27:00.390 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T06:27:00.390 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T06:27:00.390 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-09-21T06:27:00.390 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for the mgr to restart... 2024-09-21T06:27:00.390 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mgr epoch 9... 2024-09-21T06:27:00.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:00 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/4255523371' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T06:27:04.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:03 smithi067 ceph-mon[31232]: Active manager daemon x restarted 2024-09-21T06:27:04.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:03 smithi067 ceph-mon[31232]: Activating manager daemon x 2024-09-21T06:27:04.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:03 smithi067 ceph-mon[31232]: osdmap e3: 0 total, 0 up, 0 in 2024-09-21T06:27:04.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:03 smithi067 ceph-mon[31232]: mgrmap e10: x(active, starting, since 0.00407275s) 2024-09-21T06:27:04.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:03 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:27:04.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:03 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T06:27:04.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:03 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T06:27:04.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:03 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T06:27:04.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:03 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T06:27:04.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:03 smithi067 ceph-mon[31232]: Manager daemon x is now available 2024-09-21T06:27:05.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:04 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:27:05.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:04 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T06:27:05.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:04 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T06:27:05.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:04 smithi067 ceph-mon[31232]: mgrmap e11: x(active, since 1.00681s) 2024-09-21T06:27:05.320 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-09-21T06:27:05.321 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-21T06:27:05.321 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T06:27:05.321 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-09-21T06:27:05.321 INFO:teuthology.orchestra.run.smithi067.stdout:mgr epoch 9 is available 2024-09-21T06:27:05.321 INFO:teuthology.orchestra.run.smithi067.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-21T06:27:05.958 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:05 smithi067 ceph-mon[31232]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T06:27:05.958 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:05 smithi067 ceph-mon[31232]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T06:27:06.792 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:27:06 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: 2024-09-21T06:27:06.545+0000 7fbf93f47640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:06:27:06] ENGINE Error in HTTPServer.serve 2024-09-21T06:27:06.792 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:27:06 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: Traceback (most recent call last): 2024-09-21T06:27:06.792 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:27:06 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T06:27:06.792 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:27:06 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: self._connections.run(self.expiration_interval) 2024-09-21T06:27:06.792 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:27:06 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T06:27:06.793 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:27:06 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: self._run(expiration_interval) 2024-09-21T06:27:06.793 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:27:06 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T06:27:06.793 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:27:06 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T06:27:06.793 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:27:06 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T06:27:06.793 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:27:06 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T06:27:06.793 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:27:06 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T06:27:06.793 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:27:06 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: s = self.context.wrap_socket( 2024-09-21T06:27:06.793 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:27:06 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T06:27:06.793 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:27:06 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: return self.sslsocket_class._create( 2024-09-21T06:27:06.793 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:27:06 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T06:27:06.793 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:27:06 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: self.do_handshake() 2024-09-21T06:27:06.793 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:27:06 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T06:27:06.793 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:27:06 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: self._sslobj.do_handshake() 2024-09-21T06:27:06.793 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:27:06 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T06:27:06.794 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:27:06 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: 2024-09-21T06:27:07.152 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:06 smithi067 ceph-mon[31232]: mgrmap e12: x(active, since 2s) 2024-09-21T06:27:07.152 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:06 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:07.152 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:06 smithi067 ceph-mon[31232]: [21/Sep/2024:06:27:06] ENGINE Bus STARTING 2024-09-21T06:27:07.152 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:06 smithi067 ceph-mon[31232]: [21/Sep/2024:06:27:06] ENGINE Serving on http://172.21.15.67:8765 2024-09-21T06:27:07.152 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:06 smithi067 ceph-mon[31232]: [21/Sep/2024:06:27:06] ENGINE Serving on https://172.21.15.67:7150 2024-09-21T06:27:07.152 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:06 smithi067 ceph-mon[31232]: [21/Sep/2024:06:27:06] ENGINE Bus STARTED 2024-09-21T06:27:07.154 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE+zCCAuOgAwIBAgIUO9ZLuY7SdC7Dh9QpeyPM8s7Eo+EwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkyMTA2MjcwNloXDTM0\nMDkyMjA2MjcwNlowFzEVMBMGA1UEAwwMMTcyLjIxLjE1LjY3MIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEAmZH789GXagb7y67/pvruPKrh7Hr2Yg36NOat\n4xVpW1hN/0tUwV8UgCDRGTfbhvnn2ksUOeDt5ycGkqTWWHQBS2NXavZ4wIc5Ocmn\nmDGK2MzMcV0jS8Sfe/ThzQigpVco+l4qO4s432gQMCSuzAHrQFwNvxpO9e8oOfmc\nMMSaCw3g50EvRM9yzjhz1U0bHaopjy4bRj8dkmbWbdiMCNPsXhlav37N9RD7qw3j\ndNv/uDiGLWEEraQIRWq0X56SzK7X4XXfkrI7osXiuK+O7LWNXuHflnlRQO3JlfEp\n0GfBah0HjYTc4fwB+91Gn780NFUL0ZMG1AitujSL67oSXUXixTx+H6+LzynuFg4z\nGgwLFG3Sr5IjVHauze0fMAM8nZqjBZ1CD4YKJGkO8Ag8vc7Abq0qxL/rKnVzaFwf\nbbRju7n7FJJOYo/rLeXMlME04PgFNCsR6urUUzDdqiorW2GZbgo30gGB8gQRbn0p\nA8Wo9oQV8btWFKjZkhloktc3ybdG68s6isdAfrCCISspJeBwnPQv728WsID6mcF9\nwC9VeHld8yIfxiKOwm7cm4+Mcuny3iSz8dwYgV4nevM4AYeybc9xaSez85+3tWCU\nOeaGbKgbJ47KLOUCeyGlqVkWTuSZ0jyLC00crQwjLdPPBfqD++5NQ7ZiC0fwNHKj\nhvDI9NMCAwEAAaM/MD0wLQYDVR0RBCYwJIIJc21pdGhpMDY3ghFkYXNoYm9hcmRf\nc2VydmVyc4cErBUPQzAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBCwUAA4ICAQAx\nWNq6OWRe7QtiuFNEqD9bbVLMSGkLvrKd7sazoagY7saUf4bmcCP6vPYKfTJHhM3E\nYJf/zQ0wARKunEiMOPOeNtBrbf5Nf0HNhujNtKL71rTEf5qnKWPQ5FtZbCWZF3En\nQQHS9PIRrHEJ4nN1ZlgeYWMszSNPEVOtmehAiBiK5xI+9wxNOk9/z6Wcs6VgMqMx\nj+L+DsEdCvTGtrftawtSAZu40AQcAx4XBLun4LvuKRLwDDLZhBcF3l7JTau0xdOQ\naJCivzzvaf5s4Ipjq9b0eBujT+nrK61ON4unod7No0BSI8+ZKh06bu12FY9YHHYL\ngaj1a4b/Ty5ENe8pGzFWqPJ1DN6V8LkuaV252woeQLJKmLEp4CatyH/agGso3F3W\n1Z+nyxBHJuSGVB2w/3Nyx0LEoPY0EyHZ4aYyqoT8URodmA/O974dnEQucwEhpOMj\nUl4fHbhvQF+cny3L4ygx5Cs+XHxQuBXdp6absGeOZBJ2Wf8heuYhZoAo8/JHLN+R\nVoFYX0nXqJdl0P1Q+DUSc6Wa4JBHvcZrvD1xvQutyoljWhXVwmFCggXV5WvhAO5E\nL3WZlbRGDwZXCSioeZv5LC4I8DZuRPavlEEHRoAu+xqiJ6P+XsHQfCBKnaUpqFxH\nZIKRcFIroUgwH1mT+voA6gwNT+Y2ujZcyy58BWNtHQ==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKAIBAAKCAgEAmZH789GXagb7y67/pvruPKrh7Hr2Yg36NOat4xVpW1hN/0tU\nwV8UgCDRGTfbhvnn2ksUOeDt5ycGkqTWWHQBS2NXavZ4wIc5OcmnmDGK2MzMcV0j\nS8Sfe/ThzQigpVco+l4qO4s432gQMCSuzAHrQFwNvxpO9e8oOfmcMMSaCw3g50Ev\nRM9yzjhz1U0bHaopjy4bRj8dkmbWbdiMCNPsXhlav37N9RD7qw3jdNv/uDiGLWEE\nraQIRWq0X56SzK7X4XXfkrI7osXiuK+O7LWNXuHflnlRQO3JlfEp0GfBah0HjYTc\n4fwB+91Gn780NFUL0ZMG1AitujSL67oSXUXixTx+H6+LzynuFg4zGgwLFG3Sr5Ij\nVHauze0fMAM8nZqjBZ1CD4YKJGkO8Ag8vc7Abq0qxL/rKnVzaFwfbbRju7n7FJJO\nYo/rLeXMlME04PgFNCsR6urUUzDdqiorW2GZbgo30gGB8gQRbn0pA8Wo9oQV8btW\nFKjZkhloktc3ybdG68s6isdAfrCCISspJeBwnPQv728WsID6mcF9wC9VeHld8yIf\nxiKOwm7cm4+Mcuny3iSz8dwYgV4nevM4AYeybc9xaSez85+3tWCUOeaGbKgbJ47K\nLOUCeyGlqVkWTuSZ0jyLC00crQwjLdPPBfqD++5NQ7ZiC0fwNHKjhvDI9NMCAwEA\nAQKCAgADumaKccEiQs8kD7iHHjpgYKmhaw7GLJ6xYBRCmgd3vzoNg4+oLh26SpGo\n8hEzQqfdRctU6F/+J5ViQ/wsI4zQfrXlk9w0mFPMUlfuOu2q+7XO3rr3oOtaQASn\nTJlRyj7lbRnLXkLgMY+LC6jurPFAG7v9sq7OC6NHXQbsrCktMYYPNyblnb+kv/wX\n5opM+w+NOUpSN0AJiNOV0Rb9M9/eHPF2CqcHVwU1Dt5BvQGFQGIJgfxyFrRYsxFz\n7M6Vh30YLRtjXU5TUkiBwUlP98xZzFP4PDO7xsoJCNTCCXLJl7DSP7J5DfwiWCnZ\nwPOMfbAOw2n8BcXp6lKnQ1okx4iMxZeMBca16s/WbKcnjD1n0cHE50evZMM4wprW\na8pA2rxm2idI8K3c2xjfMcFE5ICAVY0ItVELT/mueuFf5+bYlXSs5ppfST1S0Bg+\ncSniJqV+FBB9d2pkPBxKnbA6zryZP+ngPiAbFdIb/4BSDgg6nDyvZfYm1t3cnWCt\nVOO0Fy/UaRCK7UxeEvDbXXJ4aKiqJlLNdw1qeeTBJMttoWt49b5fURYEZ79l6tDH\nUPJwSvUqsfVziyjtBhvoOuUtPM/1MhFGWsuAmu6TaDLljui7LIZoVQeewmrSxRuo\n4mHrkFRz0ljn4ppSJu8Mf0h+d/LgQJAhqsir2oIc/swQ7/IzuQKCAQEA1LwDbjDh\ntldqU2/22RlhjHUYp6fL8IhNcU/D8xiIn3lhIUNG/JZ7eEyEiSrOMteAsjltqTax\nhSTI06fL3TLYQpqPisuZOUZmR0bnNvke22MbjzSlhK/B9Qj6kSzYB5E+boYnyvmD\nIooJW4GLNlEeIHCdFdNjcD8rirGU2kB8CMIJONnq+O86jQ9AePTVIKghiJNY5ATQ\nZOV0cD4E/SRGV0lOD8lUX+vNFululDPyT8OaA2fPw97Iue123/+KnKDr+dr1BHqS\nD+jx+wGsxZUwdCZmwY75oA9ZUBT9uUU/pzqRgnvRAwEizCUd9Ji5lA7dmSV/HaGv\nDNLI4yq5e669nQKCAQEAuM2YvmvfNTJjkKxBtR+1RIyg4le0N9n85CQ3Bjeb0kBl\neystl1JEOGfxwFKtMSqa8d9wIitHstekG+E4Zz2uzUI2TUy9fnM2amE6Gv7isLkm\nW2ksUB8nHdFZ1qVKt+9opaZgW3wromw7MqirMDPcLQ0sUPJkd2d6Jj2OOpZH6CLM\nORUpPpTygo4FfpSPUgi2y9cxCRPh1HFWoHuuL+xEcQgVoaZ+/grLy7CKMKUN+V3S\n1SpsVZ2dHwIglkryfr9xvWBSVhfZNIucfnbA+KtjlV75NYFjiOs3tnbEEzeQkTN2\nVyWOXGrn2xKWqC8ZABEA0P2S5AAjFKn4Xec5584pLwKCAQBNhX4Sh4mY8t3yLdjc\nYB1mG72ySrlMm77szc5wJb6Trp2wi12w6f1wZyHdAlZFvLZPPBI0GZQzGqZ8SqdF\noiOgkcZBI9RgRME8tB/JrpqKA+GlG+A9K/cYHf+BK8BFca44hHa3xpj82yRxzhE2\n5/R8ic0UiQWxf7OURQYjSmBoqJzHtNGliTfi823hJgcMcXhW1qsL3J08R0lZJaCX\nxsJDSJlDs3AlzD3eXY5IP23Ll1Hmb8wMaoDnEBvDoNGYJiQX1qx2x1u2pIKZn5Cg\nJ6+Sg/PqEPnyU+WOAzg6bXu2qucfj5Mo7x8JGXWDDqyBwbEK/9kEqUiq+Zzu+0DA\nS3sZAoIBAC+c4O6mskJtchOTHfbBbZ10sfq+5/K/2hUZc4fJif74kmzSahOjoWn6\n9eE8K90QAg2xKJ3VEsr0HhcExthqnv/JQF5YGDZdDrcHx8Wux88KVOSWS1F1RUm8\nZ24S9hY/u3F6VTcc3c0iH3pk1H49zktZKoLMgK/23BTAJUXY9DvNw1otpou2KmkS\nmD3NnsNe9WG9VqflFHj8nnoIg49bSyLoF1Knp+5h77HSETiJPflyf+Bl5rWEPzf+\n/1BZjPK09mP5Th1aVZ7hHXfDtuIanjSc3I45gOCE4bzECq580WVnoeTAC1aqQIRD\n3McGI2XN+6zcuBzAIEmxpQjkixm7FwcCggEBAKnfc1MayzWXek5VmpT4KvzgV/On\nMt6U+R2Uw6MUCofeFrflGYcMm/spjPxLqnWBofeJbE2wI5JAeqoxwA69DVDOZYRN\n9hQUnu/QzsaVqXjYb1IVj1pNnLbXPW2jfc27FlZAMRhja8mcsk8TMcHjnOtQAq4Q\nRKlqvXWxKd1inlYaPbmWYZzc9yFve6Cy4mfktuJINCJomQB1jugNq/lXKuuwnYXj\n5/mB7c4te+kvxLBswOzAzBG4tLiV87LSkS1sU2CN0ReeaWGo0kILXFPUpIQEBBW2\nMn2Y84g0gqMgtyuHnp9XLeheqsxUPfVHQn8OeYHW4HbPpBigUkT43EMAHqI=\n-----END RSA PRIVATE KEY-----\n"} 2024-09-21T06:27:08.052 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:07 smithi067 ceph-mon[31232]: [21/Sep/2024:06:27:06] ENGINE Error in HTTPServer.serve 2024-09-21T06:27:08.052 INFO:journalctl@ceph.mon.a.smithi067.stdout: Traceback (most recent call last): 2024-09-21T06:27:08.053 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T06:27:08.053 INFO:journalctl@ceph.mon.a.smithi067.stdout: self._connections.run(self.expiration_interval) 2024-09-21T06:27:08.053 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T06:27:08.053 INFO:journalctl@ceph.mon.a.smithi067.stdout: self._run(expiration_interval) 2024-09-21T06:27:08.053 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T06:27:08.053 INFO:journalctl@ceph.mon.a.smithi067.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T06:27:08.053 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T06:27:08.053 INFO:journalctl@ceph.mon.a.smithi067.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T06:27:08.053 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T06:27:08.053 INFO:journalctl@ceph.mon.a.smithi067.stdout: s = self.context.wrap_socket( 2024-09-21T06:27:08.053 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T06:27:08.053 INFO:journalctl@ceph.mon.a.smithi067.stdout: return self.sslsocket_class._create( 2024-09-21T06:27:08.053 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T06:27:08.053 INFO:journalctl@ceph.mon.a.smithi067.stdout: self.do_handshake() 2024-09-21T06:27:08.054 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T06:27:08.054 INFO:journalctl@ceph.mon.a.smithi067.stdout: self._sslobj.do_handshake() 2024-09-21T06:27:08.054 INFO:journalctl@ceph.mon.a.smithi067.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T06:27:08.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:07 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:08.369 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-21T06:27:09.156 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:09 smithi067 ceph-mon[31232]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:27:09.156 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:09 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:09.156 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:09 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:10.272 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:10 smithi067 ceph-mon[31232]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:27:10.315 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-21T06:27:10.315 INFO:teuthology.orchestra.run.smithi067.stdout:Creating initial admin user... 2024-09-21T06:27:11.443 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:11 smithi067 ceph-mon[31232]: mgrmap e13: x(active, since 6s) 2024-09-21T06:27:11.713 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$k8ND32/O6vhnA6QlPO5YpeYzr7aAcZZNQSzQB7Zn9k6TakG/2UnZG", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726900031, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-21T06:27:11.713 INFO:teuthology.orchestra.run.smithi067.stdout:Fetching dashboard port number... 2024-09-21T06:27:12.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:12 smithi067 ceph-mon[31232]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:27:12.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:12 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:12.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 8443 2024-09-21T06:27:12.921 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T06:27:12.921 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout disabled 2024-09-21T06:27:12.928 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T06:27:12.929 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-09-21T06:27:12.929 INFO:teuthology.orchestra.run.smithi067.stdout:firewalld.service is not enabled 2024-09-21T06:27:12.929 INFO:teuthology.orchestra.run.smithi067.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-21T06:27:12.931 INFO:teuthology.orchestra.run.smithi067.stdout:Ceph Dashboard is now available at: 2024-09-21T06:27:12.931 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:27:12.931 INFO:teuthology.orchestra.run.smithi067.stdout: URL: https://smithi067.front.sepia.ceph.com:8443/ 2024-09-21T06:27:12.931 INFO:teuthology.orchestra.run.smithi067.stdout: User: admin 2024-09-21T06:27:12.931 INFO:teuthology.orchestra.run.smithi067.stdout: Password: jdyqtykvjo 2024-09-21T06:27:12.931 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:27:12.932 INFO:teuthology.orchestra.run.smithi067.stdout:Saving cluster configuration to /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/config directory 2024-09-21T06:27:13.158 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:13 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1592867334' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-21T06:27:14.192 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-21T06:27:14.192 INFO:teuthology.orchestra.run.smithi067.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-21T06:27:14.193 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:27:14.193 INFO:teuthology.orchestra.run.smithi067.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:27:14.193 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:27:14.193 INFO:teuthology.orchestra.run.smithi067.stdout:Or, if you are only running a single cluster on this host: 2024-09-21T06:27:14.193 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:27:14.193 INFO:teuthology.orchestra.run.smithi067.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-21T06:27:14.193 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:27:14.193 INFO:teuthology.orchestra.run.smithi067.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-21T06:27:14.193 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:27:14.193 INFO:teuthology.orchestra.run.smithi067.stdout: ceph telemetry on 2024-09-21T06:27:14.193 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:27:14.193 INFO:teuthology.orchestra.run.smithi067.stdout:For more information see: 2024-09-21T06:27:14.193 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:27:14.193 INFO:teuthology.orchestra.run.smithi067.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-21T06:27:14.194 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:27:14.194 INFO:teuthology.orchestra.run.smithi067.stdout:Bootstrap complete. 2024-09-21T06:27:14.240 INFO:tasks.cephadm:Fetching config... 2024-09-21T06:27:14.240 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-21T06:27:14.240 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-21T06:27:14.258 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-21T06:27:14.258 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-21T06:27:14.258 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-21T06:27:14.315 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-21T06:27:14.315 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-21T06:27:14.315 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/keyring of=/dev/stdout 2024-09-21T06:27:14.382 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-21T06:27:14.382 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-21T06:27:14.382 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-21T06:27:14.439 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-21T06:27:14.440 DEBUG:teuthology.orchestra.run.smithi067:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtVRZT+VINosykitNt+uL0Tfgvao4CsZcdgLaZlofN3JkFhRsHWLTb/eKSqWqezPUxNHZIWFiACqKiOy1k6Sw/zOHZsJ7csgsDTEMPVDnLiUXLXkYUwtuoAIOSCvPYUcQtiTdnQrKOb6jOipdyThQTmuc+KN2tFGtnxuYBULHvVxm0KPcs1ulLuO+15RLQ8x+Ex321w49nMc78T6EE1j16O691ArVssfQqSFHp44ICA3SMobCUxRrATrJ9bDHlZ+Q8b23Wr0p7uFSvuZ8J1FUJk2J8UNjUi3uwqOf8178IYwDeXb6ibQJTd+8jgtckWTdePFz60Ifvl9VVXrAklERZTt4vDfnhdRTx9Xlw2g1WvaR22/VW2yMkfVIaBRPlh7ei9YeEvjCPkuDGJX/TNWuoqRqvA81WCg4UrAvTlBJxDqw82XhWaCvBVSDbYJpivwkP9LYuPPaedjGD8fKJRW6l+zMTPK0vbnJPBchuKafd3ToeCjSthnKPEkPLWwoN0Pk= ceph-2a791768-77e2-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T06:27:14.520 INFO:teuthology.orchestra.run.smithi067.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtVRZT+VINosykitNt+uL0Tfgvao4CsZcdgLaZlofN3JkFhRsHWLTb/eKSqWqezPUxNHZIWFiACqKiOy1k6Sw/zOHZsJ7csgsDTEMPVDnLiUXLXkYUwtuoAIOSCvPYUcQtiTdnQrKOb6jOipdyThQTmuc+KN2tFGtnxuYBULHvVxm0KPcs1ulLuO+15RLQ8x+Ex321w49nMc78T6EE1j16O691ArVssfQqSFHp44ICA3SMobCUxRrATrJ9bDHlZ+Q8b23Wr0p7uFSvuZ8J1FUJk2J8UNjUi3uwqOf8178IYwDeXb6ibQJTd+8jgtckWTdePFz60Ifvl9VVXrAklERZTt4vDfnhdRTx9Xlw2g1WvaR22/VW2yMkfVIaBRPlh7ei9YeEvjCPkuDGJX/TNWuoqRqvA81WCg4UrAvTlBJxDqw82XhWaCvBVSDbYJpivwkP9LYuPPaedjGD8fKJRW6l+zMTPK0vbnJPBchuKafd3ToeCjSthnKPEkPLWwoN0Pk= ceph-2a791768-77e2-11ef-baf5-efdc52797490 2024-09-21T06:27:14.537 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtVRZT+VINosykitNt+uL0Tfgvao4CsZcdgLaZlofN3JkFhRsHWLTb/eKSqWqezPUxNHZIWFiACqKiOy1k6Sw/zOHZsJ7csgsDTEMPVDnLiUXLXkYUwtuoAIOSCvPYUcQtiTdnQrKOb6jOipdyThQTmuc+KN2tFGtnxuYBULHvVxm0KPcs1ulLuO+15RLQ8x+Ex321w49nMc78T6EE1j16O691ArVssfQqSFHp44ICA3SMobCUxRrATrJ9bDHlZ+Q8b23Wr0p7uFSvuZ8J1FUJk2J8UNjUi3uwqOf8178IYwDeXb6ibQJTd+8jgtckWTdePFz60Ifvl9VVXrAklERZTt4vDfnhdRTx9Xlw2g1WvaR22/VW2yMkfVIaBRPlh7ei9YeEvjCPkuDGJX/TNWuoqRqvA81WCg4UrAvTlBJxDqw82XhWaCvBVSDbYJpivwkP9LYuPPaedjGD8fKJRW6l+zMTPK0vbnJPBchuKafd3ToeCjSthnKPEkPLWwoN0Pk= ceph-2a791768-77e2-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T06:27:14.600 INFO:teuthology.orchestra.run.smithi100.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtVRZT+VINosykitNt+uL0Tfgvao4CsZcdgLaZlofN3JkFhRsHWLTb/eKSqWqezPUxNHZIWFiACqKiOy1k6Sw/zOHZsJ7csgsDTEMPVDnLiUXLXkYUwtuoAIOSCvPYUcQtiTdnQrKOb6jOipdyThQTmuc+KN2tFGtnxuYBULHvVxm0KPcs1ulLuO+15RLQ8x+Ex321w49nMc78T6EE1j16O691ArVssfQqSFHp44ICA3SMobCUxRrATrJ9bDHlZ+Q8b23Wr0p7uFSvuZ8J1FUJk2J8UNjUi3uwqOf8178IYwDeXb6ibQJTd+8jgtckWTdePFz60Ifvl9VVXrAklERZTt4vDfnhdRTx9Xlw2g1WvaR22/VW2yMkfVIaBRPlh7ei9YeEvjCPkuDGJX/TNWuoqRqvA81WCg4UrAvTlBJxDqw82XhWaCvBVSDbYJpivwkP9LYuPPaedjGD8fKJRW6l+zMTPK0vbnJPBchuKafd3ToeCjSthnKPEkPLWwoN0Pk= ceph-2a791768-77e2-11ef-baf5-efdc52797490 2024-09-21T06:27:14.613 DEBUG:teuthology.orchestra.run.smithi203:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtVRZT+VINosykitNt+uL0Tfgvao4CsZcdgLaZlofN3JkFhRsHWLTb/eKSqWqezPUxNHZIWFiACqKiOy1k6Sw/zOHZsJ7csgsDTEMPVDnLiUXLXkYUwtuoAIOSCvPYUcQtiTdnQrKOb6jOipdyThQTmuc+KN2tFGtnxuYBULHvVxm0KPcs1ulLuO+15RLQ8x+Ex321w49nMc78T6EE1j16O691ArVssfQqSFHp44ICA3SMobCUxRrATrJ9bDHlZ+Q8b23Wr0p7uFSvuZ8J1FUJk2J8UNjUi3uwqOf8178IYwDeXb6ibQJTd+8jgtckWTdePFz60Ifvl9VVXrAklERZTt4vDfnhdRTx9Xlw2g1WvaR22/VW2yMkfVIaBRPlh7ei9YeEvjCPkuDGJX/TNWuoqRqvA81WCg4UrAvTlBJxDqw82XhWaCvBVSDbYJpivwkP9LYuPPaedjGD8fKJRW6l+zMTPK0vbnJPBchuKafd3ToeCjSthnKPEkPLWwoN0Pk= ceph-2a791768-77e2-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T06:27:14.708 INFO:teuthology.orchestra.run.smithi203.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtVRZT+VINosykitNt+uL0Tfgvao4CsZcdgLaZlofN3JkFhRsHWLTb/eKSqWqezPUxNHZIWFiACqKiOy1k6Sw/zOHZsJ7csgsDTEMPVDnLiUXLXkYUwtuoAIOSCvPYUcQtiTdnQrKOb6jOipdyThQTmuc+KN2tFGtnxuYBULHvVxm0KPcs1ulLuO+15RLQ8x+Ex321w49nMc78T6EE1j16O691ArVssfQqSFHp44ICA3SMobCUxRrATrJ9bDHlZ+Q8b23Wr0p7uFSvuZ8J1FUJk2J8UNjUi3uwqOf8178IYwDeXb6ibQJTd+8jgtckWTdePFz60Ifvl9VVXrAklERZTt4vDfnhdRTx9Xlw2g1WvaR22/VW2yMkfVIaBRPlh7ei9YeEvjCPkuDGJX/TNWuoqRqvA81WCg4UrAvTlBJxDqw82XhWaCvBVSDbYJpivwkP9LYuPPaedjGD8fKJRW6l+zMTPK0vbnJPBchuKafd3ToeCjSthnKPEkPLWwoN0Pk= ceph-2a791768-77e2-11ef-baf5-efdc52797490 2024-09-21T06:27:14.721 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-21T06:27:14.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:14 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/188895453' entity='client.admin' 2024-09-21T06:27:14.982 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:27:17.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:16 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/482887545' entity='client.admin' 2024-09-21T06:27:17.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:16 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:17.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:16 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:17.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:16 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi067", "name": "osd_memory_target"} : dispatch 2024-09-21T06:27:17.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:16 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:17.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:16 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:27:17.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:16 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:17.085 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-21T06:27:17.085 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-21T06:27:17.772 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:27:19.046 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi100 2024-09-21T06:27:19.046 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-21T06:27:19.046 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/etc/ceph/ceph.conf 2024-09-21T06:27:19.065 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-21T06:27:19.065 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:27:19.123 INFO:tasks.cephadm:Adding host smithi100 to orchestrator... 2024-09-21T06:27:19.123 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph orch host add smithi100 2024-09-21T06:27:19.372 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:27:19.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:19 smithi067 ceph-mon[31232]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:27:19.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:19 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:19.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:19 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:27:19.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:19 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:27:19.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:19 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:27:19.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:19 smithi067 ceph-mon[31232]: Updating smithi067:/etc/ceph/ceph.conf 2024-09-21T06:27:19.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:19 smithi067 ceph-mon[31232]: Updating smithi067:/var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:27:19.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:19 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:19.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:19 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:19.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:19 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:20.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:20 smithi067 ceph-mon[31232]: Updating smithi067:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:27:20.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:20 smithi067 ceph-mon[31232]: Updating smithi067:/var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:27:21.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:21 smithi067 ceph-mon[31232]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi100", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:27:22.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:22 smithi067 ceph-mon[31232]: Deploying cephadm binary to smithi100 2024-09-21T06:27:23.674 INFO:teuthology.orchestra.run.smithi067.stdout:Added host 'smithi100' with addr '172.21.15.100' 2024-09-21T06:27:24.292 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T06:27:24.547 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:27:24.961 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:24 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:24.961 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:24 smithi067 ceph-mon[31232]: Added host smithi100 2024-09-21T06:27:24.961 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:24 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:27:24.961 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:24 smithi067 ceph-mon[31232]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:24.961 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:24 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:24.961 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:24 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:25.183 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:27:25.183 INFO:teuthology.orchestra.run.smithi067.stdout:[{"addr": "172.21.15.67", "hostname": "smithi067", "labels": [], "status": ""}, {"addr": "172.21.15.100", "hostname": "smithi100", "labels": [], "status": ""}] 2024-09-21T06:27:25.754 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi203 2024-09-21T06:27:25.755 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-21T06:27:25.755 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/etc/ceph/ceph.conf 2024-09-21T06:27:25.773 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-21T06:27:25.773 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:27:25.832 INFO:tasks.cephadm:Adding host smithi203 to orchestrator... 2024-09-21T06:27:25.832 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph orch host add smithi203 2024-09-21T06:27:26.086 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:27:26.248 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:25 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:26.249 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:25 smithi067 ceph-mon[31232]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:27:27.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:26 smithi067 ceph-mon[31232]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:28.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:28 smithi067 ceph-mon[31232]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi203", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:27:28.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:28 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:28.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:28 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:28.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:28 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:28.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:28 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:28.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:28 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi100", "name": "osd_memory_target"} : dispatch 2024-09-21T06:27:28.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:28 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:27:28.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:28 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:27:29.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:29 smithi067 ceph-mon[31232]: Updating smithi100:/etc/ceph/ceph.conf 2024-09-21T06:27:29.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:29 smithi067 ceph-mon[31232]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:29.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:29 smithi067 ceph-mon[31232]: Updating smithi100:/var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:27:29.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:29 smithi067 ceph-mon[31232]: Deploying cephadm binary to smithi203 2024-09-21T06:27:29.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:29 smithi067 ceph-mon[31232]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:27:29.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:29 smithi067 ceph-mon[31232]: Updating smithi100:/var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:27:29.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:29 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:29.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:29 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:29.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:29 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:30.538 INFO:teuthology.orchestra.run.smithi067.stdout:Added host 'smithi203' with addr '172.21.15.203' 2024-09-21T06:27:31.184 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T06:27:31.433 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:27:31.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:31 smithi067 ceph-mon[31232]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:31.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:31 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:31.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:31 smithi067 ceph-mon[31232]: Added host smithi203 2024-09-21T06:27:31.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:31 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:27:31.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:31 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:31.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:31 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:32.061 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:27:32.061 INFO:teuthology.orchestra.run.smithi067.stdout:[{"addr": "172.21.15.67", "hostname": "smithi067", "labels": [], "status": ""}, {"addr": "172.21.15.100", "hostname": "smithi100", "labels": [], "status": ""}, {"addr": "172.21.15.203", "hostname": "smithi203", "labels": [], "status": ""}] 2024-09-21T06:27:32.603 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-21T06:27:32.603 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd crush tunables default 2024-09-21T06:27:32.852 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:27:32.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:32 smithi067 ceph-mon[31232]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:32.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:32 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:32.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:32 smithi067 ceph-mon[31232]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:27:33.871 INFO:teuthology.orchestra.run.smithi067.stderr:adjusted tunables profile to default 2024-09-21T06:27:34.166 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:33 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/881364642' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T06:27:34.464 INFO:tasks.cephadm:Adding mon.a on smithi067 2024-09-21T06:27:34.464 INFO:tasks.cephadm:Adding mon.b on smithi100 2024-09-21T06:27:34.464 INFO:tasks.cephadm:Adding mon.c on smithi203 2024-09-21T06:27:34.464 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph orch apply mon '3;smithi067:172.21.15.67=a;smithi100:172.21.15.100=b;smithi203:172.21.15.203=c' 2024-09-21T06:27:34.712 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:27:35.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:34 smithi067 ceph-mon[31232]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:35.394 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:34 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/881364642' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T06:27:35.394 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:34 smithi067 ceph-mon[31232]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:27:35.394 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:34 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:35.394 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:34 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:35.394 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:34 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:35.394 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:34 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:35.394 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:34 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi203", "name": "osd_memory_target"} : dispatch 2024-09-21T06:27:35.394 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:34 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:27:35.394 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:34 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:27:35.394 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:34 smithi067 ceph-mon[31232]: Updating smithi203:/etc/ceph/ceph.conf 2024-09-21T06:27:35.394 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:34 smithi067 ceph-mon[31232]: Updating smithi203:/var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:27:35.413 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled mon update... 2024-09-21T06:27:36.020 DEBUG:teuthology.orchestra.run.smithi100:mon.b> sudo journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mon.b.service 2024-09-21T06:27:36.024 DEBUG:teuthology.orchestra.run.smithi203:mon.c> sudo journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mon.c.service 2024-09-21T06:27:36.026 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T06:27:36.027 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T06:27:36.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:35 smithi067 ceph-mon[31232]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:27:36.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:35 smithi067 ceph-mon[31232]: Updating smithi203:/var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:27:36.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:36.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:36.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:36.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:35 smithi067 ceph-mon[31232]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi067:172.21.15.67=a;smithi100:172.21.15.100=b;smithi203:172.21.15.203=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:27:36.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:35 smithi067 ceph-mon[31232]: Saving service mon spec with placement smithi067:172.21.15.67=a;smithi100:172.21.15.100=b;smithi203:172.21.15.203=c;count:3 2024-09-21T06:27:36.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:36.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:27:36.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:27:36.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:27:36.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:36.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:27:36.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:27:36.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:35 smithi067 ceph-mon[31232]: Deploying daemon mon.c on smithi203 2024-09-21T06:27:36.694 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:27:37.373 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T06:27:37.374 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":1,"fsid":"2a791768-77e2-11ef-baf5-efdc52797490","modified":"2024-09-21T06:26:12.167159Z","created":"2024-09-21T06:26:12.167159Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T06:27:37.374 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 1 2024-09-21T06:27:37.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:37 smithi067 ceph-mon[31232]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:37.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:37 smithi067 ceph-mon[31232]: from='client.? 172.21.15.203:0/139509038' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:27:39.621 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T06:27:39.621 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T06:27:39.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:39 smithi067 ceph-mon[31232]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:39.970 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:27:40.689 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T06:27:40.690 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":1,"fsid":"2a791768-77e2-11ef-baf5-efdc52797490","modified":"2024-09-21T06:26:12.167159Z","created":"2024-09-21T06:26:12.167159Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T06:27:40.690 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 1 2024-09-21T06:27:40.766 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: mon.c@-1(synchronizing).mds e1 new map 2024-09-21T06:27:40.766 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-21T06:27:40.766 INFO:journalctl@ceph.mon.c.smithi203.stdout: e1 2024-09-21T06:27:40.766 INFO:journalctl@ceph.mon.c.smithi203.stdout: btime 2024-09-21T06:26:15:222052+0000 2024-09-21T06:27:40.766 INFO:journalctl@ceph.mon.c.smithi203.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T06:27:40.767 INFO:journalctl@ceph.mon.c.smithi203.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T06:27:40.767 INFO:journalctl@ceph.mon.c.smithi203.stdout: legacy client fscid: -1 2024-09-21T06:27:40.767 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2024-09-21T06:27:40.767 INFO:journalctl@ceph.mon.c.smithi203.stdout: No filesystems configured 2024-09-21T06:27:40.767 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-21T06:27:40.767 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-21T06:27:40.767 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-21T06:27:40.767 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-21T06:27:40.767 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-21T06:27:40.767 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-21T06:27:41.094 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T06:27:41.095 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T06:27:41.095 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T06:27:41.095 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T06:27:41.095 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:41.095 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:27:41.095 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:41.095 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi203", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:27:41.095 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:41.095 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:41.095 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:41.095 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:41.095 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi100", "name": "osd_memory_target"} : dispatch 2024-09-21T06:27:41.096 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:27:41.096 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:27:41.096 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: Updating smithi100:/etc/ceph/ceph.conf 2024-09-21T06:27:41.096 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:41.096 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: Updating smithi100:/var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:27:41.096 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: Deploying cephadm binary to smithi203 2024-09-21T06:27:41.096 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:27:41.096 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: Updating smithi100:/var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:27:41.096 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:41.096 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:41.096 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:41.096 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:41.097 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:41.097 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: Added host smithi203 2024-09-21T06:27:41.097 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:27:41.097 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:41.097 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:41.097 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:41.097 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:41.097 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:27:41.097 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/881364642' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T06:27:41.098 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:41.098 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/881364642' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T06:27:41.098 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:27:41.098 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:41.098 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:41.098 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:41.098 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:41.098 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi203", "name": "osd_memory_target"} : dispatch 2024-09-21T06:27:41.098 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:27:41.098 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:27:41.098 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: Updating smithi203:/etc/ceph/ceph.conf 2024-09-21T06:27:41.098 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: Updating smithi203:/var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:27:41.098 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:27:41.099 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: Updating smithi203:/var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:27:41.099 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:41.099 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:41.099 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:41.099 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi067:172.21.15.67=a;smithi100:172.21.15.100=b;smithi203:172.21.15.203=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:27:41.099 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: Saving service mon spec with placement smithi067:172.21.15.67=a;smithi100:172.21.15.100=b;smithi203:172.21.15.203=c;count:3 2024-09-21T06:27:41.099 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:41.099 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:27:41.099 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:27:41.099 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:27:41.100 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:41.100 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:27:41.100 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:27:41.100 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: Deploying daemon mon.c on smithi203 2024-09-21T06:27:41.100 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:41.100 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: from='client.? 172.21.15.203:0/139509038' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:27:41.100 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:41.100 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:40 smithi203 ceph-mon[33735]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-21T06:27:41.100 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:41 smithi203 bash[33695]: 258ab452f69ce6510b5dcb94712584830e64bebb4ed68c12b243a14d7d297a69 2024-09-21T06:27:41.455 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:41 smithi203 systemd[1]: Started Ceph mon.c for 2a791768-77e2-11ef-baf5-efdc52797490. 2024-09-21T06:27:42.485 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T06:27:42.486 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T06:27:42.749 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:27:46.182 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:45 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:27:46.182 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:45 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:27:46.182 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:45 smithi203 ceph-mon[33735]: mon.a calling monitor election 2024-09-21T06:27:46.182 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:45 smithi203 ceph-mon[33735]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:46.182 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:45 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:27:46.182 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:45 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:27:46.182 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:45 smithi203 ceph-mon[33735]: mon.c calling monitor election 2024-09-21T06:27:46.183 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:45 smithi203 ceph-mon[33735]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:46.183 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:45 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:27:46.183 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:45 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:27:46.183 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:45 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:27:46.183 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:45 smithi203 ceph-mon[33735]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T06:27:46.183 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:45 smithi203 ceph-mon[33735]: monmap epoch 2 2024-09-21T06:27:46.183 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:45 smithi203 ceph-mon[33735]: fsid 2a791768-77e2-11ef-baf5-efdc52797490 2024-09-21T06:27:46.183 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:45 smithi203 ceph-mon[33735]: last_changed 2024-09-21T06:27:40.770993+0000 2024-09-21T06:27:46.183 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:45 smithi203 ceph-mon[33735]: created 2024-09-21T06:26:12.167159+0000 2024-09-21T06:27:46.183 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:45 smithi203 ceph-mon[33735]: min_mon_release 19 (squid) 2024-09-21T06:27:46.183 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:45 smithi203 ceph-mon[33735]: election_strategy: 1 2024-09-21T06:27:46.183 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:45 smithi203 ceph-mon[33735]: 0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-09-21T06:27:46.183 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:45 smithi203 ceph-mon[33735]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-09-21T06:27:46.183 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:45 smithi203 ceph-mon[33735]: fsmap 2024-09-21T06:27:46.183 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:45 smithi203 ceph-mon[33735]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:27:46.184 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:45 smithi203 ceph-mon[33735]: mgrmap e13: x(active, since 42s) 2024-09-21T06:27:46.184 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:45 smithi203 ceph-mon[33735]: overall HEALTH_OK 2024-09-21T06:27:46.184 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:45 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:46.184 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:45 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:46.184 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:45 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:46.184 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:45 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:27:46.184 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:45 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:27:46.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:45 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:27:46.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:45 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:27:46.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:45 smithi067 ceph-mon[31232]: mon.a calling monitor election 2024-09-21T06:27:46.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:45 smithi067 ceph-mon[31232]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:46.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:45 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:27:46.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:45 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:27:46.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:45 smithi067 ceph-mon[31232]: mon.c calling monitor election 2024-09-21T06:27:46.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:45 smithi067 ceph-mon[31232]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:46.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:45 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:27:46.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:45 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:27:46.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:45 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:27:46.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:45 smithi067 ceph-mon[31232]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T06:27:46.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:45 smithi067 ceph-mon[31232]: monmap epoch 2 2024-09-21T06:27:46.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:45 smithi067 ceph-mon[31232]: fsid 2a791768-77e2-11ef-baf5-efdc52797490 2024-09-21T06:27:46.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:45 smithi067 ceph-mon[31232]: last_changed 2024-09-21T06:27:40.770993+0000 2024-09-21T06:27:46.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:45 smithi067 ceph-mon[31232]: created 2024-09-21T06:26:12.167159+0000 2024-09-21T06:27:46.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:45 smithi067 ceph-mon[31232]: min_mon_release 19 (squid) 2024-09-21T06:27:46.294 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:45 smithi067 ceph-mon[31232]: election_strategy: 1 2024-09-21T06:27:46.294 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:45 smithi067 ceph-mon[31232]: 0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-09-21T06:27:46.294 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:45 smithi067 ceph-mon[31232]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-09-21T06:27:46.294 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:45 smithi067 ceph-mon[31232]: fsmap 2024-09-21T06:27:46.294 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:45 smithi067 ceph-mon[31232]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:27:46.294 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:45 smithi067 ceph-mon[31232]: mgrmap e13: x(active, since 42s) 2024-09-21T06:27:46.294 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:45 smithi067 ceph-mon[31232]: overall HEALTH_OK 2024-09-21T06:27:46.294 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:45 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:46.294 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:45 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:46.294 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:45 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:46.294 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:45 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:27:46.294 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:45 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:27:46.439 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T06:27:46.439 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":2,"fsid":"2a791768-77e2-11ef-baf5-efdc52797490","modified":"2024-09-21T06:27:40.770993Z","created":"2024-09-21T06:26:12.167159Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:3300","nonce":0},{"type":"v1","addr":"172.21.15.203:6789","nonce":0}]},"addr":"172.21.15.203:6789/0","public_addr":"172.21.15.203:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T06:27:46.439 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 2 2024-09-21T06:27:46.976 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:46 smithi203 ceph-mon[33735]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:46.976 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:46 smithi203 ceph-mon[33735]: Deploying daemon mon.b on smithi100 2024-09-21T06:27:46.976 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:46 smithi203 ceph-mon[33735]: from='client.? 172.21.15.203:0/2770418053' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:27:46.976 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:46 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:27:47.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:46 smithi067 ceph-mon[31232]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:47.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:46 smithi067 ceph-mon[31232]: Deploying daemon mon.b on smithi100 2024-09-21T06:27:47.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:46 smithi067 ceph-mon[31232]: from='client.? 172.21.15.203:0/2770418053' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:27:47.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:46 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:27:48.014 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T06:27:48.014 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T06:27:48.269 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:27:48.945 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:48 smithi203 ceph-mon[33735]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:48.945 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T06:27:48.946 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":2,"fsid":"2a791768-77e2-11ef-baf5-efdc52797490","modified":"2024-09-21T06:27:40.770993Z","created":"2024-09-21T06:26:12.167159Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:3300","nonce":0},{"type":"v1","addr":"172.21.15.203:6789","nonce":0}]},"addr":"172.21.15.203:6789/0","public_addr":"172.21.15.203:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T06:27:48.946 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 2 2024-09-21T06:27:49.169 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:48 smithi067 ceph-mon[31232]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:49.337 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:49 smithi100 ceph-mon[33401]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-21T06:27:49.338 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:49 smithi100 bash[33381]: f935de8304a8af0afce1d81ccae64dc73572b8d68734732dbbba3c6b4132ece2 2024-09-21T06:27:49.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:49 smithi100 systemd[1]: Started Ceph mon.b for 2a791768-77e2-11ef-baf5-efdc52797490. 2024-09-21T06:27:50.544 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T06:27:50.544 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T06:27:50.797 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:27:54.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:54 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:27:54.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:54 smithi203 ceph-mon[33735]: mon.a calling monitor election 2024-09-21T06:27:54.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:54 smithi203 ceph-mon[33735]: mon.c calling monitor election 2024-09-21T06:27:54.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:54 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:27:54.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:54 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:27:54.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:54 smithi203 ceph-mon[33735]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:54.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:54 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:27:54.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:54 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:27:54.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:54 smithi203 ceph-mon[33735]: mon.b calling monitor election 2024-09-21T06:27:54.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:54 smithi203 ceph-mon[33735]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:54.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:54 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:27:54.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:54 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:27:54.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:54 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:27:54.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:54 smithi203 ceph-mon[33735]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T06:27:54.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:54 smithi203 ceph-mon[33735]: monmap epoch 3 2024-09-21T06:27:54.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:54 smithi203 ceph-mon[33735]: fsid 2a791768-77e2-11ef-baf5-efdc52797490 2024-09-21T06:27:54.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:54 smithi203 ceph-mon[33735]: last_changed 2024-09-21T06:27:49.327115+0000 2024-09-21T06:27:54.734 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:54 smithi203 ceph-mon[33735]: created 2024-09-21T06:26:12.167159+0000 2024-09-21T06:27:54.734 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:54 smithi203 ceph-mon[33735]: min_mon_release 19 (squid) 2024-09-21T06:27:54.734 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:54 smithi203 ceph-mon[33735]: election_strategy: 1 2024-09-21T06:27:54.734 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:54 smithi203 ceph-mon[33735]: 0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-09-21T06:27:54.734 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:54 smithi203 ceph-mon[33735]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-09-21T06:27:54.734 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:54 smithi203 ceph-mon[33735]: 2: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.b 2024-09-21T06:27:54.734 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:54 smithi203 ceph-mon[33735]: fsmap 2024-09-21T06:27:54.734 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:54 smithi203 ceph-mon[33735]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:27:54.734 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:54 smithi203 ceph-mon[33735]: mgrmap e13: x(active, since 50s) 2024-09-21T06:27:54.734 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:54 smithi203 ceph-mon[33735]: overall HEALTH_OK 2024-09-21T06:27:54.734 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:54 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:54.734 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:54 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:54.734 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:54 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:54.734 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:54 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:54.760 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T06:27:54.761 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":3,"fsid":"2a791768-77e2-11ef-baf5-efdc52797490","modified":"2024-09-21T06:27:49.327115Z","created":"2024-09-21T06:26:12.167159Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:3300","nonce":0},{"type":"v1","addr":"172.21.15.203:6789","nonce":0}]},"addr":"172.21.15.203:6789/0","public_addr":"172.21.15.203:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:3300","nonce":0},{"type":"v1","addr":"172.21.15.100:6789","nonce":0}]},"addr":"172.21.15.100:6789/0","public_addr":"172.21.15.100:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-21T06:27:54.761 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 3 2024-09-21T06:27:54.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:54 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:27:54.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:54 smithi067 ceph-mon[31232]: mon.a calling monitor election 2024-09-21T06:27:54.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:54 smithi067 ceph-mon[31232]: mon.c calling monitor election 2024-09-21T06:27:54.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:54 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:27:54.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:54 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:27:54.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:54 smithi067 ceph-mon[31232]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:54.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:54 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:27:54.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:54 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:27:54.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:54 smithi067 ceph-mon[31232]: mon.b calling monitor election 2024-09-21T06:27:54.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:54 smithi067 ceph-mon[31232]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:54.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:54 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:27:54.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:54 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:27:54.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:54 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:27:54.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:54 smithi067 ceph-mon[31232]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T06:27:54.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:54 smithi067 ceph-mon[31232]: monmap epoch 3 2024-09-21T06:27:54.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:54 smithi067 ceph-mon[31232]: fsid 2a791768-77e2-11ef-baf5-efdc52797490 2024-09-21T06:27:54.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:54 smithi067 ceph-mon[31232]: last_changed 2024-09-21T06:27:49.327115+0000 2024-09-21T06:27:54.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:54 smithi067 ceph-mon[31232]: created 2024-09-21T06:26:12.167159+0000 2024-09-21T06:27:54.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:54 smithi067 ceph-mon[31232]: min_mon_release 19 (squid) 2024-09-21T06:27:54.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:54 smithi067 ceph-mon[31232]: election_strategy: 1 2024-09-21T06:27:54.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:54 smithi067 ceph-mon[31232]: 0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-09-21T06:27:54.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:54 smithi067 ceph-mon[31232]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-09-21T06:27:54.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:54 smithi067 ceph-mon[31232]: 2: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.b 2024-09-21T06:27:54.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:54 smithi067 ceph-mon[31232]: fsmap 2024-09-21T06:27:54.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:54 smithi067 ceph-mon[31232]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:27:54.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:54 smithi067 ceph-mon[31232]: mgrmap e13: x(active, since 50s) 2024-09-21T06:27:54.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:54 smithi067 ceph-mon[31232]: overall HEALTH_OK 2024-09-21T06:27:54.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:54 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:54.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:54 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:54.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:54 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:54.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:54 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:54.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:54 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:27:54.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:54 smithi100 ceph-mon[33401]: mon.a calling monitor election 2024-09-21T06:27:54.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:54 smithi100 ceph-mon[33401]: mon.c calling monitor election 2024-09-21T06:27:54.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:54 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:27:54.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:54 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:27:54.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:54 smithi100 ceph-mon[33401]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:54.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:54 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:27:54.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:54 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:27:54.806 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:54 smithi100 ceph-mon[33401]: mon.b calling monitor election 2024-09-21T06:27:54.806 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:54 smithi100 ceph-mon[33401]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:54.806 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:54 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:27:54.806 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:54 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:27:54.806 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:54 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:27:54.806 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:54 smithi100 ceph-mon[33401]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T06:27:54.806 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:54 smithi100 ceph-mon[33401]: monmap epoch 3 2024-09-21T06:27:54.806 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:54 smithi100 ceph-mon[33401]: fsid 2a791768-77e2-11ef-baf5-efdc52797490 2024-09-21T06:27:54.806 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:54 smithi100 ceph-mon[33401]: last_changed 2024-09-21T06:27:49.327115+0000 2024-09-21T06:27:54.806 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:54 smithi100 ceph-mon[33401]: created 2024-09-21T06:26:12.167159+0000 2024-09-21T06:27:54.806 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:54 smithi100 ceph-mon[33401]: min_mon_release 19 (squid) 2024-09-21T06:27:54.807 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:54 smithi100 ceph-mon[33401]: election_strategy: 1 2024-09-21T06:27:54.807 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:54 smithi100 ceph-mon[33401]: 0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-09-21T06:27:54.807 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:54 smithi100 ceph-mon[33401]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-09-21T06:27:54.807 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:54 smithi100 ceph-mon[33401]: 2: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.b 2024-09-21T06:27:54.807 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:54 smithi100 ceph-mon[33401]: fsmap 2024-09-21T06:27:54.807 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:54 smithi100 ceph-mon[33401]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:27:54.807 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:54 smithi100 ceph-mon[33401]: mgrmap e13: x(active, since 50s) 2024-09-21T06:27:54.807 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:54 smithi100 ceph-mon[33401]: overall HEALTH_OK 2024-09-21T06:27:54.807 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:54 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:54.807 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:54 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:54.807 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:54 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:54.808 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:54 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:55.429 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-21T06:27:55.429 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph config generate-minimal-conf 2024-09-21T06:27:55.691 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:27:55.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:55 smithi203 ceph-mon[33735]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:55.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:55 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:27:55.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:55 smithi203 ceph-mon[33735]: from='client.? 172.21.15.203:0/592224062' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:27:55.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:55 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:55.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:55 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:55.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:55 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:27:55.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:55 smithi067 ceph-mon[31232]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:55.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:55 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:27:55.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:55 smithi067 ceph-mon[31232]: from='client.? 172.21.15.203:0/592224062' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:27:55.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:55 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:55.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:55 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:55.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:55 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:27:55.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:55 smithi100 ceph-mon[33401]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:55.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:55 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:27:55.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:55 smithi100 ceph-mon[33401]: from='client.? 172.21.15.203:0/592224062' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:27:55.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:55 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:55.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:55 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:55.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:55 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:27:56.321 INFO:teuthology.orchestra.run.smithi067.stdout:# minimal ceph.conf for 2a791768-77e2-11ef-baf5-efdc52797490 2024-09-21T06:27:56.321 INFO:teuthology.orchestra.run.smithi067.stdout:[global] 2024-09-21T06:27:56.321 INFO:teuthology.orchestra.run.smithi067.stdout: fsid = 2a791768-77e2-11ef-baf5-efdc52797490 2024-09-21T06:27:56.322 INFO:teuthology.orchestra.run.smithi067.stdout: mon_host = [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] 2024-09-21T06:27:56.974 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-21T06:27:56.974 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-21T06:27:56.974 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T06:27:57.004 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-21T06:27:57.005 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:27:57.073 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-21T06:27:57.073 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T06:27:57.101 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-21T06:27:57.101 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:27:57.167 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-21T06:27:57.167 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T06:27:57.194 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-21T06:27:57.194 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:27:57.259 INFO:tasks.cephadm:Adding mgr.x on smithi067 2024-09-21T06:27:57.259 INFO:tasks.cephadm:Adding mgr.y on smithi100 2024-09-21T06:27:57.259 INFO:tasks.cephadm:Adding mgr.z on smithi203 2024-09-21T06:27:57.259 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph orch apply mgr '3;smithi067=x;smithi100=y;smithi203=z' 2024-09-21T06:27:57.322 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:57 smithi203 ceph-mon[33735]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:57.322 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.322 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.322 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:27:57.322 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:27:57.322 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:57 smithi203 ceph-mon[33735]: Updating smithi067:/etc/ceph/ceph.conf 2024-09-21T06:27:57.322 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:57 smithi203 ceph-mon[33735]: Updating smithi100:/etc/ceph/ceph.conf 2024-09-21T06:27:57.322 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:57 smithi203 ceph-mon[33735]: Updating smithi203:/etc/ceph/ceph.conf 2024-09-21T06:27:57.322 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:57 smithi203 ceph-mon[33735]: Updating smithi203:/var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:27:57.322 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:57 smithi203 ceph-mon[33735]: Updating smithi100:/var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:27:57.322 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:57 smithi203 ceph-mon[33735]: Updating smithi067:/var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:27:57.322 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:57 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/143509790' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:27:57.322 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.323 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.323 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.323 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.323 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.323 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.323 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.323 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.323 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.323 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.323 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.323 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:57 smithi203 ceph-mon[33735]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T06:27:57.323 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:27:57.323 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:27:57.323 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:27:57.323 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:57 smithi203 ceph-mon[33735]: Reconfiguring daemon mon.a on smithi067 2024-09-21T06:27:57.377 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:57 smithi067 ceph-mon[31232]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:57.377 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.377 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.377 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:27:57.377 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:27:57.377 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:57 smithi067 ceph-mon[31232]: Updating smithi067:/etc/ceph/ceph.conf 2024-09-21T06:27:57.377 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:57 smithi067 ceph-mon[31232]: Updating smithi100:/etc/ceph/ceph.conf 2024-09-21T06:27:57.377 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:57 smithi067 ceph-mon[31232]: Updating smithi203:/etc/ceph/ceph.conf 2024-09-21T06:27:57.378 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:57 smithi067 ceph-mon[31232]: Updating smithi203:/var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:27:57.378 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:57 smithi067 ceph-mon[31232]: Updating smithi100:/var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:27:57.378 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:57 smithi067 ceph-mon[31232]: Updating smithi067:/var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:27:57.378 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:57 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/143509790' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:27:57.378 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.378 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.378 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.378 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.378 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.378 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.378 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.378 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.379 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.379 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.379 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.379 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:57 smithi067 ceph-mon[31232]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T06:27:57.379 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:27:57.379 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:27:57.379 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:27:57.379 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:57 smithi067 ceph-mon[31232]: Reconfiguring daemon mon.a on smithi067 2024-09-21T06:27:57.536 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:27:57.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:57 smithi100 ceph-mon[33401]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:57.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:27:57.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:27:57.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:57 smithi100 ceph-mon[33401]: Updating smithi067:/etc/ceph/ceph.conf 2024-09-21T06:27:57.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:57 smithi100 ceph-mon[33401]: Updating smithi100:/etc/ceph/ceph.conf 2024-09-21T06:27:57.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:57 smithi100 ceph-mon[33401]: Updating smithi203:/etc/ceph/ceph.conf 2024-09-21T06:27:57.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:57 smithi100 ceph-mon[33401]: Updating smithi203:/var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:27:57.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:57 smithi100 ceph-mon[33401]: Updating smithi100:/var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:27:57.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:57 smithi100 ceph-mon[33401]: Updating smithi067:/var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:27:57.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:57 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/143509790' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:27:57.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:57.556 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:57 smithi100 ceph-mon[33401]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T06:27:57.556 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:27:57.556 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:27:57.556 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:27:57.556 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:57 smithi100 ceph-mon[33401]: Reconfiguring daemon mon.a on smithi067 2024-09-21T06:27:58.169 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled mgr update... 2024-09-21T06:27:58.812 DEBUG:teuthology.orchestra.run.smithi100:mgr.y> sudo journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mgr.y.service 2024-09-21T06:27:58.815 DEBUG:teuthology.orchestra.run.smithi203:mgr.z> sudo journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mgr.z.service 2024-09-21T06:27:58.855 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-09-21T06:27:58.855 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph config set client client_mount_timeout 600 2024-09-21T06:27:58.858 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-09-21T06:27:58.858 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph config set client debug_client 20 2024-09-21T06:27:58.901 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-09-21T06:27:58.901 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph config set client debug_ms 1 2024-09-21T06:27:58.944 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-09-21T06:27:58.944 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph config set client rados_mon_op_timeout 900 2024-09-21T06:27:58.987 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-09-21T06:27:58.987 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph config set client rados_osd_op_timeout 900 2024-09-21T06:27:59.030 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-09-21T06:27:59.030 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds 20 2024-09-21T06:27:59.073 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-09-21T06:27:59.073 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds_balancer 20 2024-09-21T06:27:59.118 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-09-21T06:27:59.118 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph config set mds debug_ms 2 2024-09-21T06:27:59.129 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:58 smithi100 ceph-mon[33401]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:59.129 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:58 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:59.129 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:58 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:59.129 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:58 smithi100 ceph-mon[33401]: Reconfiguring mon.b (monmap changed)... 2024-09-21T06:27:59.130 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:58 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:27:59.130 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:58 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:27:59.130 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:58 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:27:59.130 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:58 smithi100 ceph-mon[33401]: Reconfiguring daemon mon.b on smithi100 2024-09-21T06:27:59.130 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:58 smithi100 ceph-mon[33401]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi067=x;smithi100=y;smithi203=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:27:59.130 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:58 smithi100 ceph-mon[33401]: Saving service mgr spec with placement smithi067=x;smithi100=y;smithi203=z;count:3 2024-09-21T06:27:59.130 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:27:58 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:59.152 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:58 smithi067 ceph-mon[31232]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:59.152 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:58 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:59.153 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:58 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:59.153 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:58 smithi067 ceph-mon[31232]: Reconfiguring mon.b (monmap changed)... 2024-09-21T06:27:59.153 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:58 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:27:59.153 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:58 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:27:59.153 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:58 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:27:59.153 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:58 smithi067 ceph-mon[31232]: Reconfiguring daemon mon.b on smithi100 2024-09-21T06:27:59.153 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:58 smithi067 ceph-mon[31232]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi067=x;smithi100=y;smithi203=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:27:59.153 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:58 smithi067 ceph-mon[31232]: Saving service mgr spec with placement smithi067=x;smithi100=y;smithi203=z;count:3 2024-09-21T06:27:59.153 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:27:58 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:59.157 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-09-21T06:27:59.157 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-09-21T06:27:59.182 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:27:59.186 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-09-21T06:27:59.186 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_merge_size 5 2024-09-21T06:27:59.234 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-09-21T06:27:59.235 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_bits 3 2024-09-21T06:27:59.246 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:27:59.248 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-09-21T06:27:59.248 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_size 100 2024-09-21T06:27:59.291 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-09-21T06:27:59.291 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_frag True 2024-09-21T06:27:59.334 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-09-21T06:27:59.334 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_scatterstat True 2024-09-21T06:27:59.386 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-09-21T06:27:59.386 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph config set mds mds_op_complaint_time 180 2024-09-21T06:27:59.440 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-09-21T06:27:59.440 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph config set mds mds_verify_scatter True 2024-09-21T06:27:59.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:58 smithi203 ceph-mon[33735]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:27:59.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:58 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:59.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:58 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:59.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:58 smithi203 ceph-mon[33735]: Reconfiguring mon.b (monmap changed)... 2024-09-21T06:27:59.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:58 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:27:59.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:58 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:27:59.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:58 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:27:59.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:58 smithi203 ceph-mon[33735]: Reconfiguring daemon mon.b on smithi100 2024-09-21T06:27:59.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:58 smithi203 ceph-mon[33735]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi067=x;smithi100=y;smithi203=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:27:59.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:58 smithi203 ceph-mon[33735]: Saving service mgr spec with placement smithi067=x;smithi100=y;smithi203=z;count:3 2024-09-21T06:27:59.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:27:58 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:27:59.498 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-09-21T06:27:59.499 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph config set mds osd_op_complaint_time 180 2024-09-21T06:27:59.544 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T06:27:59.544 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph config set mds rados_mon_op_timeout 900 2024-09-21T06:27:59.598 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T06:27:59.598 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph config set mds rados_osd_op_timeout 900 2024-09-21T06:27:59.652 INFO:tasks.cephadm:setting config [mgr] client_mount_timeout = 30 2024-09-21T06:27:59.652 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph config set mgr client_mount_timeout 30 2024-09-21T06:27:59.706 INFO:tasks.cephadm:setting config [mgr] debug_client = 20 2024-09-21T06:27:59.707 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph config set mgr debug_client 20 2024-09-21T06:27:59.755 INFO:tasks.cephadm:setting config [mgr] debug_mgr = 20 2024-09-21T06:27:59.756 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph config set mgr debug_mgr 20 2024-09-21T06:27:59.807 INFO:tasks.cephadm:setting config [mgr] debug_ms = 1 2024-09-21T06:27:59.807 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph config set mgr debug_ms 1 2024-09-21T06:27:59.851 INFO:tasks.cephadm:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-21T06:27:59.851 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph config set mgr mon_warn_on_pool_no_app False 2024-09-21T06:27:59.895 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-09-21T06:27:59.895 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph config set mon mon_op_complaint_time 120 2024-09-21T06:27:59.947 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-09-21T06:27:59.947 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph config set osd osd_op_complaint_time 180 2024-09-21T06:27:59.991 DEBUG:tasks.cephadm:set 26 configs 2024-09-21T06:27:59.991 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2a791768-77e2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi067') 2024-09-21T06:28:00.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:00 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:00.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:00 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:00.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:00 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:00.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:00 smithi067 ceph-mon[31232]: Reconfiguring mon.c (monmap changed)... 2024-09-21T06:28:00.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:00 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:28:00.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:00 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:28:00.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:00 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:00.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:00 smithi067 ceph-mon[31232]: Reconfiguring daemon mon.c on smithi203 2024-09-21T06:28:00.334 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:00 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:00.334 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:00 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:00.334 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:00 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:00.334 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:00 smithi203 ceph-mon[33735]: Reconfiguring mon.c (monmap changed)... 2024-09-21T06:28:00.334 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:00 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:28:00.334 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:00 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:28:00.334 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:00 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:00.334 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:00 smithi203 ceph-mon[33735]: Reconfiguring daemon mon.c on smithi203 2024-09-21T06:28:00.472 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:28:00.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:00 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:00.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:00 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:00.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:00 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:00.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:00 smithi100 ceph-mon[33401]: Reconfiguring mon.c (monmap changed)... 2024-09-21T06:28:00.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:00 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:28:00.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:00 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:28:00.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:00 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:00.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:00 smithi100 ceph-mon[33401]: Reconfiguring daemon mon.c on smithi203 2024-09-21T06:28:01.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:01.474+0000 7f383cfdd640 1 -- 172.21.15.67:0/1658680493 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f382403fb20 msgr2=0x7f3824042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:01.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:01.474+0000 7f383cfdd640 1 --2- 172.21.15.67:0/1658680493 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f382403fb20 0x7f3824042030 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3820005fd0 tx=0x7f3820005850 comp rx=0 tx=0).stop 2024-09-21T06:28:01.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:01.474+0000 7f383cfdd640 1 -- 172.21.15.67:0/1658680493 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3838102640 msgr2=0x7f3838199c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:01.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:01.474+0000 7f383cfdd640 1 --2- 172.21.15.67:0/1658680493 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3838102640 0x7f3838199c60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3828005940 tx=0x7f38280042f0 comp rx=0 tx=0).stop 2024-09-21T06:28:01.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:01.474+0000 7f383cfdd640 1 -- 172.21.15.67:0/1658680493 shutdown_connections 2024-09-21T06:28:01.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:01.474+0000 7f383cfdd640 1 --2- 172.21.15.67:0/1658680493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38381036d0 0x7f383819a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:01.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:01.474+0000 7f383cfdd640 1 --2- 172.21.15.67:0/1658680493 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f382403fb20 0x7f3824042030 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:01.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:01.474+0000 7f383cfdd640 1 --2- 172.21.15.67:0/1658680493 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3838104070 0x7f383819ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:01.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:01.474+0000 7f383cfdd640 1 --2- 172.21.15.67:0/1658680493 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3838102640 0x7f3838199c60 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:01.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:01.474+0000 7f383cfdd640 1 -- 172.21.15.67:0/1658680493 >> 172.21.15.67:0/1658680493 conn(0x7f38380fdfb0 msgr2=0x7f38380ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:01.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:01.474+0000 7f383cfdd640 1 -- 172.21.15.67:0/1658680493 shutdown_connections 2024-09-21T06:28:01.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:01.474+0000 7f383cfdd640 1 -- 172.21.15.67:0/1658680493 wait complete. 2024-09-21T06:28:01.520 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:01 smithi067 ceph-mon[31232]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:01.520 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:01 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:01.520 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:01 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:01.520 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:01 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:28:01.520 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:01 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:01.520 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:01 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:28:01.520 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:01 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:01.520 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:01 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:28:01.520 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:01 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T06:28:01.520 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:01 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:28:01.520 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:01 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:01.520 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:01 smithi067 ceph-mon[31232]: Deploying daemon mgr.z on smithi203 2024-09-21T06:28:01.520 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:01 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3480398860' entity='client.admin' 2024-09-21T06:28:01.521 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:01 smithi067 ceph-mon[31232]: from='client.? ' entity='client.admin' 2024-09-21T06:28:01.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:01 smithi203 ceph-mon[33735]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:01.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:01 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:01.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:01 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:01.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:01 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:28:01.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:01 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:01.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:01 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:28:01.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:01 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:01.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:01 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:28:01.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:01 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T06:28:01.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:01 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:28:01.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:01 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:01.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:01 smithi203 ceph-mon[33735]: Deploying daemon mgr.z on smithi203 2024-09-21T06:28:01.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:01 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3480398860' entity='client.admin' 2024-09-21T06:28:01.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:01 smithi203 ceph-mon[33735]: from='client.? ' entity='client.admin' 2024-09-21T06:28:01.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:01 smithi100 ceph-mon[33401]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:01.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:01 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:01.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:01 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:01.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:01 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:28:01.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:01 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:01.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:01 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:28:01.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:01 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:01.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:01 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:28:01.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:01 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T06:28:01.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:01 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:28:01.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:01 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:01.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:01 smithi100 ceph-mon[33401]: Deploying daemon mgr.z on smithi203 2024-09-21T06:28:01.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:01 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3480398860' entity='client.admin' 2024-09-21T06:28:01.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:01 smithi100 ceph-mon[33401]: from='client.? ' entity='client.admin' 2024-09-21T06:28:02.399 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 21 06:28:02 smithi203 systemd[1]: Starting Ceph mgr.z for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:28:02.449 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2a791768-77e2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi067') 2024-09-21T06:28:02.649 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:02 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1658680493' entity='client.admin' 2024-09-21T06:28:02.649 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 21 06:28:02 smithi203 podman[34990]: 2024-09-21 06:28:02.299917252 +0000 UTC m=+0.015772204 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:28:02.649 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 21 06:28:02 smithi203 podman[34990]: 2024-09-21 06:28:02.400732106 +0000 UTC m=+0.116587043 container create 37bfc5ac89ca220b0a0bea9bca5c76a57ca7ded7a0a53b4c2887c993da5d0253 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-z, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-21T06:28:02.649 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 21 06:28:02 smithi203 podman[34990]: 2024-09-21 06:28:02.578273763 +0000 UTC m=+0.294128711 container init 37bfc5ac89ca220b0a0bea9bca5c76a57ca7ded7a0a53b4c2887c993da5d0253 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-z, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-21T06:28:02.649 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 21 06:28:02 smithi203 podman[34990]: 2024-09-21 06:28:02.581158829 +0000 UTC m=+0.297013783 container start 37bfc5ac89ca220b0a0bea9bca5c76a57ca7ded7a0a53b4c2887c993da5d0253 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-21T06:28:02.649 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 21 06:28:02 smithi203 ceph-mgr[35009]: -- 172.21.15.203:0/1635657517 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563c9881b0e0 con 0x563c9d5be480 2024-09-21T06:28:02.650 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 21 06:28:02 smithi203 bash[34990]: 37bfc5ac89ca220b0a0bea9bca5c76a57ca7ded7a0a53b4c2887c993da5d0253 2024-09-21T06:28:02.650 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 21 06:28:02 smithi203 systemd[1]: Started Ceph mgr.z for 2a791768-77e2-11ef-baf5-efdc52797490. 2024-09-21T06:28:02.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:02 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1658680493' entity='client.admin' 2024-09-21T06:28:02.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:02 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1658680493' entity='client.admin' 2024-09-21T06:28:02.869 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:28:02.877 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:28:02.877 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:28:02.880 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:28:02.886 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:28:02.894 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:28:02.894 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:28:02.894 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:28:02.898 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:28:02.898 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:28:02.904 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:28:02.914 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:28:02.929 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:28:02.976 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:28:03.018 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2a791768-77e2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi067') 2024-09-21T06:28:03.018 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2a791768-77e2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi067') 2024-09-21T06:28:03.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:03 smithi203 ceph-mon[33735]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:03.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:03 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:03.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:03 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:03.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:03 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:03.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:03 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:28:03.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:03 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T06:28:03.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:03 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:28:03.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:03 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:03.765 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:03 smithi067 ceph-mon[31232]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:03.765 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:03 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:03.765 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:03 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:03.766 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:03 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:03.766 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:03 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:28:03.766 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:03 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T06:28:03.766 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:03 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:28:03.766 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:03 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:03.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:03 smithi100 ceph-mon[33401]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:03.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:03 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:03.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:03 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:03.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:03 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:03.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:03 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:28:03.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:03 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T06:28:03.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:03 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:28:03.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:03 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:04.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.301+0000 7fe9716db640 1 -- 172.21.15.67:0/1350263058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe96c106900 msgr2=0x7fe96c108d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:04.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.301+0000 7fe9716db640 1 --2- 172.21.15.67:0/1350263058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe96c106900 0x7fe96c108d60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe960009860 tx=0x7fe96002a340 comp rx=0 tx=0).stop 2024-09-21T06:28:04.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.301+0000 7fe9716db640 1 -- 172.21.15.67:0/1350263058 shutdown_connections 2024-09-21T06:28:04.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.301+0000 7fe9716db640 1 --2- 172.21.15.67:0/1350263058 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe96c1094a0 0x7fe96c10b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:04.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.301+0000 7fe9716db640 1 --2- 172.21.15.67:0/1350263058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe96c106900 0x7fe96c108d60 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:04.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.301+0000 7fe9716db640 1 --2- 172.21.15.67:0/1350263058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe96c06c340 0x7fe96c1063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:04.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.301+0000 7fe9716db640 1 -- 172.21.15.67:0/1350263058 >> 172.21.15.67:0/1350263058 conn(0x7fe96c0fdfe0 msgr2=0x7fe96c100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:04.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.301+0000 7fe9716db640 1 -- 172.21.15.67:0/1350263058 shutdown_connections 2024-09-21T06:28:04.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.301+0000 7fe9716db640 1 -- 172.21.15.67:0/1350263058 wait complete. 2024-09-21T06:28:04.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.302+0000 7fe9716db640 1 Processor -- start 2024-09-21T06:28:04.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.302+0000 7fe9716db640 1 -- start start 2024-09-21T06:28:04.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.302+0000 7fe9716db640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe96c06c340 0x7fe96c1039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:04.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.302+0000 7fe9716db640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe96c106900 0x7fe96c1022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:04.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.302+0000 7fe9716db640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe96c1094a0 0x7fe96c102800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:04.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.302+0000 7fe9716db640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe96c10efc0 con 0x7fe96c106900 2024-09-21T06:28:04.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.302+0000 7fe9716db640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe96c10ee40 con 0x7fe96c1094a0 2024-09-21T06:28:04.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.302+0000 7fe9716db640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe96c10f140 con 0x7fe96c06c340 2024-09-21T06:28:04.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.302+0000 7fe96a7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe96c106900 0x7fe96c1022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:04.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.302+0000 7fe96a7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe96c106900 0x7fe96c1022c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60342/0 (socket says 172.21.15.67:60342) 2024-09-21T06:28:04.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.302+0000 7fe96affd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe96c06c340 0x7fe96c1039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:04.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.302+0000 7fe96a7fc640 1 -- 172.21.15.67:0/3027235393 learned_addr learned my addr 172.21.15.67:0/3027235393 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:28:04.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.302+0000 7fe96b7fe640 1 --2- 172.21.15.67:0/3027235393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe96c1094a0 0x7fe96c102800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:04.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.303+0000 7fe96affd640 1 -- 172.21.15.67:0/3027235393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe96c1094a0 msgr2=0x7fe96c102800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:04.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.303+0000 7fe96affd640 1 --2- 172.21.15.67:0/3027235393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe96c1094a0 0x7fe96c102800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:04.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.303+0000 7fe96affd640 1 -- 172.21.15.67:0/3027235393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe96c106900 msgr2=0x7fe96c1022c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:04.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.303+0000 7fe96affd640 1 --2- 172.21.15.67:0/3027235393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe96c106900 0x7fe96c1022c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:04.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.303+0000 7fe96affd640 1 -- 172.21.15.67:0/3027235393 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe96c102f00 con 0x7fe96c06c340 2024-09-21T06:28:04.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.303+0000 7fe96a7fc640 1 --2- 172.21.15.67:0/3027235393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe96c106900 0x7fe96c1022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:28:04.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.303+0000 7fe96affd640 1 --2- 172.21.15.67:0/3027235393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe96c06c340 0x7fe96c1039f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00e9f0 tx=0x7fe95c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:04.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.303+0000 7fe953fff640 1 -- 172.21.15.67:0/3027235393 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c00ccc0 con 0x7fe96c06c340 2024-09-21T06:28:04.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.303+0000 7fe953fff640 1 -- 172.21.15.67:0/3027235393 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7fe95c010880 con 0x7fe96c06c340 2024-09-21T06:28:04.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.303+0000 7fe9716db640 1 -- 172.21.15.67:0/3027235393 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe96c1a3da0 con 0x7fe96c06c340 2024-09-21T06:28:04.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.303+0000 7fe953fff640 1 -- 172.21.15.67:0/3027235393 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c018660 con 0x7fe96c06c340 2024-09-21T06:28:04.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.303+0000 7fe9716db640 1 -- 172.21.15.67:0/3027235393 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe96c1a41a0 con 0x7fe96c06c340 2024-09-21T06:28:04.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.304+0000 7fe9716db640 1 -- 172.21.15.67:0/3027235393 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe96c106d50 con 0x7fe96c06c340 2024-09-21T06:28:04.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.304+0000 7fe953fff640 1 -- 172.21.15.67:0/3027235393 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7fe95c018800 con 0x7fe96c06c340 2024-09-21T06:28:04.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.305+0000 7fe953fff640 1 --2- 172.21.15.67:0/3027235393 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe95403fb20 0x7fe954042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:04.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.305+0000 7fe953fff640 1 -- 172.21.15.67:0/3027235393 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe95c054800 con 0x7fe96c06c340 2024-09-21T06:28:04.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.305+0000 7fe96a7fc640 1 --2- 172.21.15.67:0/3027235393 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe95403fb20 0x7fe954042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:04.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.306+0000 7fe96a7fc640 1 --2- 172.21.15.67:0/3027235393 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe95403fb20 0x7fe954042030 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe960005cd0 tx=0x7fe96002bf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:04.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.310+0000 7fe953fff640 1 -- 172.21.15.67:0/3027235393 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe95c020360 con 0x7fe96c06c340 2024-09-21T06:28:04.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.504+0000 7fe9716db640 1 -- 172.21.15.67:0/3027235393 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7fe96c063fc0 con 0x7fe96c06c340 2024-09-21T06:28:04.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.511+0000 7fe953fff640 1 -- 172.21.15.67:0/3027235393 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v14)=0 v14) ==== 120+0+0 (secure 0 0 0) 0x7fe95c010d30 con 0x7fe96c06c340 2024-09-21T06:28:04.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.513+0000 7fe9716db640 1 -- 172.21.15.67:0/3027235393 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe95403fb20 msgr2=0x7fe954042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:04.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.513+0000 7fe9716db640 1 --2- 172.21.15.67:0/3027235393 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe95403fb20 0x7fe954042030 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe960005cd0 tx=0x7fe96002bf90 comp rx=0 tx=0).stop 2024-09-21T06:28:04.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.513+0000 7fe9716db640 1 -- 172.21.15.67:0/3027235393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe96c06c340 msgr2=0x7fe96c1039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:04.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.513+0000 7fe9716db640 1 --2- 172.21.15.67:0/3027235393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe96c06c340 0x7fe96c1039f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00e9f0 tx=0x7fe95c00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:28:04.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.513+0000 7fe9716db640 1 -- 172.21.15.67:0/3027235393 shutdown_connections 2024-09-21T06:28:04.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.513+0000 7fe9716db640 1 --2- 172.21.15.67:0/3027235393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe96c1094a0 0x7fe96c102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:04.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.513+0000 7fe9716db640 1 --2- 172.21.15.67:0/3027235393 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe95403fb20 0x7fe954042030 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:04.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.513+0000 7fe9716db640 1 --2- 172.21.15.67:0/3027235393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe96c106900 0x7fe96c1022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:04.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.513+0000 7fe9716db640 1 --2- 172.21.15.67:0/3027235393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe96c06c340 0x7fe96c1039f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:04.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.513+0000 7fe9716db640 1 -- 172.21.15.67:0/3027235393 >> 172.21.15.67:0/3027235393 conn(0x7fe96c0fdfe0 msgr2=0x7fe96c104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:04.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.513+0000 7fe9716db640 1 -- 172.21.15.67:0/3027235393 shutdown_connections 2024-09-21T06:28:04.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.513+0000 7fe9716db640 1 -- 172.21.15.67:0/3027235393 wait complete. 2024-09-21T06:28:04.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:04 smithi100 ceph-mon[33401]: Deploying daemon mgr.y on smithi100 2024-09-21T06:28:04.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:04 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2476143657' entity='client.admin' 2024-09-21T06:28:04.699 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:04 smithi067 ceph-mon[31232]: Deploying daemon mgr.y on smithi100 2024-09-21T06:28:04.699 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:04 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2476143657' entity='client.admin' 2024-09-21T06:28:04.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.791+0000 7f1672312640 1 -- 172.21.15.67:0/2425348797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f166c102670 msgr2=0x7f166c102ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:04.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.791+0000 7f1672312640 1 --2- 172.21.15.67:0/2425348797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f166c102670 0x7f166c102ae0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f165c009900 tx=0x7f165c02a480 comp rx=0 tx=0).stop 2024-09-21T06:28:04.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.792+0000 7f1672312640 1 -- 172.21.15.67:0/2425348797 shutdown_connections 2024-09-21T06:28:04.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.792+0000 7f1672312640 1 --2- 172.21.15.67:0/2425348797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f166c103700 0x7f166c103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:04.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.792+0000 7f1672312640 1 --2- 172.21.15.67:0/2425348797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f166c1040a0 0x7f166c10aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:04.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.792+0000 7f1672312640 1 --2- 172.21.15.67:0/2425348797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f166c102670 0x7f166c102ae0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:04.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.792+0000 7f1672312640 1 -- 172.21.15.67:0/2425348797 >> 172.21.15.67:0/2425348797 conn(0x7f166c0fdfe0 msgr2=0x7f166c100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:04.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.792+0000 7f1672312640 1 -- 172.21.15.67:0/2425348797 shutdown_connections 2024-09-21T06:28:04.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.792+0000 7f1672312640 1 -- 172.21.15.67:0/2425348797 wait complete. 2024-09-21T06:28:04.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.792+0000 7f1672312640 1 Processor -- start 2024-09-21T06:28:04.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.792+0000 7f1672312640 1 -- start start 2024-09-21T06:28:04.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.793+0000 7f1672312640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f166c102670 0x7f166c06c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:04.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.793+0000 7f1672312640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f166c103700 0x7f166c19ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:04.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.793+0000 7f1672312640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f166c1040a0 0x7f166c1a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:04.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.793+0000 7f1672312640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f166c10dec0 con 0x7f166c102670 2024-09-21T06:28:04.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.793+0000 7f1672312640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f166c10dd40 con 0x7f166c1040a0 2024-09-21T06:28:04.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.793+0000 7f1672312640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f166c10e040 con 0x7f166c103700 2024-09-21T06:28:04.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.793+0000 7f166bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f166c102670 0x7f166c06c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:04.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.793+0000 7f166bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f166c102670 0x7f166c06c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60368/0 (socket says 172.21.15.67:60368) 2024-09-21T06:28:04.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.793+0000 7f166bfff640 1 -- 172.21.15.67:0/268988168 learned_addr learned my addr 172.21.15.67:0/268988168 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:28:04.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.793+0000 7f16708af640 1 --2- 172.21.15.67:0/268988168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f166c1040a0 0x7f166c1a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:04.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.793+0000 7f1663fff640 1 --2- 172.21.15.67:0/268988168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f166c103700 0x7f166c19ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:04.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.793+0000 7f166bfff640 1 -- 172.21.15.67:0/268988168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f166c103700 msgr2=0x7f166c19ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:04.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.793+0000 7f166bfff640 1 --2- 172.21.15.67:0/268988168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f166c103700 0x7f166c19ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:04.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.793+0000 7f166bfff640 1 -- 172.21.15.67:0/268988168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f166c1040a0 msgr2=0x7f166c1a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:04.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.793+0000 7f166bfff640 1 --2- 172.21.15.67:0/268988168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f166c1040a0 0x7f166c1a0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:04.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.793+0000 7f166bfff640 1 -- 172.21.15.67:0/268988168 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166c1a1800 con 0x7f166c102670 2024-09-21T06:28:04.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.793+0000 7f16708af640 1 --2- 172.21.15.67:0/268988168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f166c1040a0 0x7f166c1a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:28:04.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.793+0000 7f166bfff640 1 --2- 172.21.15.67:0/268988168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f166c102670 0x7f166c06c3e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f165c005940 tx=0x7f165c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:04.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.794+0000 7f1669ffb640 1 -- 172.21.15.67:0/268988168 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c03f070 con 0x7f166c102670 2024-09-21T06:28:04.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.794+0000 7f1669ffb640 1 -- 172.21.15.67:0/268988168 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7f165c031d00 con 0x7f166c102670 2024-09-21T06:28:04.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.794+0000 7f1669ffb640 1 -- 172.21.15.67:0/268988168 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c035030 con 0x7f166c102670 2024-09-21T06:28:04.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.794+0000 7f1672312640 1 -- 172.21.15.67:0/268988168 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f166c1a1a90 con 0x7f166c102670 2024-09-21T06:28:04.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.794+0000 7f1672312640 1 -- 172.21.15.67:0/268988168 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f166c1a1fc0 con 0x7f166c102670 2024-09-21T06:28:04.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.795+0000 7f1672312640 1 -- 172.21.15.67:0/268988168 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f166c102ae0 con 0x7f166c102670 2024-09-21T06:28:04.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.795+0000 7f1669ffb640 1 -- 172.21.15.67:0/268988168 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7f165c042070 con 0x7f166c102670 2024-09-21T06:28:04.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.795+0000 7f1669ffb640 1 --2- 172.21.15.67:0/268988168 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f165803fb20 0x7f1658042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:04.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.795+0000 7f1669ffb640 1 -- 172.21.15.67:0/268988168 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f165c075080 con 0x7f166c102670 2024-09-21T06:28:04.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.795+0000 7f1663fff640 1 --2- 172.21.15.67:0/268988168 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f165803fb20 0x7f1658042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:04.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.796+0000 7f1663fff640 1 --2- 172.21.15.67:0/268988168 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f165803fb20 0x7f1658042030 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1654009bd0 tx=0x7f1654005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:04.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.803+0000 7f1669ffb640 1 -- 172.21.15.67:0/268988168 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f165c0303a0 con 0x7f166c102670 2024-09-21T06:28:04.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:04 smithi203 ceph-mon[33735]: Deploying daemon mgr.y on smithi100 2024-09-21T06:28:04.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:04 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2476143657' entity='client.admin' 2024-09-21T06:28:04.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.994+0000 7f1672312640 1 -- 172.21.15.67:0/268988168 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f166c063fc0 con 0x7f166c102670 2024-09-21T06:28:05.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.011+0000 7f1669ffb640 1 -- 172.21.15.67:0/268988168 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v15)=0 v15) ==== 115+0+0 (secure 0 0 0) 0x7f165c040260 con 0x7f166c102670 2024-09-21T06:28:05.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.014+0000 7f1672312640 1 -- 172.21.15.67:0/268988168 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f165803fb20 msgr2=0x7f1658042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:05.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.014+0000 7f1672312640 1 --2- 172.21.15.67:0/268988168 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f165803fb20 0x7f1658042030 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1654009bd0 tx=0x7f1654005f90 comp rx=0 tx=0).stop 2024-09-21T06:28:05.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.014+0000 7f1672312640 1 -- 172.21.15.67:0/268988168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f166c102670 msgr2=0x7f166c06c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:05.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.014+0000 7f1672312640 1 --2- 172.21.15.67:0/268988168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f166c102670 0x7f166c06c3e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f165c005940 tx=0x7f165c0042f0 comp rx=0 tx=0).stop 2024-09-21T06:28:05.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.014+0000 7f1672312640 1 -- 172.21.15.67:0/268988168 shutdown_connections 2024-09-21T06:28:05.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.014+0000 7f1672312640 1 --2- 172.21.15.67:0/268988168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f166c1040a0 0x7f166c1a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:05.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.014+0000 7f1672312640 1 --2- 172.21.15.67:0/268988168 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f165803fb20 0x7f1658042030 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:05.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.014+0000 7f1672312640 1 --2- 172.21.15.67:0/268988168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f166c103700 0x7f166c19ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:05.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.014+0000 7f1672312640 1 --2- 172.21.15.67:0/268988168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f166c102670 0x7f166c06c3e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:05.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.014+0000 7f1672312640 1 -- 172.21.15.67:0/268988168 >> 172.21.15.67:0/268988168 conn(0x7f166c0fdfe0 msgr2=0x7f166c106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:05.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.014+0000 7f1672312640 1 -- 172.21.15.67:0/268988168 shutdown_connections 2024-09-21T06:28:05.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.014+0000 7f1672312640 1 -- 172.21.15.67:0/268988168 wait complete. 2024-09-21T06:28:05.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.196+0000 7fd24968e640 1 -- 172.21.15.67:0/3959851988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd244104800 msgr2=0x7fd244104c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:05.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.196+0000 7fd24968e640 1 --2- 172.21.15.67:0/3959851988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd244104800 0x7fd244104c10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd22c009900 tx=0x7fd22c02a480 comp rx=0 tx=0).stop 2024-09-21T06:28:05.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.196+0000 7fd24968e640 1 -- 172.21.15.67:0/3959851988 shutdown_connections 2024-09-21T06:28:05.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.196+0000 7fd24968e640 1 --2- 172.21.15.67:0/3959851988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2441051a0 0x7fd24410bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:05.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.196+0000 7fd24968e640 1 --2- 172.21.15.67:0/3959851988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd244104800 0x7fd244104c10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:05.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.197+0000 7fd24968e640 1 --2- 172.21.15.67:0/3959851988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd244103770 0x7fd244103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:05.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.197+0000 7fd24968e640 1 -- 172.21.15.67:0/3959851988 >> 172.21.15.67:0/3959851988 conn(0x7fd2440ff0c0 msgr2=0x7fd244101590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:05.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.197+0000 7fd24968e640 1 -- 172.21.15.67:0/3959851988 shutdown_connections 2024-09-21T06:28:05.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.197+0000 7fd24968e640 1 -- 172.21.15.67:0/3959851988 wait complete. 2024-09-21T06:28:05.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.197+0000 7fd24968e640 1 Processor -- start 2024-09-21T06:28:05.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.197+0000 7fd24968e640 1 -- start start 2024-09-21T06:28:05.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.197+0000 7fd24968e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd244103770 0x7fd244199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.197+0000 7fd24968e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd244104800 0x7fd24419a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.197+0000 7fd24968e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2441051a0 0x7fd24419af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.198+0000 7fd24968e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd24410eff0 con 0x7fd2441051a0 2024-09-21T06:28:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.198+0000 7fd24968e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd24410ee70 con 0x7fd244104800 2024-09-21T06:28:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.198+0000 7fd24968e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd24410f170 con 0x7fd244103770 2024-09-21T06:28:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.198+0000 7fd2437fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2441051a0 0x7fd24419af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.198+0000 7fd242ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd244103770 0x7fd244199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.198+0000 7fd2437fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2441051a0 0x7fd24419af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60394/0 (socket says 172.21.15.67:60394) 2024-09-21T06:28:05.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.198+0000 7fd2437fe640 1 -- 172.21.15.67:0/1414994882 learned_addr learned my addr 172.21.15.67:0/1414994882 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:28:05.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.198+0000 7fd23bfff640 1 --2- 172.21.15.67:0/1414994882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd244104800 0x7fd24419a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:05.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.198+0000 7fd2437fe640 1 -- 172.21.15.67:0/1414994882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd244103770 msgr2=0x7fd244199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:05.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.198+0000 7fd2437fe640 1 --2- 172.21.15.67:0/1414994882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd244103770 0x7fd244199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:05.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.198+0000 7fd2437fe640 1 -- 172.21.15.67:0/1414994882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd244104800 msgr2=0x7fd24419a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:05.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.198+0000 7fd2437fe640 1 --2- 172.21.15.67:0/1414994882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd244104800 0x7fd24419a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:05.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.198+0000 7fd2437fe640 1 -- 172.21.15.67:0/1414994882 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd24419b760 con 0x7fd2441051a0 2024-09-21T06:28:05.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.198+0000 7fd2437fe640 1 --2- 172.21.15.67:0/1414994882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2441051a0 0x7fd24419af40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00eb10 tx=0x7fd23c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:05.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.198+0000 7fd240ff9640 1 -- 172.21.15.67:0/1414994882 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c004b20 con 0x7fd2441051a0 2024-09-21T06:28:05.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.198+0000 7fd240ff9640 1 -- 172.21.15.67:0/1414994882 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7fd23c004cc0 con 0x7fd2441051a0 2024-09-21T06:28:05.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.198+0000 7fd24968e640 1 -- 172.21.15.67:0/1414994882 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd24419ba50 con 0x7fd2441051a0 2024-09-21T06:28:05.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.198+0000 7fd240ff9640 1 -- 172.21.15.67:0/1414994882 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c010430 con 0x7fd2441051a0 2024-09-21T06:28:05.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.198+0000 7fd24968e640 1 -- 172.21.15.67:0/1414994882 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2441a2150 con 0x7fd2441051a0 2024-09-21T06:28:05.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.200+0000 7fd24968e640 1 -- 172.21.15.67:0/1414994882 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd244103be0 con 0x7fd2441051a0 2024-09-21T06:28:05.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.200+0000 7fd240ff9640 1 -- 172.21.15.67:0/1414994882 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7fd23c0105d0 con 0x7fd2441051a0 2024-09-21T06:28:05.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.200+0000 7fd240ff9640 1 --2- 172.21.15.67:0/1414994882 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd23003fb20 0x7fd230042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:05.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.200+0000 7fd240ff9640 1 -- 172.21.15.67:0/1414994882 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd23c054a00 con 0x7fd2441051a0 2024-09-21T06:28:05.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.200+0000 7fd242ffd640 1 --2- 172.21.15.67:0/1414994882 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd23003fb20 0x7fd230042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:05.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.201+0000 7fd242ffd640 1 --2- 172.21.15.67:0/1414994882 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd23003fb20 0x7fd230042030 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd234009bd0 tx=0x7fd234005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:05.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.205+0000 7fd240ff9640 1 -- 172.21.15.67:0/1414994882 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd23c0108c0 con 0x7fd2441051a0 2024-09-21T06:28:05.303 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 21 06:28:04 smithi100 systemd[1]: Started Ceph mgr.y for 2a791768-77e2-11ef-baf5-efdc52797490. 2024-09-21T06:28:05.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.394+0000 7fd24968e640 1 -- 172.21.15.67:0/1414994882 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7fd24402e940 con 0x7fd2441051a0 2024-09-21T06:28:05.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.399+0000 7fd240ff9640 1 -- 172.21.15.67:0/1414994882 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v16)=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7fd23c019d50 con 0x7fd2441051a0 2024-09-21T06:28:05.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.401+0000 7fd24968e640 1 -- 172.21.15.67:0/1414994882 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd23003fb20 msgr2=0x7fd230042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:05.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.401+0000 7fd24968e640 1 --2- 172.21.15.67:0/1414994882 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd23003fb20 0x7fd230042030 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd234009bd0 tx=0x7fd234005f90 comp rx=0 tx=0).stop 2024-09-21T06:28:05.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.401+0000 7fd24968e640 1 -- 172.21.15.67:0/1414994882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2441051a0 msgr2=0x7fd24419af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:05.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.401+0000 7fd24968e640 1 --2- 172.21.15.67:0/1414994882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2441051a0 0x7fd24419af40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00eb10 tx=0x7fd23c00c2d0 comp rx=0 tx=0).stop 2024-09-21T06:28:05.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.402+0000 7fd24968e640 1 -- 172.21.15.67:0/1414994882 shutdown_connections 2024-09-21T06:28:05.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.402+0000 7fd24968e640 1 --2- 172.21.15.67:0/1414994882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd244104800 0x7fd24419a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:05.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.402+0000 7fd24968e640 1 --2- 172.21.15.67:0/1414994882 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd23003fb20 0x7fd230042030 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:05.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.402+0000 7fd24968e640 1 --2- 172.21.15.67:0/1414994882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2441051a0 0x7fd24419af40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:05.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.402+0000 7fd24968e640 1 --2- 172.21.15.67:0/1414994882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd244103770 0x7fd244199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:05.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.402+0000 7fd24968e640 1 -- 172.21.15.67:0/1414994882 >> 172.21.15.67:0/1414994882 conn(0x7fd2440ff0c0 msgr2=0x7fd244107b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:05.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.402+0000 7fd24968e640 1 -- 172.21.15.67:0/1414994882 shutdown_connections 2024-09-21T06:28:05.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:05.402+0000 7fd24968e640 1 -- 172.21.15.67:0/1414994882 wait complete. 2024-09-21T06:28:05.697 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:05 smithi067 ceph-mon[31232]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:05.697 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:05 smithi067 ceph-mon[31232]: from='client.? ' entity='client.admin' 2024-09-21T06:28:05.697 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:05 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/268988168' entity='client.admin' 2024-09-21T06:28:05.697 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:05 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:05.697 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:05 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:05.697 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:05 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:05.697 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:05 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:05.697 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:05 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:28:05.697 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:05 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1414994882' entity='client.admin' 2024-09-21T06:28:05.801 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:05 smithi100 ceph-mon[33401]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:05.802 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:05 smithi100 ceph-mon[33401]: from='client.? ' entity='client.admin' 2024-09-21T06:28:05.802 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:05 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/268988168' entity='client.admin' 2024-09-21T06:28:05.802 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:05 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:05.802 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:05 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:05.802 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:05 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:05.802 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:05 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:05.802 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:05 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:28:05.802 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:05 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1414994882' entity='client.admin' 2024-09-21T06:28:05.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:05 smithi203 ceph-mon[33735]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:05.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:05 smithi203 ceph-mon[33735]: from='client.? ' entity='client.admin' 2024-09-21T06:28:05.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:05 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/268988168' entity='client.admin' 2024-09-21T06:28:05.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:05 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:05.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:05 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:05.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:05 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:05.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:05 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:05.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:05 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:28:05.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:05 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1414994882' entity='client.admin' 2024-09-21T06:28:06.345 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:28:07.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.103+0000 7fd07871d640 1 -- 172.21.15.67:0/2194858313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd070109470 msgr2=0x7fd07010b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:07.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.103+0000 7fd07871d640 1 --2- 172.21.15.67:0/2194858313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd070109470 0x7fd07010b980 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd068009ca0 tx=0x7fd06802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:28:07.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.103+0000 7fd07871d640 1 -- 172.21.15.67:0/2194858313 shutdown_connections 2024-09-21T06:28:07.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.103+0000 7fd07871d640 1 --2- 172.21.15.67:0/2194858313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd070109470 0x7fd07010b980 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:07.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.103+0000 7fd07871d640 1 --2- 172.21.15.67:0/2194858313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0701068d0 0x7fd070108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:07.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.103+0000 7fd07871d640 1 --2- 172.21.15.67:0/2194858313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd07006b980 0x7fd070106390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:07.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.103+0000 7fd07871d640 1 -- 172.21.15.67:0/2194858313 >> 172.21.15.67:0/2194858313 conn(0x7fd0700fdfb0 msgr2=0x7fd070100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:07.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.103+0000 7fd07871d640 1 -- 172.21.15.67:0/2194858313 shutdown_connections 2024-09-21T06:28:07.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.103+0000 7fd07871d640 1 -- 172.21.15.67:0/2194858313 wait complete. 2024-09-21T06:28:07.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.104+0000 7fd07871d640 1 Processor -- start 2024-09-21T06:28:07.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.104+0000 7fd07871d640 1 -- start start 2024-09-21T06:28:07.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.104+0000 7fd07871d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd07006b980 0x7fd070199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:07.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.104+0000 7fd07871d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0701068d0 0x7fd07019a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:07.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.104+0000 7fd07871d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd070109470 0x7fd07019af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:07.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.104+0000 7fd07871d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd07010efc0 con 0x7fd0701068d0 2024-09-21T06:28:07.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.104+0000 7fd07871d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd07010ee40 con 0x7fd07006b980 2024-09-21T06:28:07.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.104+0000 7fd07871d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd07010f140 con 0x7fd070109470 2024-09-21T06:28:07.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.104+0000 7fd075cb8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0701068d0 0x7fd07019a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:07.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.104+0000 7fd0764b9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd07006b980 0x7fd070199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:07.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.104+0000 7fd0764b9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd07006b980 0x7fd070199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.67:33252/0 (socket says 172.21.15.67:33252) 2024-09-21T06:28:07.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.104+0000 7fd0764b9640 1 -- 172.21.15.67:0/977434797 learned_addr learned my addr 172.21.15.67:0/977434797 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:28:07.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.104+0000 7fd075cb8640 1 --2- 172.21.15.67:0/977434797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0701068d0 0x7fd07019a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60412/0 (socket says 172.21.15.67:60412) 2024-09-21T06:28:07.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.104+0000 7fd076cba640 1 --2- 172.21.15.67:0/977434797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd070109470 0x7fd07019af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:07.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.104+0000 7fd0764b9640 1 -- 172.21.15.67:0/977434797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd070109470 msgr2=0x7fd07019af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:07.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.104+0000 7fd0764b9640 1 --2- 172.21.15.67:0/977434797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd070109470 0x7fd07019af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:07.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.104+0000 7fd0764b9640 1 -- 172.21.15.67:0/977434797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0701068d0 msgr2=0x7fd07019a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:07.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.104+0000 7fd0764b9640 1 --2- 172.21.15.67:0/977434797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0701068d0 0x7fd07019a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:07.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.104+0000 7fd0764b9640 1 -- 172.21.15.67:0/977434797 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd07019b730 con 0x7fd07006b980 2024-09-21T06:28:07.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.104+0000 7fd075cb8640 1 --2- 172.21.15.67:0/977434797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0701068d0 0x7fd07019a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:28:07.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.105+0000 7fd0764b9640 1 --2- 172.21.15.67:0/977434797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd07006b980 0x7fd070199d00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00e9f0 tx=0x7fd06c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:07.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.105+0000 7fd0677fe640 1 -- 172.21.15.67:0/977434797 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c00cd80 con 0x7fd07006b980 2024-09-21T06:28:07.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.105+0000 7fd0677fe640 1 -- 172.21.15.67:0/977434797 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7fd06c010880 con 0x7fd07006b980 2024-09-21T06:28:07.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.105+0000 7fd0677fe640 1 -- 172.21.15.67:0/977434797 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c018640 con 0x7fd07006b980 2024-09-21T06:28:07.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.105+0000 7fd07871d640 1 -- 172.21.15.67:0/977434797 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd07019ba20 con 0x7fd07006b980 2024-09-21T06:28:07.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.105+0000 7fd07871d640 1 -- 172.21.15.67:0/977434797 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0701a2120 con 0x7fd07006b980 2024-09-21T06:28:07.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.106+0000 7fd07871d640 1 -- 172.21.15.67:0/977434797 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd07010f780 con 0x7fd07006b980 2024-09-21T06:28:07.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.106+0000 7fd0677fe640 1 -- 172.21.15.67:0/977434797 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7fd06c010430 con 0x7fd07006b980 2024-09-21T06:28:07.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.106+0000 7fd0677fe640 1 --2- 172.21.15.67:0/977434797 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd05c03fb20 0x7fd05c042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:07.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.107+0000 7fd0677fe640 1 -- 172.21.15.67:0/977434797 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd06c054930 con 0x7fd07006b980 2024-09-21T06:28:07.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.107+0000 7fd075cb8640 1 --2- 172.21.15.67:0/977434797 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd05c03fb20 0x7fd05c042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:07.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.107+0000 7fd075cb8640 1 --2- 172.21.15.67:0/977434797 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd05c03fb20 0x7fd05c042030 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd060009f40 tx=0x7fd060005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:07.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.116+0000 7fd0677fe640 1 -- 172.21.15.67:0/977434797 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd06c00b2c0 con 0x7fd07006b980 2024-09-21T06:28:07.184 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:07 smithi067 ceph-mon[31232]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:07.184 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:07 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:07.184 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:07 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:07.184 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:07 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:07.185 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:07 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:07.185 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:07 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:07.185 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:07 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:28:07.185 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:07 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:07.185 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:07 smithi067 ceph-mon[31232]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T06:28:07.185 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:07 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:28:07.185 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:07 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:28:07.185 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:07 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:07.185 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:07 smithi067 ceph-mon[31232]: Reconfiguring daemon mgr.x on smithi067 2024-09-21T06:28:07.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.313+0000 7fd07871d640 1 -- 172.21.15.67:0/977434797 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7fd07002e810 con 0x7fd07006b980 2024-09-21T06:28:07.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.319+0000 7fd0677fe640 1 -- 172.21.15.67:0/977434797 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v17)=0 v17) ==== 127+0+0 (secure 0 0 0) 0x7fd06c018820 con 0x7fd07006b980 2024-09-21T06:28:07.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.321+0000 7fd07871d640 1 -- 172.21.15.67:0/977434797 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd05c03fb20 msgr2=0x7fd05c042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:07.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.321+0000 7fd07871d640 1 --2- 172.21.15.67:0/977434797 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd05c03fb20 0x7fd05c042030 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd060009f40 tx=0x7fd060005f90 comp rx=0 tx=0).stop 2024-09-21T06:28:07.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.321+0000 7fd07871d640 1 -- 172.21.15.67:0/977434797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd07006b980 msgr2=0x7fd070199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:07.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.321+0000 7fd07871d640 1 --2- 172.21.15.67:0/977434797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd07006b980 0x7fd070199d00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00e9f0 tx=0x7fd06c00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:28:07.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.321+0000 7fd07871d640 1 -- 172.21.15.67:0/977434797 shutdown_connections 2024-09-21T06:28:07.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.321+0000 7fd07871d640 1 --2- 172.21.15.67:0/977434797 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd05c03fb20 0x7fd05c042030 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:07.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.321+0000 7fd07871d640 1 --2- 172.21.15.67:0/977434797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd070109470 0x7fd07019af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:07.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.321+0000 7fd07871d640 1 --2- 172.21.15.67:0/977434797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0701068d0 0x7fd07019a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:07.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.321+0000 7fd07871d640 1 --2- 172.21.15.67:0/977434797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd07006b980 0x7fd070199d00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:07.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.321+0000 7fd07871d640 1 -- 172.21.15.67:0/977434797 >> 172.21.15.67:0/977434797 conn(0x7fd0700fdfb0 msgr2=0x7fd070104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:07.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.321+0000 7fd07871d640 1 -- 172.21.15.67:0/977434797 shutdown_connections 2024-09-21T06:28:07.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.321+0000 7fd07871d640 1 -- 172.21.15.67:0/977434797 wait complete. 2024-09-21T06:28:07.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:07 smithi203 ceph-mon[33735]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:07.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:07 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:07.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:07 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:07.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:07 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:07.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:07 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:07.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:07 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:07.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:07 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:28:07.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:07 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:07.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:07 smithi203 ceph-mon[33735]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T06:28:07.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:07 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:28:07.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:07 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:28:07.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:07 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:07.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:07 smithi203 ceph-mon[33735]: Reconfiguring daemon mgr.x on smithi067 2024-09-21T06:28:07.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:07 smithi100 ceph-mon[33401]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:07.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:07 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:07.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:07 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:07.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:07 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:07.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:07 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:07.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:07 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:07.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:07 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:28:07.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:07 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:07.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:07 smithi100 ceph-mon[33401]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T06:28:07.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:07 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:28:07.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:07 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:28:07.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:07 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:07.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:07 smithi100 ceph-mon[33401]: Reconfiguring daemon mgr.x on smithi067 2024-09-21T06:28:08.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.218+0000 7f916c7a2640 1 -- 172.21.15.67:0/964628338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9164106be0 msgr2=0x7f9164109000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:08.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.218+0000 7f916c7a2640 1 --2- 172.21.15.67:0/964628338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9164106be0 0x7f9164109000 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9154009900 tx=0x7f915402a410 comp rx=0 tx=0).stop 2024-09-21T06:28:08.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.218+0000 7f916c7a2640 1 -- 172.21.15.67:0/964628338 shutdown_connections 2024-09-21T06:28:08.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.218+0000 7f916c7a2640 1 --2- 172.21.15.67:0/964628338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9164109540 0x7f916410b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.218+0000 7f916c7a2640 1 --2- 172.21.15.67:0/964628338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9164106be0 0x7f9164109000 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.218+0000 7f916c7a2640 1 --2- 172.21.15.67:0/964628338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9164101b40 0x7f9164101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.218+0000 7f916c7a2640 1 -- 172.21.15.67:0/964628338 >> 172.21.15.67:0/964628338 conn(0x7f91640fd770 msgr2=0x7f91640ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:08.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.219+0000 7f916c7a2640 1 -- 172.21.15.67:0/964628338 shutdown_connections 2024-09-21T06:28:08.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.219+0000 7f916c7a2640 1 -- 172.21.15.67:0/964628338 wait complete. 2024-09-21T06:28:08.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.219+0000 7f916c7a2640 1 Processor -- start 2024-09-21T06:28:08.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.219+0000 7f916c7a2640 1 -- start start 2024-09-21T06:28:08.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.220+0000 7f916c7a2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9164101b40 0x7f9164199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:08.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.220+0000 7f916c7a2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9164106be0 0x7f916419a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:08.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.220+0000 7f916c7a2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9164109540 0x7f916419af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:08.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.220+0000 7f916c7a2640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f916410f000 con 0x7f9164106be0 2024-09-21T06:28:08.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.220+0000 7f916c7a2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f916410ee80 con 0x7f9164109540 2024-09-21T06:28:08.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.220+0000 7f916c7a2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f916410f180 con 0x7f9164101b40 2024-09-21T06:28:08.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.220+0000 7f9169d3d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9164106be0 0x7f916419a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:08.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.220+0000 7f916ad3f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9164109540 0x7f916419af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:08.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.220+0000 7f9169d3d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9164106be0 0x7f916419a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57110/0 (socket says 172.21.15.67:57110) 2024-09-21T06:28:08.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.220+0000 7f916a53e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9164101b40 0x7f9164199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:08.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.220+0000 7f9169d3d640 1 -- 172.21.15.67:0/2909674577 learned_addr learned my addr 172.21.15.67:0/2909674577 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:28:08.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.220+0000 7f916ad3f640 1 -- 172.21.15.67:0/2909674577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9164101b40 msgr2=0x7f9164199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:08.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.220+0000 7f916ad3f640 1 --2- 172.21.15.67:0/2909674577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9164101b40 0x7f9164199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.220+0000 7f916ad3f640 1 -- 172.21.15.67:0/2909674577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9164106be0 msgr2=0x7f916419a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:08.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.220+0000 7f916ad3f640 1 --2- 172.21.15.67:0/2909674577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9164106be0 0x7f916419a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.220+0000 7f916ad3f640 1 -- 172.21.15.67:0/2909674577 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f916419b770 con 0x7f9164109540 2024-09-21T06:28:08.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.220+0000 7f9169d3d640 1 --2- 172.21.15.67:0/2909674577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9164106be0 0x7f916419a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:28:08.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.220+0000 7f916ad3f640 1 --2- 172.21.15.67:0/2909674577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9164109540 0x7f916419af50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f915c009fd0 tx=0x7f915c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:08.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.221+0000 7f915b7fe640 1 -- 172.21.15.67:0/2909674577 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c00ce20 con 0x7f9164109540 2024-09-21T06:28:08.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.221+0000 7f915b7fe640 1 -- 172.21.15.67:0/2909674577 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7f915c005020 con 0x7f9164109540 2024-09-21T06:28:08.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.221+0000 7f916c7a2640 1 -- 172.21.15.67:0/2909674577 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f916419ba60 con 0x7f9164109540 2024-09-21T06:28:08.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.221+0000 7f915b7fe640 1 -- 172.21.15.67:0/2909674577 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c010690 con 0x7f9164109540 2024-09-21T06:28:08.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.221+0000 7f916c7a2640 1 -- 172.21.15.67:0/2909674577 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91641a2160 con 0x7f9164109540 2024-09-21T06:28:08.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.222+0000 7f915b7fe640 1 -- 172.21.15.67:0/2909674577 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7f915c010830 con 0x7f9164109540 2024-09-21T06:28:08.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.222+0000 7f916c7a2640 1 -- 172.21.15.67:0/2909674577 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9164101f50 con 0x7f9164109540 2024-09-21T06:28:08.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.222+0000 7f915b7fe640 1 --2- 172.21.15.67:0/2909674577 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f915003fb20 0x7f9150042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:08.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.222+0000 7f915b7fe640 1 -- 172.21.15.67:0/2909674577 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f915c054cc0 con 0x7f9164109540 2024-09-21T06:28:08.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.222+0000 7f916a53e640 1 --2- 172.21.15.67:0/2909674577 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f915003fb20 0x7f9150042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:08.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.223+0000 7f916a53e640 1 --2- 172.21.15.67:0/2909674577 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f915003fb20 0x7f9150042030 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9160009bd0 tx=0x7f9160005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:08.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.227+0000 7f915b7fe640 1 -- 172.21.15.67:0/2909674577 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f915c01fe40 con 0x7f9164109540 2024-09-21T06:28:08.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.325+0000 7f915b7fe640 1 -- 172.21.15.67:0/2909674577 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 14) ==== 103177+0+0 (secure 0 0 0) 0x7f915c019a00 con 0x7f9164109540 2024-09-21T06:28:08.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.420+0000 7f916c7a2640 1 -- 172.21.15.67:0/2909674577 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f91640ff290 con 0x7f9164109540 2024-09-21T06:28:08.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.426+0000 7f915b7fe640 1 -- 172.21.15.67:0/2909674577 <== mon.2 v2:172.21.15.100:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v19)=0 v19) ==== 125+0+0 (secure 0 0 0) 0x7f91640ff290 con 0x7f9164109540 2024-09-21T06:28:08.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.429+0000 7f916c7a2640 1 -- 172.21.15.67:0/2909674577 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f915003fb20 msgr2=0x7f9150042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:08.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.429+0000 7f916c7a2640 1 --2- 172.21.15.67:0/2909674577 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f915003fb20 0x7f9150042030 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9160009bd0 tx=0x7f9160005f90 comp rx=0 tx=0).stop 2024-09-21T06:28:08.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.429+0000 7f916c7a2640 1 -- 172.21.15.67:0/2909674577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9164109540 msgr2=0x7f916419af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:08.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.429+0000 7f916c7a2640 1 --2- 172.21.15.67:0/2909674577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9164109540 0x7f916419af50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f915c009fd0 tx=0x7f915c00ef10 comp rx=0 tx=0).stop 2024-09-21T06:28:08.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.429+0000 7f916c7a2640 1 -- 172.21.15.67:0/2909674577 shutdown_connections 2024-09-21T06:28:08.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.429+0000 7f916c7a2640 1 --2- 172.21.15.67:0/2909674577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9164109540 0x7f916419af50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.429+0000 7f916c7a2640 1 --2- 172.21.15.67:0/2909674577 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f915003fb20 0x7f9150042030 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.429+0000 7f916c7a2640 1 --2- 172.21.15.67:0/2909674577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9164106be0 0x7f916419a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.429+0000 7f916c7a2640 1 --2- 172.21.15.67:0/2909674577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9164101b40 0x7f9164199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.429+0000 7f916c7a2640 1 -- 172.21.15.67:0/2909674577 >> 172.21.15.67:0/2909674577 conn(0x7f91640fd770 msgr2=0x7f9164107bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:08.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.429+0000 7f916c7a2640 1 -- 172.21.15.67:0/2909674577 shutdown_connections 2024-09-21T06:28:08.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.429+0000 7f916c7a2640 1 -- 172.21.15.67:0/2909674577 wait complete. 2024-09-21T06:28:08.573 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:08 smithi067 ceph-mon[31232]: from='client.? ' entity='client.admin' 2024-09-21T06:28:08.573 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:08 smithi067 ceph-mon[31232]: Standby manager daemon z started 2024-09-21T06:28:08.573 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:08 smithi067 ceph-mon[31232]: from='mgr.? 172.21.15.203:0/110627805' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T06:28:08.573 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:08 smithi067 ceph-mon[31232]: from='mgr.? 172.21.15.203:0/110627805' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T06:28:08.573 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:08 smithi067 ceph-mon[31232]: from='mgr.? 172.21.15.203:0/110627805' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T06:28:08.573 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:08 smithi067 ceph-mon[31232]: from='mgr.? 172.21.15.203:0/110627805' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T06:28:08.573 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:08 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/4173866898' entity='client.admin' 2024-09-21T06:28:08.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:08 smithi203 ceph-mon[33735]: from='client.? ' entity='client.admin' 2024-09-21T06:28:08.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:08 smithi203 ceph-mon[33735]: Standby manager daemon z started 2024-09-21T06:28:08.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:08 smithi203 ceph-mon[33735]: from='mgr.? 172.21.15.203:0/110627805' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T06:28:08.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:08 smithi203 ceph-mon[33735]: from='mgr.? 172.21.15.203:0/110627805' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T06:28:08.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:08 smithi203 ceph-mon[33735]: from='mgr.? 172.21.15.203:0/110627805' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T06:28:08.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:08 smithi203 ceph-mon[33735]: from='mgr.? 172.21.15.203:0/110627805' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T06:28:08.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:08 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/4173866898' entity='client.admin' 2024-09-21T06:28:08.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:08 smithi100 ceph-mon[33401]: from='client.? ' entity='client.admin' 2024-09-21T06:28:08.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:08 smithi100 ceph-mon[33401]: Standby manager daemon z started 2024-09-21T06:28:08.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:08 smithi100 ceph-mon[33401]: from='mgr.? 172.21.15.203:0/110627805' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T06:28:08.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:08 smithi100 ceph-mon[33401]: from='mgr.? 172.21.15.203:0/110627805' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T06:28:08.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:08 smithi100 ceph-mon[33401]: from='mgr.? 172.21.15.203:0/110627805' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T06:28:08.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:08 smithi100 ceph-mon[33401]: from='mgr.? 172.21.15.203:0/110627805' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T06:28:08.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:08 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/4173866898' entity='client.admin' 2024-09-21T06:28:08.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.549+0000 7f63d344d640 1 -- 172.21.15.67:0/3225736746 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63cc103700 msgr2=0x7f63cc103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:08.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.549+0000 7f63d344d640 1 --2- 172.21.15.67:0/3225736746 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63cc103700 0x7f63cc103b10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f63bc009900 tx=0x7f63bc02a480 comp rx=0 tx=0).stop 2024-09-21T06:28:08.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.549+0000 7f63d344d640 1 -- 172.21.15.67:0/3225736746 shutdown_connections 2024-09-21T06:28:08.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.549+0000 7f63d344d640 1 --2- 172.21.15.67:0/3225736746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63cc1040a0 0x7f63cc10aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.549+0000 7f63d344d640 1 --2- 172.21.15.67:0/3225736746 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63cc103700 0x7f63cc103b10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.549+0000 7f63d344d640 1 --2- 172.21.15.67:0/3225736746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63cc102670 0x7f63cc102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.549+0000 7f63d344d640 1 -- 172.21.15.67:0/3225736746 >> 172.21.15.67:0/3225736746 conn(0x7f63cc0fdfe0 msgr2=0x7f63cc100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:08.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.549+0000 7f63d344d640 1 -- 172.21.15.67:0/3225736746 shutdown_connections 2024-09-21T06:28:08.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.549+0000 7f63d344d640 1 -- 172.21.15.67:0/3225736746 wait complete. 2024-09-21T06:28:08.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.550+0000 7f63d344d640 1 Processor -- start 2024-09-21T06:28:08.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.550+0000 7f63d344d640 1 -- start start 2024-09-21T06:28:08.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.550+0000 7f63d344d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63cc102670 0x7f63cc06c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:08.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.550+0000 7f63d344d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63cc103700 0x7f63cc19eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:08.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.550+0000 7f63d344d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63cc1040a0 0x7f63cc1a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:08.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.550+0000 7f63d344d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f63cc10def0 con 0x7f63cc1040a0 2024-09-21T06:28:08.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.550+0000 7f63d344d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f63cc10dd70 con 0x7f63cc102670 2024-09-21T06:28:08.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.550+0000 7f63d344d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f63cc10e070 con 0x7f63cc103700 2024-09-21T06:28:08.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.550+0000 7f63d19ea640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63cc1040a0 0x7f63cc1a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:08.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.550+0000 7f63d19ea640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63cc1040a0 0x7f63cc1a1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57098/0 (socket says 172.21.15.67:57098) 2024-09-21T06:28:08.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.550+0000 7f63d19ea640 1 -- 172.21.15.67:0/4173866898 learned_addr learned my addr 172.21.15.67:0/4173866898 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:28:08.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.550+0000 7f63d11e9640 1 --2- 172.21.15.67:0/4173866898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63cc102670 0x7f63cc06c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:08.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.550+0000 7f63d09e8640 1 --2- 172.21.15.67:0/4173866898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63cc103700 0x7f63cc19eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:08.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.550+0000 7f63d19ea640 1 -- 172.21.15.67:0/4173866898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63cc103700 msgr2=0x7f63cc19eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:08.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.550+0000 7f63d19ea640 1 --2- 172.21.15.67:0/4173866898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63cc103700 0x7f63cc19eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.550+0000 7f63d19ea640 1 -- 172.21.15.67:0/4173866898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63cc102670 msgr2=0x7f63cc06c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:08.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.550+0000 7f63d19ea640 1 --2- 172.21.15.67:0/4173866898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63cc102670 0x7f63cc06c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.550+0000 7f63d19ea640 1 -- 172.21.15.67:0/4173866898 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63cc1a1860 con 0x7f63cc1040a0 2024-09-21T06:28:08.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.550+0000 7f63d11e9640 1 --2- 172.21.15.67:0/4173866898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63cc102670 0x7f63cc06c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:28:08.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.551+0000 7f63d19ea640 1 --2- 172.21.15.67:0/4173866898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63cc1040a0 0x7f63cc1a1060 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f63c400efd0 tx=0x7f63c400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:08.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.551+0000 7f63c27fc640 1 -- 172.21.15.67:0/4173866898 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c4003b70 con 0x7f63cc1040a0 2024-09-21T06:28:08.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.551+0000 7f63d344d640 1 -- 172.21.15.67:0/4173866898 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63cc1a1b50 con 0x7f63cc1040a0 2024-09-21T06:28:08.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.551+0000 7f63c27fc640 1 -- 172.21.15.67:0/4173866898 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7f63c4003d10 con 0x7f63cc1040a0 2024-09-21T06:28:08.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.551+0000 7f63c27fc640 1 -- 172.21.15.67:0/4173866898 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c4010730 con 0x7f63cc1040a0 2024-09-21T06:28:08.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.551+0000 7f63d344d640 1 -- 172.21.15.67:0/4173866898 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63cc1a2040 con 0x7f63cc1040a0 2024-09-21T06:28:08.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.552+0000 7f63c27fc640 1 -- 172.21.15.67:0/4173866898 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7f63c40108d0 con 0x7f63cc1040a0 2024-09-21T06:28:08.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.552+0000 7f63d344d640 1 -- 172.21.15.67:0/4173866898 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63cc102ae0 con 0x7f63cc1040a0 2024-09-21T06:28:08.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.552+0000 7f63c27fc640 1 --2- 172.21.15.67:0/4173866898 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f63b803fb20 0x7f63b8042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:08.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.552+0000 7f63c27fc640 1 -- 172.21.15.67:0/4173866898 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f63c40549c0 con 0x7f63cc1040a0 2024-09-21T06:28:08.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.552+0000 7f63d11e9640 1 --2- 172.21.15.67:0/4173866898 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f63b803fb20 0x7f63b8042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:08.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.553+0000 7f63d11e9640 1 --2- 172.21.15.67:0/4173866898 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f63b803fb20 0x7f63b8042030 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f63c8009fc0 tx=0x7f63c8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:08.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.557+0000 7f63c27fc640 1 -- 172.21.15.67:0/4173866898 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f63c4056e20 con 0x7f63cc1040a0 2024-09-21T06:28:08.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.748+0000 7f63d344d640 1 -- 172.21.15.67:0/4173866898 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f63cc063fc0 con 0x7f63cc1040a0 2024-09-21T06:28:08.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.752+0000 7f63c27fc640 1 -- 172.21.15.67:0/4173866898 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v18)=0 v18) ==== 111+0+0 (secure 0 0 0) 0x7f63c40173d0 con 0x7f63cc1040a0 2024-09-21T06:28:08.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.754+0000 7f63d344d640 1 -- 172.21.15.67:0/4173866898 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f63b803fb20 msgr2=0x7f63b8042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:08.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.754+0000 7f63d344d640 1 --2- 172.21.15.67:0/4173866898 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f63b803fb20 0x7f63b8042030 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f63c8009fc0 tx=0x7f63c8005f90 comp rx=0 tx=0).stop 2024-09-21T06:28:08.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.754+0000 7f63d344d640 1 -- 172.21.15.67:0/4173866898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63cc1040a0 msgr2=0x7f63cc1a1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:08.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.754+0000 7f63d344d640 1 --2- 172.21.15.67:0/4173866898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63cc1040a0 0x7f63cc1a1060 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f63c400efd0 tx=0x7f63c400c6a0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.754+0000 7f63d344d640 1 -- 172.21.15.67:0/4173866898 shutdown_connections 2024-09-21T06:28:08.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.754+0000 7f63d344d640 1 --2- 172.21.15.67:0/4173866898 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f63b803fb20 0x7f63b8042030 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.754+0000 7f63d344d640 1 --2- 172.21.15.67:0/4173866898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63cc1040a0 0x7f63cc1a1060 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.754+0000 7f63d344d640 1 --2- 172.21.15.67:0/4173866898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63cc103700 0x7f63cc19eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.754+0000 7f63d344d640 1 --2- 172.21.15.67:0/4173866898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63cc102670 0x7f63cc06c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:08.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.754+0000 7f63d344d640 1 -- 172.21.15.67:0/4173866898 >> 172.21.15.67:0/4173866898 conn(0x7f63cc0fdfe0 msgr2=0x7f63cc106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:08.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.754+0000 7f63d344d640 1 -- 172.21.15.67:0/4173866898 shutdown_connections 2024-09-21T06:28:08.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:07.754+0000 7f63d344d640 1 -- 172.21.15.67:0/4173866898 wait complete. 2024-09-21T06:28:09.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:09 smithi203 ceph-mon[33735]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:09.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:09 smithi203 ceph-mon[33735]: mgrmap e14: x(active, since 64s), standbys: z 2024-09-21T06:28:09.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:09 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T06:28:09.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:09 smithi203 ceph-mon[33735]: from='client.? ' entity='client.admin' 2024-09-21T06:28:09.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:09 smithi203 ceph-mon[33735]: from='client.? ' entity='client.admin' 2024-09-21T06:28:09.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:09 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2317409681' entity='client.admin' 2024-09-21T06:28:09.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:09 smithi203 ceph-mon[33735]: from='client.? ' entity='client.admin' 2024-09-21T06:28:09.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:09 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:09.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:09 smithi067 ceph-mon[31232]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:09.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:09 smithi067 ceph-mon[31232]: mgrmap e14: x(active, since 64s), standbys: z 2024-09-21T06:28:09.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:09 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T06:28:09.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:09 smithi067 ceph-mon[31232]: from='client.? ' entity='client.admin' 2024-09-21T06:28:09.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:09 smithi067 ceph-mon[31232]: from='client.? ' entity='client.admin' 2024-09-21T06:28:09.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:09 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2317409681' entity='client.admin' 2024-09-21T06:28:09.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:09 smithi067 ceph-mon[31232]: from='client.? ' entity='client.admin' 2024-09-21T06:28:09.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:09 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:09.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:09 smithi100 ceph-mon[33401]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:09.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:09 smithi100 ceph-mon[33401]: mgrmap e14: x(active, since 64s), standbys: z 2024-09-21T06:28:09.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:09 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T06:28:09.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:09 smithi100 ceph-mon[33401]: from='client.? ' entity='client.admin' 2024-09-21T06:28:09.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:09 smithi100 ceph-mon[33401]: from='client.? ' entity='client.admin' 2024-09-21T06:28:09.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:09 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2317409681' entity='client.admin' 2024-09-21T06:28:09.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:09 smithi100 ceph-mon[33401]: from='client.? ' entity='client.admin' 2024-09-21T06:28:09.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:09 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:10.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.571+0000 7f8bf3885640 1 -- 172.21.15.67:0/26147625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8bec1094b0 msgr2=0x7f8bec10b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:10.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.571+0000 7f8bf3885640 1 --2- 172.21.15.67:0/26147625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8bec1094b0 0x7f8bec10b9c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8be4009ca0 tx=0x7f8be402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:28:10.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.571+0000 7f8bf3885640 1 -- 172.21.15.67:0/26147625 shutdown_connections 2024-09-21T06:28:10.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.571+0000 7f8bf3885640 1 --2- 172.21.15.67:0/26147625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bec102570 0x7f8bec108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:10.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.571+0000 7f8bf3885640 1 --2- 172.21.15.67:0/26147625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8bec1094b0 0x7f8bec10b9c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:10.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.571+0000 7f8bf3885640 1 --2- 172.21.15.67:0/26147625 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bec101b40 0x7f8bec101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:10.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.571+0000 7f8bf3885640 1 -- 172.21.15.67:0/26147625 >> 172.21.15.67:0/26147625 conn(0x7f8bec0fd770 msgr2=0x7f8bec0ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:10.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.571+0000 7f8bf3885640 1 -- 172.21.15.67:0/26147625 shutdown_connections 2024-09-21T06:28:10.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.571+0000 7f8bf3885640 1 -- 172.21.15.67:0/26147625 wait complete. 2024-09-21T06:28:10.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.572+0000 7f8bf3885640 1 Processor -- start 2024-09-21T06:28:10.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.572+0000 7f8bf3885640 1 -- start start 2024-09-21T06:28:10.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.572+0000 7f8bf3885640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bec101b40 0x7f8bec199d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:10.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.572+0000 7f8bf3885640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8bec102570 0x7f8bec19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:10.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.572+0000 7f8bf3885640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bec1094b0 0x7f8bec19af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:10.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.572+0000 7f8bf3885640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec10f020 con 0x7f8bec102570 2024-09-21T06:28:10.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.572+0000 7f8bf3885640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec10eea0 con 0x7f8bec101b40 2024-09-21T06:28:10.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.572+0000 7f8bf3885640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec10f1a0 con 0x7f8bec1094b0 2024-09-21T06:28:10.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.572+0000 7f8bf0e20640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8bec102570 0x7f8bec19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:10.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.572+0000 7f8bf0e20640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8bec102570 0x7f8bec19a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57146/0 (socket says 172.21.15.67:57146) 2024-09-21T06:28:10.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.573+0000 7f8bf0e20640 1 -- 172.21.15.67:0/2317409681 learned_addr learned my addr 172.21.15.67:0/2317409681 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:28:10.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.573+0000 7f8bf1621640 1 --2- 172.21.15.67:0/2317409681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bec101b40 0x7f8bec199d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:10.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.573+0000 7f8bf1e22640 1 --2- 172.21.15.67:0/2317409681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bec1094b0 0x7f8bec19af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:10.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.573+0000 7f8bf0e20640 1 -- 172.21.15.67:0/2317409681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bec1094b0 msgr2=0x7f8bec19af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:10.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.573+0000 7f8bf0e20640 1 --2- 172.21.15.67:0/2317409681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bec1094b0 0x7f8bec19af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:10.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.573+0000 7f8bf0e20640 1 -- 172.21.15.67:0/2317409681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bec101b40 msgr2=0x7f8bec199d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:10.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.573+0000 7f8bf0e20640 1 --2- 172.21.15.67:0/2317409681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bec101b40 0x7f8bec199d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:10.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.573+0000 7f8bf0e20640 1 -- 172.21.15.67:0/2317409681 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bec19b760 con 0x7f8bec102570 2024-09-21T06:28:10.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.573+0000 7f8bf1621640 1 --2- 172.21.15.67:0/2317409681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bec101b40 0x7f8bec199d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:28:10.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.573+0000 7f8bf0e20640 1 --2- 172.21.15.67:0/2317409681 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8bec102570 0x7f8bec19a3f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8be800d950 tx=0x7f8be800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:10.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.573+0000 7f8bde7fc640 1 -- 172.21.15.67:0/2317409681 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be8017030 con 0x7f8bec102570 2024-09-21T06:28:10.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.573+0000 7f8bf3885640 1 -- 172.21.15.67:0/2317409681 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bec19ba50 con 0x7f8bec102570 2024-09-21T06:28:10.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.573+0000 7f8bde7fc640 1 -- 172.21.15.67:0/2317409681 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7f8be8004020 con 0x7f8bec102570 2024-09-21T06:28:10.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.573+0000 7f8bde7fc640 1 -- 172.21.15.67:0/2317409681 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be8005470 con 0x7f8bec102570 2024-09-21T06:28:10.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.573+0000 7f8bf3885640 1 -- 172.21.15.67:0/2317409681 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bec1a2150 con 0x7f8bec102570 2024-09-21T06:28:10.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.575+0000 7f8bf3885640 1 -- 172.21.15.67:0/2317409681 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bec101f50 con 0x7f8bec102570 2024-09-21T06:28:10.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.575+0000 7f8bde7fc640 1 -- 172.21.15.67:0/2317409681 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 14) ==== 103177+0+0 (secure 0 0 0) 0x7f8be8009a30 con 0x7f8bec102570 2024-09-21T06:28:10.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.576+0000 7f8bde7fc640 1 --2- 172.21.15.67:0/2317409681 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f8bd807afd0 0x7f8bd807d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:10.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.576+0000 7f8bde7fc640 1 -- 172.21.15.67:0/2317409681 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8be809c660 con 0x7f8bec102570 2024-09-21T06:28:10.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.576+0000 7f8bf1621640 1 --2- 172.21.15.67:0/2317409681 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f8bd807afd0 0x7f8bd807d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:10.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.576+0000 7f8bf1621640 1 --2- 172.21.15.67:0/2317409681 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f8bd807afd0 0x7f8bd807d4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8be0009fc0 tx=0x7f8be0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:10.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.581+0000 7f8bde7fc640 1 -- 172.21.15.67:0/2317409681 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8be8064ce0 con 0x7f8bec102570 2024-09-21T06:28:10.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.774+0000 7f8bf3885640 1 -- 172.21.15.67:0/2317409681 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f8bec063fc0 con 0x7f8bec102570 2024-09-21T06:28:10.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.778+0000 7f8bde7fc640 1 -- 172.21.15.67:0/2317409681 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v21)=0 v21) ==== 120+0+0 (secure 0 0 0) 0x7f8be80690c0 con 0x7f8bec102570 2024-09-21T06:28:10.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.780+0000 7f8bf3885640 1 -- 172.21.15.67:0/2317409681 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f8bd807afd0 msgr2=0x7f8bd807d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:10.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.780+0000 7f8bf3885640 1 --2- 172.21.15.67:0/2317409681 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f8bd807afd0 0x7f8bd807d4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8be0009fc0 tx=0x7f8be0005f90 comp rx=0 tx=0).stop 2024-09-21T06:28:10.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.780+0000 7f8bf3885640 1 -- 172.21.15.67:0/2317409681 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8bec102570 msgr2=0x7f8bec19a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:10.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.780+0000 7f8bf3885640 1 --2- 172.21.15.67:0/2317409681 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8bec102570 0x7f8bec19a3f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8be800d950 tx=0x7f8be800de10 comp rx=0 tx=0).stop 2024-09-21T06:28:10.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.780+0000 7f8bf3885640 1 -- 172.21.15.67:0/2317409681 shutdown_connections 2024-09-21T06:28:10.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.780+0000 7f8bf3885640 1 --2- 172.21.15.67:0/2317409681 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f8bd807afd0 0x7f8bd807d4e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:10.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.780+0000 7f8bf3885640 1 --2- 172.21.15.67:0/2317409681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bec1094b0 0x7f8bec19af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:10.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.780+0000 7f8bf3885640 1 --2- 172.21.15.67:0/2317409681 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8bec102570 0x7f8bec19a3f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:10.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.780+0000 7f8bf3885640 1 --2- 172.21.15.67:0/2317409681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bec101b40 0x7f8bec199d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:10.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.780+0000 7f8bf3885640 1 -- 172.21.15.67:0/2317409681 >> 172.21.15.67:0/2317409681 conn(0x7f8bec0fd770 msgr2=0x7f8bec107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:10.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.780+0000 7f8bf3885640 1 -- 172.21.15.67:0/2317409681 shutdown_connections 2024-09-21T06:28:10.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.780+0000 7f8bf3885640 1 -- 172.21.15.67:0/2317409681 wait complete. 2024-09-21T06:28:10.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.223+0000 7f75b40ab640 1 -- 172.21.15.67:0/303590651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75ac106930 msgr2=0x7f75ac108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:10.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.223+0000 7f75b40ab640 1 --2- 172.21.15.67:0/303590651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75ac106930 0x7f75ac108d50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f75a0009900 tx=0x7f75a002a410 comp rx=0 tx=0).stop 2024-09-21T06:28:10.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.223+0000 7f75b40ab640 1 -- 172.21.15.67:0/303590651 shutdown_connections 2024-09-21T06:28:10.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.223+0000 7f75b40ab640 1 --2- 172.21.15.67:0/303590651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75ac109470 0x7f75ac10b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:10.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.223+0000 7f75b40ab640 1 --2- 172.21.15.67:0/303590651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75ac106930 0x7f75ac108d50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:10.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.223+0000 7f75b40ab640 1 --2- 172.21.15.67:0/303590651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75ac06b920 0x7f75ac1063f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:10.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.223+0000 7f75b40ab640 1 -- 172.21.15.67:0/303590651 >> 172.21.15.67:0/303590651 conn(0x7f75ac0fdfb0 msgr2=0x7f75ac100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:10.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.223+0000 7f75b40ab640 1 -- 172.21.15.67:0/303590651 shutdown_connections 2024-09-21T06:28:10.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.223+0000 7f75b40ab640 1 -- 172.21.15.67:0/303590651 wait complete. 2024-09-21T06:28:10.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.224+0000 7f75b40ab640 1 Processor -- start 2024-09-21T06:28:10.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.224+0000 7f75b40ab640 1 -- start start 2024-09-21T06:28:10.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.224+0000 7f75b40ab640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75ac06b920 0x7f75ac199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:10.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.224+0000 7f75b40ab640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75ac106930 0x7f75ac19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:10.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.224+0000 7f75b40ab640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75ac109470 0x7f75ac19af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:10.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.224+0000 7f75b40ab640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac10ef90 con 0x7f75ac109470 2024-09-21T06:28:10.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.224+0000 7f75b40ab640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac10ee10 con 0x7f75ac06b920 2024-09-21T06:28:10.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.224+0000 7f75b40ab640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac10f110 con 0x7f75ac106930 2024-09-21T06:28:10.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.224+0000 7f75b2648640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75ac109470 0x7f75ac19af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:10.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.224+0000 7f75b1e47640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75ac06b920 0x7f75ac199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:10.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.224+0000 7f75b1646640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75ac106930 0x7f75ac19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:10.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.224+0000 7f75b2648640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75ac109470 0x7f75ac19af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60326/0 (socket says 172.21.15.67:60326) 2024-09-21T06:28:10.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.224+0000 7f75b1e47640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75ac06b920 0x7f75ac199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.67:33180/0 (socket says 172.21.15.67:33180) 2024-09-21T06:28:10.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.224+0000 7f75b2648640 1 -- 172.21.15.67:0/2476143657 learned_addr learned my addr 172.21.15.67:0/2476143657 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:28:10.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.224+0000 7f75b1646640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75ac106930 0x7f75ac19a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.67:54426/0 (socket says 172.21.15.67:54426) 2024-09-21T06:28:10.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.224+0000 7f75b2648640 1 -- 172.21.15.67:0/2476143657 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75ac106930 msgr2=0x7f75ac19a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:10.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.224+0000 7f75b2648640 1 --2- 172.21.15.67:0/2476143657 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75ac106930 0x7f75ac19a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:10.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.224+0000 7f75b2648640 1 -- 172.21.15.67:0/2476143657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75ac06b920 msgr2=0x7f75ac199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:10.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.224+0000 7f75b2648640 1 --2- 172.21.15.67:0/2476143657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75ac06b920 0x7f75ac199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:10.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.225+0000 7f75b2648640 1 -- 172.21.15.67:0/2476143657 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75ac19b760 con 0x7f75ac109470 2024-09-21T06:28:10.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.225+0000 7f75b1e47640 1 --2- 172.21.15.67:0/2476143657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75ac06b920 0x7f75ac199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:28:10.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.225+0000 7f75b2648640 1 --2- 172.21.15.67:0/2476143657 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75ac109470 0x7f75ac19af40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f75a400d750 tx=0x7f75a400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:10.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.225+0000 7f75aaffd640 1 -- 172.21.15.67:0/2476143657 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a4017070 con 0x7f75ac109470 2024-09-21T06:28:10.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.225+0000 7f75aaffd640 1 -- 172.21.15.67:0/2476143657 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7f75a4005470 con 0x7f75ac109470 2024-09-21T06:28:10.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.225+0000 7f75b40ab640 1 -- 172.21.15.67:0/2476143657 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75ac19ba50 con 0x7f75ac109470 2024-09-21T06:28:10.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.225+0000 7f75aaffd640 1 -- 172.21.15.67:0/2476143657 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a4004aa0 con 0x7f75ac109470 2024-09-21T06:28:10.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.225+0000 7f75b40ab640 1 -- 172.21.15.67:0/2476143657 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75ac1023a0 con 0x7f75ac109470 2024-09-21T06:28:10.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.226+0000 7f75b40ab640 1 -- 172.21.15.67:0/2476143657 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75ac10f750 con 0x7f75ac109470 2024-09-21T06:28:10.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.226+0000 7f75aaffd640 1 -- 172.21.15.67:0/2476143657 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7f75a4004c40 con 0x7f75ac109470 2024-09-21T06:28:10.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.227+0000 7f75aaffd640 1 --2- 172.21.15.67:0/2476143657 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f759403fb20 0x7f7594042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:10.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.227+0000 7f75aaffd640 1 -- 172.21.15.67:0/2476143657 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f75a4053fe0 con 0x7f75ac109470 2024-09-21T06:28:10.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.227+0000 7f75b1e47640 1 --2- 172.21.15.67:0/2476143657 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f759403fb20 0x7f7594042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:10.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.227+0000 7f75b1e47640 1 --2- 172.21.15.67:0/2476143657 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f759403fb20 0x7f7594042030 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f759c009fd0 tx=0x7f759c009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:10.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.232+0000 7f75aaffd640 1 -- 172.21.15.67:0/2476143657 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f75a4009bd0 con 0x7f75ac109470 2024-09-21T06:28:10.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.432+0000 7f75b40ab640 1 -- 172.21.15.67:0/2476143657 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f75ac02e6e0 con 0x7f75ac109470 2024-09-21T06:28:10.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.436+0000 7f75aaffd640 1 -- 172.21.15.67:0/2476143657 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v13)=0 v13) ==== 106+0+0 (secure 0 0 0) 0x7f75a4022d10 con 0x7f75ac109470 2024-09-21T06:28:10.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.438+0000 7f75b40ab640 1 -- 172.21.15.67:0/2476143657 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f759403fb20 msgr2=0x7f7594042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:10.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.438+0000 7f75b40ab640 1 --2- 172.21.15.67:0/2476143657 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f759403fb20 0x7f7594042030 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f759c009fd0 tx=0x7f759c009680 comp rx=0 tx=0).stop 2024-09-21T06:28:10.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.438+0000 7f75b40ab640 1 -- 172.21.15.67:0/2476143657 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75ac109470 msgr2=0x7f75ac19af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:10.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.438+0000 7f75b40ab640 1 --2- 172.21.15.67:0/2476143657 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75ac109470 0x7f75ac19af40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f75a400d750 tx=0x7f75a400dc10 comp rx=0 tx=0).stop 2024-09-21T06:28:10.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.438+0000 7f75b40ab640 1 -- 172.21.15.67:0/2476143657 shutdown_connections 2024-09-21T06:28:10.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.438+0000 7f75b40ab640 1 --2- 172.21.15.67:0/2476143657 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f759403fb20 0x7f7594042030 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:10.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.438+0000 7f75b40ab640 1 --2- 172.21.15.67:0/2476143657 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75ac109470 0x7f75ac19af40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:10.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.438+0000 7f75b40ab640 1 --2- 172.21.15.67:0/2476143657 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75ac106930 0x7f75ac19a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:10.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.438+0000 7f75b40ab640 1 --2- 172.21.15.67:0/2476143657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75ac06b920 0x7f75ac199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:10.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.438+0000 7f75b40ab640 1 -- 172.21.15.67:0/2476143657 >> 172.21.15.67:0/2476143657 conn(0x7f75ac0fdfb0 msgr2=0x7f75ac105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:10.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.438+0000 7f75b40ab640 1 -- 172.21.15.67:0/2476143657 shutdown_connections 2024-09-21T06:28:10.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:04.438+0000 7f75b40ab640 1 -- 172.21.15.67:0/2476143657 wait complete. 2024-09-21T06:28:10.591 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:10 smithi067 ceph-mon[31232]: Standby manager daemon y started 2024-09-21T06:28:10.591 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:10 smithi067 ceph-mon[31232]: from='mgr.? 172.21.15.100:0/2471760599' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T06:28:10.591 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:10 smithi067 ceph-mon[31232]: from='mgr.? 172.21.15.100:0/2471760599' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T06:28:10.591 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:10 smithi067 ceph-mon[31232]: from='mgr.? 172.21.15.100:0/2471760599' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T06:28:10.591 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:10 smithi067 ceph-mon[31232]: from='mgr.? 172.21.15.100:0/2471760599' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T06:28:10.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:10 smithi203 ceph-mon[33735]: Standby manager daemon y started 2024-09-21T06:28:10.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:10 smithi203 ceph-mon[33735]: from='mgr.? 172.21.15.100:0/2471760599' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T06:28:10.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:10 smithi203 ceph-mon[33735]: from='mgr.? 172.21.15.100:0/2471760599' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T06:28:10.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:10 smithi203 ceph-mon[33735]: from='mgr.? 172.21.15.100:0/2471760599' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T06:28:10.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:10 smithi203 ceph-mon[33735]: from='mgr.? 172.21.15.100:0/2471760599' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T06:28:10.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:10 smithi100 ceph-mon[33401]: Standby manager daemon y started 2024-09-21T06:28:10.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:10 smithi100 ceph-mon[33401]: from='mgr.? 172.21.15.100:0/2471760599' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T06:28:10.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:10 smithi100 ceph-mon[33401]: from='mgr.? 172.21.15.100:0/2471760599' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T06:28:10.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:10 smithi100 ceph-mon[33401]: from='mgr.? 172.21.15.100:0/2471760599' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T06:28:10.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:10 smithi100 ceph-mon[33401]: from='mgr.? 172.21.15.100:0/2471760599' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T06:28:11.117 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:28:11.117 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:28:11.120 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:28:11.120 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:28:11.120 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:28:11.123 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:28:11.123 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:28:11.512 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:11 smithi067 ceph-mon[31232]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:11.513 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:11 smithi067 ceph-mon[31232]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-21T06:28:11.513 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:11 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T06:28:11.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.882+0000 7f8804d00640 1 -- 172.21.15.67:0/2023760545 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88001068d0 msgr2=0x7f8800108d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:11.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.882+0000 7f8804d00640 1 --2- 172.21.15.67:0/2023760545 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88001068d0 0x7f8800108d30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009900 tx=0x7f87f402a410 comp rx=0 tx=0).stop 2024-09-21T06:28:11.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.882+0000 7f8804d00640 1 -- 172.21.15.67:0/2023760545 shutdown_connections 2024-09-21T06:28:11.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.882+0000 7f8804d00640 1 --2- 172.21.15.67:0/2023760545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8800109470 0x7f880010b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:11.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.882+0000 7f8804d00640 1 --2- 172.21.15.67:0/2023760545 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88001068d0 0x7f8800108d30 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:11.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.882+0000 7f8804d00640 1 --2- 172.21.15.67:0/2023760545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f880006b980 0x7f8800106390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:11.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.882+0000 7f8804d00640 1 -- 172.21.15.67:0/2023760545 >> 172.21.15.67:0/2023760545 conn(0x7f88000fdfb0 msgr2=0x7f8800100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:11.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.882+0000 7f8804d00640 1 -- 172.21.15.67:0/2023760545 shutdown_connections 2024-09-21T06:28:11.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.882+0000 7f8804d00640 1 -- 172.21.15.67:0/2023760545 wait complete. 2024-09-21T06:28:11.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.883+0000 7f8804d00640 1 Processor -- start 2024-09-21T06:28:11.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.883+0000 7f8804d00640 1 -- start start 2024-09-21T06:28:11.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.883+0000 7f8804d00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f880006b980 0x7f8800199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:11.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.883+0000 7f8804d00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88001068d0 0x7f880019a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:11.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.883+0000 7f8804d00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8800109470 0x7f880019af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:11.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.883+0000 7f8804d00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f880010efb0 con 0x7f880006b980 2024-09-21T06:28:11.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.883+0000 7f8804d00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f880010ee30 con 0x7f88001068d0 2024-09-21T06:28:11.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.883+0000 7f8804d00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f880010f130 con 0x7f8800109470 2024-09-21T06:28:11.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.883+0000 7f87fe59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f880006b980 0x7f8800199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:11.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.883+0000 7f87fe59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f880006b980 0x7f8800199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57156/0 (socket says 172.21.15.67:57156) 2024-09-21T06:28:11.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.883+0000 7f87fe59c640 1 -- 172.21.15.67:0/2826606483 learned_addr learned my addr 172.21.15.67:0/2826606483 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:28:11.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.883+0000 7f87fdd9b640 1 --2- 172.21.15.67:0/2826606483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88001068d0 0x7f880019a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:11.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.883+0000 7f87fed9d640 1 --2- 172.21.15.67:0/2826606483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8800109470 0x7f880019af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:11.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.883+0000 7f87fdd9b640 1 -- 172.21.15.67:0/2826606483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8800109470 msgr2=0x7f880019af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:11.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.883+0000 7f87fdd9b640 1 --2- 172.21.15.67:0/2826606483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8800109470 0x7f880019af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:11.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.883+0000 7f87fdd9b640 1 -- 172.21.15.67:0/2826606483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f880006b980 msgr2=0x7f8800199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:11.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.884+0000 7f87fdd9b640 1 --2- 172.21.15.67:0/2826606483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f880006b980 0x7f8800199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:11.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.884+0000 7f87fdd9b640 1 -- 172.21.15.67:0/2826606483 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880019b780 con 0x7f88001068d0 2024-09-21T06:28:11.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.884+0000 7f87fe59c640 1 --2- 172.21.15.67:0/2826606483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f880006b980 0x7f8800199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:28:11.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.884+0000 7f87fed9d640 1 --2- 172.21.15.67:0/2826606483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8800109470 0x7f880019af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:28:11.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.884+0000 7f87fdd9b640 1 --2- 172.21.15.67:0/2826606483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88001068d0 0x7f880019a3c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f87f402a930 tx=0x7f87f4004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:11.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.884+0000 7f87ef7fe640 1 -- 172.21.15.67:0/2826606483 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f403f070 con 0x7f88001068d0 2024-09-21T06:28:11.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.884+0000 7f87ef7fe640 1 -- 172.21.15.67:0/2826606483 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7f87f40318b0 con 0x7f88001068d0 2024-09-21T06:28:11.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.884+0000 7f87ef7fe640 1 -- 172.21.15.67:0/2826606483 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f4035030 con 0x7f88001068d0 2024-09-21T06:28:11.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.884+0000 7f8804d00640 1 -- 172.21.15.67:0/2826606483 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f880019ba10 con 0x7f88001068d0 2024-09-21T06:28:11.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.884+0000 7f8804d00640 1 -- 172.21.15.67:0/2826606483 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88001a2170 con 0x7f88001068d0 2024-09-21T06:28:11.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.885+0000 7f8804d00640 1 -- 172.21.15.67:0/2826606483 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f880010f770 con 0x7f88001068d0 2024-09-21T06:28:11.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.886+0000 7f87ef7fe640 1 -- 172.21.15.67:0/2826606483 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 14) ==== 103177+0+0 (secure 0 0 0) 0x7f87f4042070 con 0x7f88001068d0 2024-09-21T06:28:11.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.886+0000 7f87ef7fe640 1 --2- 172.21.15.67:0/2826606483 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f87e807afd0 0x7f87e807d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:11.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.886+0000 7f87ef7fe640 1 -- 172.21.15.67:0/2826606483 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f87f40bd5c0 con 0x7f88001068d0 2024-09-21T06:28:11.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.886+0000 7f87fe59c640 1 --2- 172.21.15.67:0/2826606483 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f87e807afd0 0x7f87e807d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:11.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.887+0000 7f87fe59c640 1 --2- 172.21.15.67:0/2826606483 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f87e807afd0 0x7f87e807d4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f87f0011550 tx=0x7f87f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:11.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.891+0000 7f87ef7fe640 1 -- 172.21.15.67:0/2826606483 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f87f408ef50 con 0x7f88001068d0 2024-09-21T06:28:11.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:09.081+0000 7f8804d00640 1 -- 172.21.15.67:0/2826606483 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f880002e6e0 con 0x7f88001068d0 2024-09-21T06:28:11.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:09.087+0000 7f87ef7fe640 1 -- 172.21.15.67:0/2826606483 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v22)=0 v22) ==== 119+0+0 (secure 0 0 0) 0x7f87f4031d60 con 0x7f88001068d0 2024-09-21T06:28:11.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:09.089+0000 7f8804d00640 1 -- 172.21.15.67:0/2826606483 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f87e807afd0 msgr2=0x7f87e807d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:11.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:09.089+0000 7f8804d00640 1 --2- 172.21.15.67:0/2826606483 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f87e807afd0 0x7f87e807d4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f87f0011550 tx=0x7f87f0005f90 comp rx=0 tx=0).stop 2024-09-21T06:28:11.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:09.089+0000 7f8804d00640 1 -- 172.21.15.67:0/2826606483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88001068d0 msgr2=0x7f880019a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:11.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:09.089+0000 7f8804d00640 1 --2- 172.21.15.67:0/2826606483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88001068d0 0x7f880019a3c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f87f402a930 tx=0x7f87f4004270 comp rx=0 tx=0).stop 2024-09-21T06:28:11.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:09.089+0000 7f8804d00640 1 -- 172.21.15.67:0/2826606483 shutdown_connections 2024-09-21T06:28:11.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:09.089+0000 7f8804d00640 1 --2- 172.21.15.67:0/2826606483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88001068d0 0x7f880019a3c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:11.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:09.089+0000 7f8804d00640 1 --2- 172.21.15.67:0/2826606483 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f87e807afd0 0x7f87e807d4e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:11.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:09.089+0000 7f8804d00640 1 --2- 172.21.15.67:0/2826606483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8800109470 0x7f880019af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:11.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:09.089+0000 7f8804d00640 1 --2- 172.21.15.67:0/2826606483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f880006b980 0x7f8800199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:11.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:09.089+0000 7f8804d00640 1 -- 172.21.15.67:0/2826606483 >> 172.21.15.67:0/2826606483 conn(0x7f88000fdfb0 msgr2=0x7f8800104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:11.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:09.089+0000 7f8804d00640 1 -- 172.21.15.67:0/2826606483 shutdown_connections 2024-09-21T06:28:11.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:09.089+0000 7f8804d00640 1 -- 172.21.15.67:0/2826606483 wait complete. 2024-09-21T06:28:11.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:11 smithi203 ceph-mon[33735]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:11.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:11 smithi203 ceph-mon[33735]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-21T06:28:11.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:11 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T06:28:11.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.742+0000 7f63caf65640 1 -- 172.21.15.67:0/1263619065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63c4108d60 msgr2=0x7f63c410b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:11.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.742+0000 7f63caf65640 1 --2- 172.21.15.67:0/1263619065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63c4108d60 0x7f63c410b180 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009900 tx=0x7f63b002a410 comp rx=0 tx=0).stop 2024-09-21T06:28:11.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.742+0000 7f63caf65640 1 -- 172.21.15.67:0/1263619065 shutdown_connections 2024-09-21T06:28:11.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.742+0000 7f63caf65640 1 --2- 172.21.15.67:0/1263619065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63c410b6c0 0x7f63c410dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:11.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.742+0000 7f63caf65640 1 --2- 172.21.15.67:0/1263619065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63c4108d60 0x7f63c410b180 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:11.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.742+0000 7f63caf65640 1 --2- 172.21.15.67:0/1263619065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63c406c410 0x7f63c4108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:11.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.742+0000 7f63caf65640 1 -- 172.21.15.67:0/1263619065 >> 172.21.15.67:0/1263619065 conn(0x7f63c40fdfe0 msgr2=0x7f63c4100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:11.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.742+0000 7f63caf65640 1 -- 172.21.15.67:0/1263619065 shutdown_connections 2024-09-21T06:28:11.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.742+0000 7f63caf65640 1 -- 172.21.15.67:0/1263619065 wait complete. 2024-09-21T06:28:11.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.743+0000 7f63caf65640 1 Processor -- start 2024-09-21T06:28:11.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.743+0000 7f63caf65640 1 -- start start 2024-09-21T06:28:11.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.743+0000 7f63caf65640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63c406c410 0x7f63c419bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:11.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.743+0000 7f63caf65640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63c4108d60 0x7f63c419c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:11.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.743+0000 7f63caf65640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63c410b6c0 0x7f63c419d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:11.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.743+0000 7f63caf65640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f63c41111f0 con 0x7f63c4108d60 2024-09-21T06:28:11.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.743+0000 7f63caf65640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f63c4111070 con 0x7f63c410b6c0 2024-09-21T06:28:11.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.743+0000 7f63caf65640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f63c4111370 con 0x7f63c406c410 2024-09-21T06:28:11.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.743+0000 7f63c3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63c4108d60 0x7f63c419c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:11.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.743+0000 7f63c3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63c4108d60 0x7f63c419c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57192/0 (socket says 172.21.15.67:57192) 2024-09-21T06:28:11.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.743+0000 7f63c3fff640 1 -- 172.21.15.67:0/2868745700 learned_addr learned my addr 172.21.15.67:0/2868745700 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:28:11.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.744+0000 7f63c9502640 1 --2- 172.21.15.67:0/2868745700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63c410b6c0 0x7f63c419d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:11.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.744+0000 7f63c8d01640 1 --2- 172.21.15.67:0/2868745700 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63c406c410 0x7f63c419bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:11.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.744+0000 7f63c3fff640 1 -- 172.21.15.67:0/2868745700 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63c406c410 msgr2=0x7f63c419bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:11.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.744+0000 7f63c3fff640 1 --2- 172.21.15.67:0/2868745700 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63c406c410 0x7f63c419bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:11.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.744+0000 7f63c3fff640 1 -- 172.21.15.67:0/2868745700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63c410b6c0 msgr2=0x7f63c419d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:11.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.744+0000 7f63c3fff640 1 --2- 172.21.15.67:0/2868745700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63c410b6c0 0x7f63c419d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:11.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.744+0000 7f63c3fff640 1 -- 172.21.15.67:0/2868745700 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c419d950 con 0x7f63c4108d60 2024-09-21T06:28:11.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.744+0000 7f63c9502640 1 --2- 172.21.15.67:0/2868745700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63c410b6c0 0x7f63c419d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:28:11.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.744+0000 7f63c8d01640 1 --2- 172.21.15.67:0/2868745700 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63c406c410 0x7f63c419bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:28:11.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.744+0000 7f63c3fff640 1 --2- 172.21.15.67:0/2868745700 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63c4108d60 0x7f63c419c590 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f63b0005ab0 tx=0x7f63b0005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:11.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.744+0000 7f63c1ffb640 1 -- 172.21.15.67:0/2868745700 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b003f070 con 0x7f63c4108d60 2024-09-21T06:28:11.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.744+0000 7f63c1ffb640 1 -- 172.21.15.67:0/2868745700 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7f63b00044d0 con 0x7f63c4108d60 2024-09-21T06:28:11.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.744+0000 7f63caf65640 1 -- 172.21.15.67:0/2868745700 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63c419dbe0 con 0x7f63c4108d60 2024-09-21T06:28:11.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.744+0000 7f63c1ffb640 1 -- 172.21.15.67:0/2868745700 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b0035030 con 0x7f63c4108d60 2024-09-21T06:28:11.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.744+0000 7f63caf65640 1 -- 172.21.15.67:0/2868745700 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63c41a4170 con 0x7f63c4108d60 2024-09-21T06:28:11.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.745+0000 7f63caf65640 1 -- 172.21.15.67:0/2868745700 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63c4108820 con 0x7f63c4108d60 2024-09-21T06:28:11.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.748+0000 7f63c1ffb640 1 -- 172.21.15.67:0/2868745700 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f63b0042070 con 0x7f63c4108d60 2024-09-21T06:28:11.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.749+0000 7f63c1ffb640 1 --2- 172.21.15.67:0/2868745700 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f63b40b6480 0x7f63b40b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:11.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.749+0000 7f63c8d01640 1 --2- 172.21.15.67:0/2868745700 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f63b40b6480 0x7f63b40b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:11.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.749+0000 7f63c1ffb640 1 -- 172.21.15.67:0/2868745700 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f63b0106460 con 0x7f63c4108d60 2024-09-21T06:28:11.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.749+0000 7f63c8d01640 1 --2- 172.21.15.67:0/2868745700 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f63b40b6480 0x7f63b40b8990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f63b8005d00 tx=0x7f63b8005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:11.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.752+0000 7f63c1ffb640 1 -- 172.21.15.67:0/2868745700 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f63b00ce9c0 con 0x7f63c4108d60 2024-09-21T06:28:11.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:11 smithi100 ceph-mon[33401]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:11.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:11 smithi100 ceph-mon[33401]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-21T06:28:11.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:11 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T06:28:11.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.300+0000 7f94bc9ae640 1 -- 172.21.15.67:0/1313166434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94b81036e0 msgr2=0x7f94b8103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:11.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.300+0000 7f94bc9ae640 1 --2- 172.21.15.67:0/1313166434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94b81036e0 0x7f94b8103af0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f94ac009ca0 tx=0x7f94ac02a950 comp rx=0 tx=0).stop 2024-09-21T06:28:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.300+0000 7f94bc9ae640 1 -- 172.21.15.67:0/1313166434 shutdown_connections 2024-09-21T06:28:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.300+0000 7f94bc9ae640 1 --2- 172.21.15.67:0/1313166434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94b8104080 0x7f94b810aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.300+0000 7f94bc9ae640 1 --2- 172.21.15.67:0/1313166434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94b81036e0 0x7f94b8103af0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.300+0000 7f94bc9ae640 1 --2- 172.21.15.67:0/1313166434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94b8102650 0x7f94b8102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.300+0000 7f94bc9ae640 1 -- 172.21.15.67:0/1313166434 >> 172.21.15.67:0/1313166434 conn(0x7f94b80fdfe0 msgr2=0x7f94b8100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.301+0000 7f94bc9ae640 1 -- 172.21.15.67:0/1313166434 shutdown_connections 2024-09-21T06:28:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.301+0000 7f94bc9ae640 1 -- 172.21.15.67:0/1313166434 wait complete. 2024-09-21T06:28:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.302+0000 7f94bc9ae640 1 Processor -- start 2024-09-21T06:28:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.302+0000 7f94bc9ae640 1 -- start start 2024-09-21T06:28:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.302+0000 7f94bc9ae640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94b8102650 0x7f94b8198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.302+0000 7f94bc9ae640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94b81036e0 0x7f94b81992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.302+0000 7f94bc9ae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94b8104080 0x7f94b8199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.302+0000 7f94bc9ae640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f94b810deb0 con 0x7f94b8102650 2024-09-21T06:28:11.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.302+0000 7f94bc9ae640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f94b810dd30 con 0x7f94b8104080 2024-09-21T06:28:11.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.302+0000 7f94bc9ae640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f94b810e030 con 0x7f94b81036e0 2024-09-21T06:28:11.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.302+0000 7f94b659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94b8102650 0x7f94b8198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:11.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.302+0000 7f94b659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94b8102650 0x7f94b8198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57164/0 (socket says 172.21.15.67:57164) 2024-09-21T06:28:11.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.302+0000 7f94b659c640 1 -- 172.21.15.67:0/4133134880 learned_addr learned my addr 172.21.15.67:0/4133134880 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:28:11.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.302+0000 7f94b6d9d640 1 --2- 172.21.15.67:0/4133134880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94b8104080 0x7f94b8199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:11.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.302+0000 7f94b5d9b640 1 --2- 172.21.15.67:0/4133134880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94b81036e0 0x7f94b81992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:11.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.302+0000 7f94b659c640 1 -- 172.21.15.67:0/4133134880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94b81036e0 msgr2=0x7f94b81992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:11.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.303+0000 7f94b659c640 1 --2- 172.21.15.67:0/4133134880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94b81036e0 0x7f94b81992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:11.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.303+0000 7f94b659c640 1 -- 172.21.15.67:0/4133134880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94b8104080 msgr2=0x7f94b8199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:11.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.303+0000 7f94b659c640 1 --2- 172.21.15.67:0/4133134880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94b8104080 0x7f94b8199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:11.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.303+0000 7f94b659c640 1 -- 172.21.15.67:0/4133134880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b819a5f0 con 0x7f94b8102650 2024-09-21T06:28:11.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.303+0000 7f94b659c640 1 --2- 172.21.15.67:0/4133134880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94b8102650 0x7f94b8198c50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f94a8007b50 tx=0x7f94a800e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:11.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.303+0000 7f94a77fe640 1 -- 172.21.15.67:0/4133134880 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a8007c40 con 0x7f94b8102650 2024-09-21T06:28:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.303+0000 7f94bc9ae640 1 -- 172.21.15.67:0/4133134880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94b819a8e0 con 0x7f94b8102650 2024-09-21T06:28:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.303+0000 7f94a77fe640 1 -- 172.21.15.67:0/4133134880 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7f94a8007de0 con 0x7f94b8102650 2024-09-21T06:28:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.303+0000 7f94a77fe640 1 -- 172.21.15.67:0/4133134880 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a800aad0 con 0x7f94b8102650 2024-09-21T06:28:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.303+0000 7f94bc9ae640 1 -- 172.21.15.67:0/4133134880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94b806bd10 con 0x7f94b8102650 2024-09-21T06:28:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.304+0000 7f94bc9ae640 1 -- 172.21.15.67:0/4133134880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94b810e670 con 0x7f94b8102650 2024-09-21T06:28:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.305+0000 7f94a77fe640 1 -- 172.21.15.67:0/4133134880 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f94a800acb0 con 0x7f94b8102650 2024-09-21T06:28:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.306+0000 7f94a77fe640 1 --2- 172.21.15.67:0/4133134880 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f94a00b6480 0x7f94a00b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.306+0000 7f94b5d9b640 1 --2- 172.21.15.67:0/4133134880 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f94a00b6480 0x7f94a00b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.306+0000 7f94a77fe640 1 -- 172.21.15.67:0/4133134880 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f94a80e6b80 con 0x7f94b8102650 2024-09-21T06:28:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.307+0000 7f94b5d9b640 1 --2- 172.21.15.67:0/4133134880 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f94a00b6480 0x7f94a00b8990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f94ac004ab0 tx=0x7f94ac004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.311+0000 7f94a77fe640 1 -- 172.21.15.67:0/4133134880 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f94a8023060 con 0x7f94b8102650 2024-09-21T06:28:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.511+0000 7f94bc9ae640 1 -- 172.21.15.67:0/4133134880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f94b81092a0 con 0x7f94b8102650 2024-09-21T06:28:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.515+0000 7f94a77fe640 1 -- 172.21.15.67:0/4133134880 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v23)=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f94a80af170 con 0x7f94b8102650 2024-09-21T06:28:11.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.518+0000 7f94bc9ae640 1 -- 172.21.15.67:0/4133134880 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f94a00b6480 msgr2=0x7f94a00b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:11.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.518+0000 7f94bc9ae640 1 --2- 172.21.15.67:0/4133134880 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f94a00b6480 0x7f94a00b8990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f94ac004ab0 tx=0x7f94ac004a40 comp rx=0 tx=0).stop 2024-09-21T06:28:11.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.518+0000 7f94bc9ae640 1 -- 172.21.15.67:0/4133134880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94b8102650 msgr2=0x7f94b8198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:11.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.518+0000 7f94bc9ae640 1 --2- 172.21.15.67:0/4133134880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94b8102650 0x7f94b8198c50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f94a8007b50 tx=0x7f94a800e420 comp rx=0 tx=0).stop 2024-09-21T06:28:11.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.518+0000 7f94bc9ae640 1 -- 172.21.15.67:0/4133134880 shutdown_connections 2024-09-21T06:28:11.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.518+0000 7f94bc9ae640 1 --2- 172.21.15.67:0/4133134880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94b8104080 0x7f94b8199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:11.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.518+0000 7f94bc9ae640 1 --2- 172.21.15.67:0/4133134880 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f94a00b6480 0x7f94a00b8990 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:11.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.518+0000 7f94bc9ae640 1 --2- 172.21.15.67:0/4133134880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94b81036e0 0x7f94b81992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:11.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.518+0000 7f94bc9ae640 1 --2- 172.21.15.67:0/4133134880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94b8102650 0x7f94b8198c50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:11.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.518+0000 7f94bc9ae640 1 -- 172.21.15.67:0/4133134880 >> 172.21.15.67:0/4133134880 conn(0x7f94b80fdfe0 msgr2=0x7f94b8106a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:11.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.518+0000 7f94bc9ae640 1 -- 172.21.15.67:0/4133134880 shutdown_connections 2024-09-21T06:28:11.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.518+0000 7f94bc9ae640 1 -- 172.21.15.67:0/4133134880 wait complete. 2024-09-21T06:28:11.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.944+0000 7f63caf65640 1 -- 172.21.15.67:0/2868745700 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f63c402e640 con 0x7f63c4108d60 2024-09-21T06:28:11.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.948+0000 7f63c1ffb640 1 -- 172.21.15.67:0/2868745700 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v24)=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7f63b00d2da0 con 0x7f63c4108d60 2024-09-21T06:28:11.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.951+0000 7f63caf65640 1 -- 172.21.15.67:0/2868745700 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f63b40b6480 msgr2=0x7f63b40b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:11.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.951+0000 7f63caf65640 1 --2- 172.21.15.67:0/2868745700 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f63b40b6480 0x7f63b40b8990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f63b8005d00 tx=0x7f63b8005ed0 comp rx=0 tx=0).stop 2024-09-21T06:28:11.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.951+0000 7f63caf65640 1 -- 172.21.15.67:0/2868745700 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63c4108d60 msgr2=0x7f63c419c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:11.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.951+0000 7f63caf65640 1 --2- 172.21.15.67:0/2868745700 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63c4108d60 0x7f63c419c590 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f63b0005ab0 tx=0x7f63b0005250 comp rx=0 tx=0).stop 2024-09-21T06:28:11.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.951+0000 7f63caf65640 1 -- 172.21.15.67:0/2868745700 shutdown_connections 2024-09-21T06:28:11.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.951+0000 7f63caf65640 1 --2- 172.21.15.67:0/2868745700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63c410b6c0 0x7f63c419d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:11.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.951+0000 7f63caf65640 1 --2- 172.21.15.67:0/2868745700 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f63b40b6480 0x7f63b40b8990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:11.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.951+0000 7f63caf65640 1 --2- 172.21.15.67:0/2868745700 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63c4108d60 0x7f63c419c590 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:11.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.951+0000 7f63caf65640 1 --2- 172.21.15.67:0/2868745700 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63c406c410 0x7f63c419bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:11.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.951+0000 7f63caf65640 1 -- 172.21.15.67:0/2868745700 >> 172.21.15.67:0/2868745700 conn(0x7f63c40fdfe0 msgr2=0x7f63c4109d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:11.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.951+0000 7f63caf65640 1 -- 172.21.15.67:0/2868745700 shutdown_connections 2024-09-21T06:28:11.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:11.951+0000 7f63caf65640 1 -- 172.21.15.67:0/2868745700 wait complete. 2024-09-21T06:28:12.722 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:12 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/4133134880' entity='client.admin' 2024-09-21T06:28:12.722 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:12 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2868745700' entity='client.admin' 2024-09-21T06:28:12.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:12 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/4133134880' entity='client.admin' 2024-09-21T06:28:12.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:12 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2868745700' entity='client.admin' 2024-09-21T06:28:12.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.909+0000 7fec67cd5640 1 -- 172.21.15.67:0/3604147370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec601047e0 msgr2=0x7fec60104bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:12.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.909+0000 7fec67cd5640 1 --2- 172.21.15.67:0/3604147370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec601047e0 0x7fec60104bf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fec54009900 tx=0x7fec5402a480 comp rx=0 tx=0).stop 2024-09-21T06:28:12.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.909+0000 7fec67cd5640 1 -- 172.21.15.67:0/3604147370 shutdown_connections 2024-09-21T06:28:12.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.909+0000 7fec67cd5640 1 --2- 172.21.15.67:0/3604147370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec60105180 0x7fec6010bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:12.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.909+0000 7fec67cd5640 1 --2- 172.21.15.67:0/3604147370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec601047e0 0x7fec60104bf0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:12.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.909+0000 7fec67cd5640 1 --2- 172.21.15.67:0/3604147370 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec60103750 0x7fec60103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:12.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.909+0000 7fec67cd5640 1 -- 172.21.15.67:0/3604147370 >> 172.21.15.67:0/3604147370 conn(0x7fec600ff0c0 msgr2=0x7fec60101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:12.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.909+0000 7fec67cd5640 1 -- 172.21.15.67:0/3604147370 shutdown_connections 2024-09-21T06:28:12.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.909+0000 7fec67cd5640 1 -- 172.21.15.67:0/3604147370 wait complete. 2024-09-21T06:28:12.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.909+0000 7fec67cd5640 1 Processor -- start 2024-09-21T06:28:12.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.909+0000 7fec67cd5640 1 -- start start 2024-09-21T06:28:12.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.910+0000 7fec67cd5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec60103750 0x7fec60199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:12.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.910+0000 7fec67cd5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec601047e0 0x7fec6019a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:12.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.910+0000 7fec67cd5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec60105180 0x7fec6019af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:12.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.910+0000 7fec67cd5640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fec6010efb0 con 0x7fec601047e0 2024-09-21T06:28:12.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.910+0000 7fec67cd5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fec6010ee30 con 0x7fec60103750 2024-09-21T06:28:12.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.910+0000 7fec67cd5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fec6010f130 con 0x7fec60105180 2024-09-21T06:28:12.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.910+0000 7fec65270640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec601047e0 0x7fec6019a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:12.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.910+0000 7fec65270640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec601047e0 0x7fec6019a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57212/0 (socket says 172.21.15.67:57212) 2024-09-21T06:28:12.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.910+0000 7fec65270640 1 -- 172.21.15.67:0/3704002022 learned_addr learned my addr 172.21.15.67:0/3704002022 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:28:12.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.910+0000 7fec66272640 1 --2- 172.21.15.67:0/3704002022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec60105180 0x7fec6019af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:12.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.910+0000 7fec65a71640 1 --2- 172.21.15.67:0/3704002022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec60103750 0x7fec60199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:12.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.910+0000 7fec65270640 1 -- 172.21.15.67:0/3704002022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec60105180 msgr2=0x7fec6019af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:12.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.910+0000 7fec65270640 1 --2- 172.21.15.67:0/3704002022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec60105180 0x7fec6019af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:12.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.910+0000 7fec65270640 1 -- 172.21.15.67:0/3704002022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec60103750 msgr2=0x7fec60199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:12.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.910+0000 7fec65270640 1 --2- 172.21.15.67:0/3704002022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec60103750 0x7fec60199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:12.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.910+0000 7fec65270640 1 -- 172.21.15.67:0/3704002022 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec6019b6f0 con 0x7fec601047e0 2024-09-21T06:28:12.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.910+0000 7fec65270640 1 --2- 172.21.15.67:0/3704002022 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec601047e0 0x7fec6019a3c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fec54005940 tx=0x7fec54004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:12.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.911+0000 7fec5effd640 1 -- 172.21.15.67:0/3704002022 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5403f070 con 0x7fec601047e0 2024-09-21T06:28:12.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.911+0000 7fec5effd640 1 -- 172.21.15.67:0/3704002022 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7fec54031d00 con 0x7fec601047e0 2024-09-21T06:28:12.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.911+0000 7fec67cd5640 1 -- 172.21.15.67:0/3704002022 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec6019b980 con 0x7fec601047e0 2024-09-21T06:28:12.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.911+0000 7fec5effd640 1 -- 172.21.15.67:0/3704002022 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec54035030 con 0x7fec601047e0 2024-09-21T06:28:12.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.911+0000 7fec67cd5640 1 -- 172.21.15.67:0/3704002022 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec601a1fa0 con 0x7fec601047e0 2024-09-21T06:28:12.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.912+0000 7fec67cd5640 1 -- 172.21.15.67:0/3704002022 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec60103bc0 con 0x7fec601047e0 2024-09-21T06:28:12.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.913+0000 7fec5effd640 1 -- 172.21.15.67:0/3704002022 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7fec54042070 con 0x7fec601047e0 2024-09-21T06:28:12.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.915+0000 7fec5effd640 1 --2- 172.21.15.67:0/3704002022 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fec480b6480 0x7fec480b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:12.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.915+0000 7fec65a71640 1 --2- 172.21.15.67:0/3704002022 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fec480b6480 0x7fec480b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:12.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.915+0000 7fec5effd640 1 -- 172.21.15.67:0/3704002022 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fec54106740 con 0x7fec601047e0 2024-09-21T06:28:12.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.915+0000 7fec65a71640 1 --2- 172.21.15.67:0/3704002022 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fec480b6480 0x7fec480b8990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fec50007350 tx=0x7fec500072e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:12.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:12.918+0000 7fec5effd640 1 -- 172.21.15.67:0/3704002022 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fec540ceca0 con 0x7fec601047e0 2024-09-21T06:28:12.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:12 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/4133134880' entity='client.admin' 2024-09-21T06:28:12.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:12 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2868745700' entity='client.admin' 2024-09-21T06:28:13.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:13.109+0000 7fec67cd5640 1 -- 172.21.15.67:0/3704002022 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7fec6010a930 con 0x7fec601047e0 2024-09-21T06:28:13.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:13.113+0000 7fec5effd640 1 -- 172.21.15.67:0/3704002022 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v25)=0 v25) ==== 117+0+0 (secure 0 0 0) 0x7fec6010a930 con 0x7fec601047e0 2024-09-21T06:28:13.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:13.116+0000 7fec67cd5640 1 -- 172.21.15.67:0/3704002022 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fec480b6480 msgr2=0x7fec480b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:13.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:13.116+0000 7fec67cd5640 1 --2- 172.21.15.67:0/3704002022 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fec480b6480 0x7fec480b8990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fec50007350 tx=0x7fec500072e0 comp rx=0 tx=0).stop 2024-09-21T06:28:13.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:13.116+0000 7fec67cd5640 1 -- 172.21.15.67:0/3704002022 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec601047e0 msgr2=0x7fec6019a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:13.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:13.116+0000 7fec67cd5640 1 --2- 172.21.15.67:0/3704002022 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec601047e0 0x7fec6019a3c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fec54005940 tx=0x7fec54004360 comp rx=0 tx=0).stop 2024-09-21T06:28:13.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:13.116+0000 7fec67cd5640 1 -- 172.21.15.67:0/3704002022 shutdown_connections 2024-09-21T06:28:13.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:13.116+0000 7fec67cd5640 1 --2- 172.21.15.67:0/3704002022 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fec480b6480 0x7fec480b8990 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:13.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:13.116+0000 7fec67cd5640 1 --2- 172.21.15.67:0/3704002022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec60105180 0x7fec6019af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:13.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:13.116+0000 7fec67cd5640 1 --2- 172.21.15.67:0/3704002022 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec601047e0 0x7fec6019a3c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:13.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:13.116+0000 7fec67cd5640 1 --2- 172.21.15.67:0/3704002022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec60103750 0x7fec60199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:13.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:13.116+0000 7fec67cd5640 1 -- 172.21.15.67:0/3704002022 >> 172.21.15.67:0/3704002022 conn(0x7fec600ff0c0 msgr2=0x7fec60107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:13.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:13.116+0000 7fec67cd5640 1 -- 172.21.15.67:0/3704002022 shutdown_connections 2024-09-21T06:28:13.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:13.116+0000 7fec67cd5640 1 -- 172.21.15.67:0/3704002022 wait complete. 2024-09-21T06:28:13.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.465+0000 7f6654ef3640 1 -- 172.21.15.67:0/2879453925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6650109470 msgr2=0x7f665010b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:13.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.465+0000 7f6654ef3640 1 --2- 172.21.15.67:0/2879453925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6650109470 0x7f665010b980 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6648009ca0 tx=0x7f664802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:28:13.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.465+0000 7f6654ef3640 1 -- 172.21.15.67:0/2879453925 shutdown_connections 2024-09-21T06:28:13.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.465+0000 7f6654ef3640 1 --2- 172.21.15.67:0/2879453925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6650106930 0x7f6650108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:13.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.465+0000 7f6654ef3640 1 --2- 172.21.15.67:0/2879453925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6650109470 0x7f665010b980 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:13.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.465+0000 7f6654ef3640 1 --2- 172.21.15.67:0/2879453925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f665006b920 0x7f66501063f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:13.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.465+0000 7f6654ef3640 1 -- 172.21.15.67:0/2879453925 >> 172.21.15.67:0/2879453925 conn(0x7f66500fdfb0 msgr2=0x7f6650100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:13.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.465+0000 7f6654ef3640 1 -- 172.21.15.67:0/2879453925 shutdown_connections 2024-09-21T06:28:13.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.465+0000 7f6654ef3640 1 -- 172.21.15.67:0/2879453925 wait complete. 2024-09-21T06:28:13.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.466+0000 7f6654ef3640 1 Processor -- start 2024-09-21T06:28:13.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.466+0000 7f6654ef3640 1 -- start start 2024-09-21T06:28:13.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.466+0000 7f6654ef3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f665006b920 0x7f66501039b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:13.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.466+0000 7f6654ef3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6650106930 0x7f66501022f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:13.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.466+0000 7f6654ef3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6650109470 0x7f6650102830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:13.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.466+0000 7f6654ef3640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f665010efb0 con 0x7f665006b920 2024-09-21T06:28:13.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.466+0000 7f6654ef3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f665010ee30 con 0x7f6650106930 2024-09-21T06:28:13.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.466+0000 7f6654ef3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f665010f130 con 0x7f6650109470 2024-09-21T06:28:13.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.466+0000 7f664e59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f665006b920 0x7f66501039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:13.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.466+0000 7f6645d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6650106930 0x7f66501022f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:13.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.466+0000 7f664e59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f665006b920 0x7f66501039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57130/0 (socket says 172.21.15.67:57130) 2024-09-21T06:28:13.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.467+0000 7f664e59c640 1 -- 172.21.15.67:0/883256182 learned_addr learned my addr 172.21.15.67:0/883256182 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:28:13.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.467+0000 7f664ed9d640 1 --2- 172.21.15.67:0/883256182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6650109470 0x7f6650102830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:13.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.467+0000 7f6645d9b640 1 -- 172.21.15.67:0/883256182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6650109470 msgr2=0x7f6650102830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:13.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.467+0000 7f6645d9b640 1 --2- 172.21.15.67:0/883256182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6650109470 0x7f6650102830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:13.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.467+0000 7f6645d9b640 1 -- 172.21.15.67:0/883256182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f665006b920 msgr2=0x7f66501039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:13.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.467+0000 7f6645d9b640 1 --2- 172.21.15.67:0/883256182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f665006b920 0x7f66501039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:13.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.467+0000 7f6645d9b640 1 -- 172.21.15.67:0/883256182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6650102ed0 con 0x7f6650106930 2024-09-21T06:28:13.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.467+0000 7f6645d9b640 1 --2- 172.21.15.67:0/883256182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6650106930 0x7f66501022f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f663800e9f0 tx=0x7f663800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:13.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.467+0000 7f6647fff640 1 -- 172.21.15.67:0/883256182 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663800cd80 con 0x7f6650106930 2024-09-21T06:28:13.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.467+0000 7f6647fff640 1 -- 172.21.15.67:0/883256182 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7f6638004470 con 0x7f6650106930 2024-09-21T06:28:13.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.467+0000 7f6654ef3640 1 -- 172.21.15.67:0/883256182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66501a3da0 con 0x7f6650106930 2024-09-21T06:28:13.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.467+0000 7f6647fff640 1 -- 172.21.15.67:0/883256182 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638003e00 con 0x7f6650106930 2024-09-21T06:28:13.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.468+0000 7f6654ef3640 1 -- 172.21.15.67:0/883256182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66501a41a0 con 0x7f6650106930 2024-09-21T06:28:13.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.469+0000 7f6654ef3640 1 -- 172.21.15.67:0/883256182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6650106d40 con 0x7f6650106930 2024-09-21T06:28:13.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.469+0000 7f6647fff640 1 -- 172.21.15.67:0/883256182 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 14) ==== 103177+0+0 (secure 0 0 0) 0x7f6638010430 con 0x7f6650106930 2024-09-21T06:28:13.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.470+0000 7f6647fff640 1 --2- 172.21.15.67:0/883256182 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f663c07afd0 0x7f663c07d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:13.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.470+0000 7f6647fff640 1 -- 172.21.15.67:0/883256182 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f663809cfa0 con 0x7f6650106930 2024-09-21T06:28:13.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.470+0000 7f664e59c640 1 --2- 172.21.15.67:0/883256182 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f663c07afd0 0x7f663c07d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:13.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.470+0000 7f664e59c640 1 --2- 172.21.15.67:0/883256182 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f663c07afd0 0x7f663c07d4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6640009cd0 tx=0x7f6640005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:13.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.474+0000 7f6647fff640 1 -- 172.21.15.67:0/883256182 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f66380699f0 con 0x7f6650106930 2024-09-21T06:28:13.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.688+0000 7f6654ef3640 1 -- 172.21.15.67:0/883256182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f6650063fc0 con 0x7f6650106930 2024-09-21T06:28:13.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.693+0000 7f6647fff640 1 -- 172.21.15.67:0/883256182 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v20)=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f663806e8a0 con 0x7f6650106930 2024-09-21T06:28:13.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.695+0000 7f6654ef3640 1 -- 172.21.15.67:0/883256182 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f663c07afd0 msgr2=0x7f663c07d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:13.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.695+0000 7f6654ef3640 1 --2- 172.21.15.67:0/883256182 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f663c07afd0 0x7f663c07d4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6640009cd0 tx=0x7f6640005f90 comp rx=0 tx=0).stop 2024-09-21T06:28:13.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.695+0000 7f6654ef3640 1 -- 172.21.15.67:0/883256182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6650106930 msgr2=0x7f66501022f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:13.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.695+0000 7f6654ef3640 1 --2- 172.21.15.67:0/883256182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6650106930 0x7f66501022f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f663800e9f0 tx=0x7f663800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:28:13.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.695+0000 7f6654ef3640 1 -- 172.21.15.67:0/883256182 shutdown_connections 2024-09-21T06:28:13.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.695+0000 7f6654ef3640 1 --2- 172.21.15.67:0/883256182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6650106930 0x7f66501022f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:13.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.695+0000 7f6654ef3640 1 --2- 172.21.15.67:0/883256182 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f663c07afd0 0x7f663c07d4e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:13.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.695+0000 7f6654ef3640 1 --2- 172.21.15.67:0/883256182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6650109470 0x7f6650102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:13.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.695+0000 7f6654ef3640 1 --2- 172.21.15.67:0/883256182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f665006b920 0x7f66501039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:13.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.695+0000 7f6654ef3640 1 -- 172.21.15.67:0/883256182 >> 172.21.15.67:0/883256182 conn(0x7f66500fdfb0 msgr2=0x7f6650104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:13.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.695+0000 7f6654ef3640 1 -- 172.21.15.67:0/883256182 shutdown_connections 2024-09-21T06:28:13.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:08.695+0000 7f6654ef3640 1 -- 172.21.15.67:0/883256182 wait complete. 2024-09-21T06:28:13.706 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:13 smithi067 ceph-mon[31232]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:13.706 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:13 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3704002022' entity='client.admin' 2024-09-21T06:28:13.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:13 smithi100 ceph-mon[33401]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:13.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:13 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3704002022' entity='client.admin' 2024-09-21T06:28:13.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:13 smithi203 ceph-mon[33735]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:13.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:13 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3704002022' entity='client.admin' 2024-09-21T06:28:14.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.524+0000 7f5396a0b640 1 -- 172.21.15.67:0/2835560277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5390103750 msgr2=0x7f5390103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:14.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.524+0000 7f5396a0b640 1 --2- 172.21.15.67:0/2835560277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5390103750 0x7f5390103bc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5380009900 tx=0x7f538002a480 comp rx=0 tx=0).stop 2024-09-21T06:28:14.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.524+0000 7f5396a0b640 1 -- 172.21.15.67:0/2835560277 shutdown_connections 2024-09-21T06:28:14.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.524+0000 7f5396a0b640 1 --2- 172.21.15.67:0/2835560277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53901047e0 0x7f5390104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:14.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.524+0000 7f5396a0b640 1 --2- 172.21.15.67:0/2835560277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5390105180 0x7f539010bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:14.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.524+0000 7f5396a0b640 1 --2- 172.21.15.67:0/2835560277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5390103750 0x7f5390103bc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:14.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.524+0000 7f5396a0b640 1 -- 172.21.15.67:0/2835560277 >> 172.21.15.67:0/2835560277 conn(0x7f53900ff0c0 msgr2=0x7f5390101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:14.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.524+0000 7f5396a0b640 1 -- 172.21.15.67:0/2835560277 shutdown_connections 2024-09-21T06:28:14.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.524+0000 7f5396a0b640 1 -- 172.21.15.67:0/2835560277 wait complete. 2024-09-21T06:28:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.525+0000 7f5396a0b640 1 Processor -- start 2024-09-21T06:28:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.525+0000 7f5396a0b640 1 -- start start 2024-09-21T06:28:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.525+0000 7f5396a0b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5390103750 0x7f5390199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.525+0000 7f5396a0b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53901047e0 0x7f539019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.526+0000 7f5396a0b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5390105180 0x7f539019af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.526+0000 7f5396a0b640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f539010efa0 con 0x7f5390105180 2024-09-21T06:28:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.526+0000 7f5396a0b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f539010ee20 con 0x7f5390103750 2024-09-21T06:28:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.526+0000 7f5396a0b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f539010f120 con 0x7f53901047e0 2024-09-21T06:28:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.526+0000 7f5387fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53901047e0 0x7f539019a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.526+0000 7f5394fa8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5390105180 0x7f539019af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.526+0000 7f538ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5390103750 0x7f5390199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:14.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.526+0000 7f5387fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53901047e0 0x7f539019a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.67:37426/0 (socket says 172.21.15.67:37426) 2024-09-21T06:28:14.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.526+0000 7f5387fff640 1 -- 172.21.15.67:0/846890512 learned_addr learned my addr 172.21.15.67:0/846890512 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:28:14.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.526+0000 7f5394fa8640 1 -- 172.21.15.67:0/846890512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53901047e0 msgr2=0x7f539019a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:14.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.526+0000 7f5394fa8640 1 --2- 172.21.15.67:0/846890512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53901047e0 0x7f539019a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:14.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.526+0000 7f5394fa8640 1 -- 172.21.15.67:0/846890512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5390103750 msgr2=0x7f5390199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:14.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.526+0000 7f5394fa8640 1 --2- 172.21.15.67:0/846890512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5390103750 0x7f5390199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:14.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.526+0000 7f5394fa8640 1 -- 172.21.15.67:0/846890512 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539019b720 con 0x7f5390105180 2024-09-21T06:28:14.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.526+0000 7f5394fa8640 1 --2- 172.21.15.67:0/846890512 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5390105180 0x7f539019af00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f538800e940 tx=0x7f538800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:14.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.526+0000 7f538dffb640 1 -- 172.21.15.67:0/846890512 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5388004d20 con 0x7f5390105180 2024-09-21T06:28:14.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.527+0000 7f538dffb640 1 -- 172.21.15.67:0/846890512 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7f5388005020 con 0x7f5390105180 2024-09-21T06:28:14.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.527+0000 7f5396a0b640 1 -- 172.21.15.67:0/846890512 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f539019ba10 con 0x7f5390105180 2024-09-21T06:28:14.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.527+0000 7f538dffb640 1 -- 172.21.15.67:0/846890512 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53880116d0 con 0x7f5390105180 2024-09-21T06:28:14.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.527+0000 7f5396a0b640 1 -- 172.21.15.67:0/846890512 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53901a2110 con 0x7f5390105180 2024-09-21T06:28:14.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.528+0000 7f5396a0b640 1 -- 172.21.15.67:0/846890512 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5390103bc0 con 0x7f5390105180 2024-09-21T06:28:14.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.529+0000 7f538dffb640 1 -- 172.21.15.67:0/846890512 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f538800cb20 con 0x7f5390105180 2024-09-21T06:28:14.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.530+0000 7f538dffb640 1 --2- 172.21.15.67:0/846890512 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f537c0b6500 0x7f537c0b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:14.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.530+0000 7f538ffff640 1 --2- 172.21.15.67:0/846890512 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f537c0b6500 0x7f537c0b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:14.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.530+0000 7f538dffb640 1 -- 172.21.15.67:0/846890512 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f53880e55b0 con 0x7f5390105180 2024-09-21T06:28:14.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.531+0000 7f538ffff640 1 --2- 172.21.15.67:0/846890512 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f537c0b6500 0x7f537c0b8a10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5380000c00 tx=0x7f5380009560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:14.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.535+0000 7f538dffb640 1 -- 172.21.15.67:0/846890512 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5388016030 con 0x7f5390105180 2024-09-21T06:28:14.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.723+0000 7f5396a0b640 1 -- 172.21.15.67:0/846890512 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f5390063fc0 con 0x7f5390105180 2024-09-21T06:28:14.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.727+0000 7f538dffb640 1 -- 172.21.15.67:0/846890512 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v26)=0 v26) ==== 123+0+0 (secure 0 0 0) 0x7f53880b1ef0 con 0x7f5390105180 2024-09-21T06:28:14.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.728+0000 7f538dffb640 1 -- 172.21.15.67:0/846890512 <== mon.0 v2:172.21.15.67:3300/0 8 ==== config(24 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f53880acfd0 con 0x7f5390105180 2024-09-21T06:28:14.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.730+0000 7f5396a0b640 1 -- 172.21.15.67:0/846890512 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f537c0b6500 msgr2=0x7f537c0b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:14.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.730+0000 7f5396a0b640 1 --2- 172.21.15.67:0/846890512 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f537c0b6500 0x7f537c0b8a10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5380000c00 tx=0x7f5380009560 comp rx=0 tx=0).stop 2024-09-21T06:28:14.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.730+0000 7f5396a0b640 1 -- 172.21.15.67:0/846890512 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5390105180 msgr2=0x7f539019af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:14.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.730+0000 7f5396a0b640 1 --2- 172.21.15.67:0/846890512 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5390105180 0x7f539019af00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f538800e940 tx=0x7f538800ee00 comp rx=0 tx=0).stop 2024-09-21T06:28:14.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.730+0000 7f5396a0b640 1 -- 172.21.15.67:0/846890512 shutdown_connections 2024-09-21T06:28:14.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.730+0000 7f5396a0b640 1 --2- 172.21.15.67:0/846890512 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f537c0b6500 0x7f537c0b8a10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:14.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.730+0000 7f5396a0b640 1 --2- 172.21.15.67:0/846890512 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5390105180 0x7f539019af00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:14.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.730+0000 7f5396a0b640 1 --2- 172.21.15.67:0/846890512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53901047e0 0x7f539019a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:14.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.730+0000 7f5396a0b640 1 --2- 172.21.15.67:0/846890512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5390103750 0x7f5390199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:14.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.730+0000 7f5396a0b640 1 -- 172.21.15.67:0/846890512 >> 172.21.15.67:0/846890512 conn(0x7f53900ff0c0 msgr2=0x7f5390107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:14.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.730+0000 7f5396a0b640 1 -- 172.21.15.67:0/846890512 shutdown_connections 2024-09-21T06:28:14.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:14.730+0000 7f5396a0b640 1 -- 172.21.15.67:0/846890512 wait complete. 2024-09-21T06:28:15.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.384+0000 7f3d240d6640 1 -- 172.21.15.67:0/3607476259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d1c1036d0 msgr2=0x7f3d1c103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:15.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.384+0000 7f3d240d6640 1 --2- 172.21.15.67:0/3607476259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d1c1036d0 0x7f3d1c103ae0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c009900 tx=0x7f3d0c02a480 comp rx=0 tx=0).stop 2024-09-21T06:28:15.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.384+0000 7f3d240d6640 1 -- 172.21.15.67:0/3607476259 shutdown_connections 2024-09-21T06:28:15.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.384+0000 7f3d240d6640 1 --2- 172.21.15.67:0/3607476259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d1c104070 0x7f3d1c10aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:15.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.384+0000 7f3d240d6640 1 --2- 172.21.15.67:0/3607476259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d1c1036d0 0x7f3d1c103ae0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:15.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.384+0000 7f3d240d6640 1 --2- 172.21.15.67:0/3607476259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d1c102640 0x7f3d1c102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:15.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.384+0000 7f3d240d6640 1 -- 172.21.15.67:0/3607476259 >> 172.21.15.67:0/3607476259 conn(0x7f3d1c0fdfb0 msgr2=0x7f3d1c100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:15.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.384+0000 7f3d240d6640 1 -- 172.21.15.67:0/3607476259 shutdown_connections 2024-09-21T06:28:15.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.384+0000 7f3d240d6640 1 -- 172.21.15.67:0/3607476259 wait complete. 2024-09-21T06:28:15.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.385+0000 7f3d240d6640 1 Processor -- start 2024-09-21T06:28:15.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.385+0000 7f3d240d6640 1 -- start start 2024-09-21T06:28:15.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.385+0000 7f3d240d6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d1c102640 0x7f3d1c106230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:15.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.385+0000 7f3d240d6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d1c1036d0 0x7f3d1c104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:15.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.385+0000 7f3d240d6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d1c104070 0x7f3d1c105040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:15.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.385+0000 7f3d240d6640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c10e000 con 0x7f3d1c102640 2024-09-21T06:28:15.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.385+0000 7f3d240d6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c10de80 con 0x7f3d1c1036d0 2024-09-21T06:28:15.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.385+0000 7f3d240d6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c10e180 con 0x7f3d1c104070 2024-09-21T06:28:15.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.385+0000 7f3d21e72640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d1c102640 0x7f3d1c106230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:15.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.385+0000 7f3d21e72640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d1c102640 0x7f3d1c106230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57246/0 (socket says 172.21.15.67:57246) 2024-09-21T06:28:15.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.385+0000 7f3d21e72640 1 -- 172.21.15.67:0/4086538937 learned_addr learned my addr 172.21.15.67:0/4086538937 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:28:15.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.385+0000 7f3d21671640 1 --2- 172.21.15.67:0/4086538937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d1c1036d0 0x7f3d1c104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:15.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.385+0000 7f3d22673640 1 --2- 172.21.15.67:0/4086538937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d1c104070 0x7f3d1c105040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:15.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.386+0000 7f3d21e72640 1 -- 172.21.15.67:0/4086538937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d1c104070 msgr2=0x7f3d1c105040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:15.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.386+0000 7f3d21e72640 1 --2- 172.21.15.67:0/4086538937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d1c104070 0x7f3d1c105040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:15.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.386+0000 7f3d21e72640 1 -- 172.21.15.67:0/4086538937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d1c1036d0 msgr2=0x7f3d1c104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:15.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.386+0000 7f3d21e72640 1 --2- 172.21.15.67:0/4086538937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d1c1036d0 0x7f3d1c104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:15.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.386+0000 7f3d21e72640 1 -- 172.21.15.67:0/4086538937 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d1c105740 con 0x7f3d1c102640 2024-09-21T06:28:15.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.386+0000 7f3d21671640 1 --2- 172.21.15.67:0/4086538937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d1c1036d0 0x7f3d1c104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:15.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.386+0000 7f3d21e72640 1 --2- 172.21.15.67:0/4086538937 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d1c102640 0x7f3d1c106230 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800e9f0 tx=0x7f3d1800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:15.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.386+0000 7f3d12ffd640 1 -- 172.21.15.67:0/4086538937 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1800cd80 con 0x7f3d1c102640 2024-09-21T06:28:15.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.386+0000 7f3d12ffd640 1 -- 172.21.15.67:0/4086538937 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(24 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f3d18005490 con 0x7f3d1c102640 2024-09-21T06:28:15.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.386+0000 7f3d240d6640 1 -- 172.21.15.67:0/4086538937 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d1c1a3eb0 con 0x7f3d1c102640 2024-09-21T06:28:15.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.386+0000 7f3d12ffd640 1 -- 172.21.15.67:0/4086538937 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d18010430 con 0x7f3d1c102640 2024-09-21T06:28:15.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.386+0000 7f3d240d6640 1 -- 172.21.15.67:0/4086538937 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d1c1a4250 con 0x7f3d1c102640 2024-09-21T06:28:15.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.387+0000 7f3d240d6640 1 -- 172.21.15.67:0/4086538937 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d1c10e7c0 con 0x7f3d1c102640 2024-09-21T06:28:15.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.389+0000 7f3d12ffd640 1 -- 172.21.15.67:0/4086538937 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f3d180046c0 con 0x7f3d1c102640 2024-09-21T06:28:15.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.390+0000 7f3d12ffd640 1 --2- 172.21.15.67:0/4086538937 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f3d080b6480 0x7f3d080b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:15.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.390+0000 7f3d21671640 1 --2- 172.21.15.67:0/4086538937 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f3d080b6480 0x7f3d080b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:15.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.390+0000 7f3d12ffd640 1 -- 172.21.15.67:0/4086538937 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3d180e5b30 con 0x7f3d1c102640 2024-09-21T06:28:15.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.391+0000 7f3d21671640 1 --2- 172.21.15.67:0/4086538937 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f3d080b6480 0x7f3d080b8990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c005ab0 tx=0x7f3d0c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:15.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.394+0000 7f3d12ffd640 1 -- 172.21.15.67:0/4086538937 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3d180ae070 con 0x7f3d1c102640 2024-09-21T06:28:15.545 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:15 smithi067 ceph-mon[31232]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:15.545 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:15 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/846890512' entity='client.admin' 2024-09-21T06:28:15.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.586+0000 7f3d240d6640 1 -- 172.21.15.67:0/4086538937 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7f3d1c1a44e0 con 0x7f3d1c102640 2024-09-21T06:28:15.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.591+0000 7f3d12ffd640 1 -- 172.21.15.67:0/4086538937 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v27)=0 v27) ==== 116+0+0 (secure 0 0 0) 0x7f3d180b2450 con 0x7f3d1c102640 2024-09-21T06:28:15.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.593+0000 7f3d240d6640 1 -- 172.21.15.67:0/4086538937 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f3d080b6480 msgr2=0x7f3d080b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:15.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.593+0000 7f3d240d6640 1 --2- 172.21.15.67:0/4086538937 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f3d080b6480 0x7f3d080b8990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c005ab0 tx=0x7f3d0c005a20 comp rx=0 tx=0).stop 2024-09-21T06:28:15.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.593+0000 7f3d240d6640 1 -- 172.21.15.67:0/4086538937 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d1c102640 msgr2=0x7f3d1c106230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:15.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.593+0000 7f3d240d6640 1 --2- 172.21.15.67:0/4086538937 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d1c102640 0x7f3d1c106230 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800e9f0 tx=0x7f3d1800edf0 comp rx=0 tx=0).stop 2024-09-21T06:28:15.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.593+0000 7f3d240d6640 1 -- 172.21.15.67:0/4086538937 shutdown_connections 2024-09-21T06:28:15.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.593+0000 7f3d240d6640 1 --2- 172.21.15.67:0/4086538937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d1c1036d0 0x7f3d1c104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:15.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.593+0000 7f3d240d6640 1 --2- 172.21.15.67:0/4086538937 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f3d080b6480 0x7f3d080b8990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:15.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.593+0000 7f3d240d6640 1 --2- 172.21.15.67:0/4086538937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d1c104070 0x7f3d1c105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:15.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.593+0000 7f3d240d6640 1 --2- 172.21.15.67:0/4086538937 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d1c102640 0x7f3d1c106230 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:15.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.593+0000 7f3d240d6640 1 -- 172.21.15.67:0/4086538937 >> 172.21.15.67:0/4086538937 conn(0x7f3d1c0fdfb0 msgr2=0x7f3d1c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:15.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.593+0000 7f3d240d6640 1 -- 172.21.15.67:0/4086538937 shutdown_connections 2024-09-21T06:28:15.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:15.593+0000 7f3d240d6640 1 -- 172.21.15.67:0/4086538937 wait complete. 2024-09-21T06:28:15.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:15 smithi100 ceph-mon[33401]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:15.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:15 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/846890512' entity='client.admin' 2024-09-21T06:28:15.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:15 smithi203 ceph-mon[33735]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:15.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:15 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/846890512' entity='client.admin' 2024-09-21T06:28:16.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.606+0000 7fda3e84e640 1 -- 172.21.15.67:0/258205137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda381040a0 msgr2=0x7fda3810aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:16.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.606+0000 7fda3e84e640 1 --2- 172.21.15.67:0/258205137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda381040a0 0x7fda3810aa40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fda30009ca0 tx=0x7fda3002a950 comp rx=0 tx=0).stop 2024-09-21T06:28:16.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.606+0000 7fda3e84e640 1 -- 172.21.15.67:0/258205137 shutdown_connections 2024-09-21T06:28:16.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.606+0000 7fda3e84e640 1 --2- 172.21.15.67:0/258205137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda38103700 0x7fda38103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:16.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.606+0000 7fda3e84e640 1 --2- 172.21.15.67:0/258205137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda381040a0 0x7fda3810aa40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:16.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.606+0000 7fda3e84e640 1 --2- 172.21.15.67:0/258205137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda38102670 0x7fda38102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:16.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.606+0000 7fda3e84e640 1 -- 172.21.15.67:0/258205137 >> 172.21.15.67:0/258205137 conn(0x7fda380fdfe0 msgr2=0x7fda38100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:16.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.606+0000 7fda3e84e640 1 -- 172.21.15.67:0/258205137 shutdown_connections 2024-09-21T06:28:16.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.606+0000 7fda3e84e640 1 -- 172.21.15.67:0/258205137 wait complete. 2024-09-21T06:28:16.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.607+0000 7fda3e84e640 1 Processor -- start 2024-09-21T06:28:16.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.607+0000 7fda3e84e640 1 -- start start 2024-09-21T06:28:16.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.607+0000 7fda3e84e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda38102670 0x7fda3806c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:16.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.607+0000 7fda3e84e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda38103700 0x7fda3819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:16.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.607+0000 7fda3e84e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda381040a0 0x7fda381a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:16.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.607+0000 7fda3e84e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fda3810e010 con 0x7fda38102670 2024-09-21T06:28:16.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.607+0000 7fda3e84e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fda3810de90 con 0x7fda38103700 2024-09-21T06:28:16.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.607+0000 7fda3e84e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fda3810e190 con 0x7fda381040a0 2024-09-21T06:28:16.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.607+0000 7fda37fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda38102670 0x7fda3806c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:16.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.607+0000 7fda37fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda38102670 0x7fda3806c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57254/0 (socket says 172.21.15.67:57254) 2024-09-21T06:28:16.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.607+0000 7fda37fff640 1 -- 172.21.15.67:0/4203207658 learned_addr learned my addr 172.21.15.67:0/4203207658 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:28:16.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.608+0000 7fda377fe640 1 --2- 172.21.15.67:0/4203207658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda38103700 0x7fda3819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:16.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.608+0000 7fda3cdeb640 1 --2- 172.21.15.67:0/4203207658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda381040a0 0x7fda381a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:16.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.608+0000 7fda37fff640 1 -- 172.21.15.67:0/4203207658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda381040a0 msgr2=0x7fda381a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:16.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.608+0000 7fda37fff640 1 --2- 172.21.15.67:0/4203207658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda381040a0 0x7fda381a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:16.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.608+0000 7fda37fff640 1 -- 172.21.15.67:0/4203207658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda38103700 msgr2=0x7fda3819eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:16.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.608+0000 7fda37fff640 1 --2- 172.21.15.67:0/4203207658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda38103700 0x7fda3819eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:16.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.608+0000 7fda37fff640 1 -- 172.21.15.67:0/4203207658 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda381a18a0 con 0x7fda38102670 2024-09-21T06:28:16.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.608+0000 7fda3cdeb640 1 --2- 172.21.15.67:0/4203207658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda381040a0 0x7fda381a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:28:16.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.608+0000 7fda37fff640 1 --2- 172.21.15.67:0/4203207658 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda38102670 0x7fda3806c380 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fda2800d950 tx=0x7fda2800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:16.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.608+0000 7fda357fa640 1 -- 172.21.15.67:0/4203207658 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28017030 con 0x7fda38102670 2024-09-21T06:28:16.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.608+0000 7fda357fa640 1 -- 172.21.15.67:0/4203207658 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(24 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fda28004020 con 0x7fda38102670 2024-09-21T06:28:16.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.608+0000 7fda3e84e640 1 -- 172.21.15.67:0/4203207658 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda381a1b90 con 0x7fda38102670 2024-09-21T06:28:16.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.608+0000 7fda357fa640 1 -- 172.21.15.67:0/4203207658 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28005570 con 0x7fda38102670 2024-09-21T06:28:16.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.608+0000 7fda3e84e640 1 -- 172.21.15.67:0/4203207658 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda381a2120 con 0x7fda38102670 2024-09-21T06:28:16.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.609+0000 7fda3e84e640 1 -- 172.21.15.67:0/4203207658 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda38102ae0 con 0x7fda38102670 2024-09-21T06:28:16.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.611+0000 7fda357fa640 1 -- 172.21.15.67:0/4203207658 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7fda28009a30 con 0x7fda38102670 2024-09-21T06:28:16.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.612+0000 7fda357fa640 1 --2- 172.21.15.67:0/4203207658 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fda240b6500 0x7fda240b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:16.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.612+0000 7fda377fe640 1 --2- 172.21.15.67:0/4203207658 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fda240b6500 0x7fda240b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:16.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.612+0000 7fda357fa640 1 -- 172.21.15.67:0/4203207658 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fda280e50c0 con 0x7fda38102670 2024-09-21T06:28:16.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.612+0000 7fda377fe640 1 --2- 172.21.15.67:0/4203207658 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fda240b6500 0x7fda240b8a10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fda20005fd0 tx=0x7fda20005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:16.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.616+0000 7fda357fa640 1 -- 172.21.15.67:0/4203207658 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fda280ad620 con 0x7fda38102670 2024-09-21T06:28:16.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:16 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/4086538937' entity='client.admin' 2024-09-21T06:28:16.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:16 smithi067 ceph-mon[31232]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:16.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.807+0000 7fda3e84e640 1 -- 172.21.15.67:0/4203207658 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fda38063fc0 con 0x7fda38102670 2024-09-21T06:28:16.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.813+0000 7fda357fa640 1 -- 172.21.15.67:0/4203207658 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v28)=0 v28) ==== 123+0+0 (secure 0 0 0) 0x7fda280b1a00 con 0x7fda38102670 2024-09-21T06:28:16.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.815+0000 7fda357fa640 1 -- 172.21.15.67:0/4203207658 <== mon.0 v2:172.21.15.67:3300/0 8 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda280aca30 con 0x7fda38102670 2024-09-21T06:28:16.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.816+0000 7fda3e84e640 1 -- 172.21.15.67:0/4203207658 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fda240b6500 msgr2=0x7fda240b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:16.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.816+0000 7fda3e84e640 1 --2- 172.21.15.67:0/4203207658 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fda240b6500 0x7fda240b8a10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fda20005fd0 tx=0x7fda20005850 comp rx=0 tx=0).stop 2024-09-21T06:28:16.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.816+0000 7fda3e84e640 1 -- 172.21.15.67:0/4203207658 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda38102670 msgr2=0x7fda3806c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:16.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.816+0000 7fda3e84e640 1 --2- 172.21.15.67:0/4203207658 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda38102670 0x7fda3806c380 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fda2800d950 tx=0x7fda2800de10 comp rx=0 tx=0).stop 2024-09-21T06:28:16.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.816+0000 7fda3e84e640 1 -- 172.21.15.67:0/4203207658 shutdown_connections 2024-09-21T06:28:16.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.816+0000 7fda3e84e640 1 --2- 172.21.15.67:0/4203207658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda38103700 0x7fda3819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:16.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.816+0000 7fda3e84e640 1 --2- 172.21.15.67:0/4203207658 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fda240b6500 0x7fda240b8a10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:16.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.816+0000 7fda3e84e640 1 --2- 172.21.15.67:0/4203207658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda381040a0 0x7fda381a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:16.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.816+0000 7fda3e84e640 1 --2- 172.21.15.67:0/4203207658 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda38102670 0x7fda3806c380 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:16.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.816+0000 7fda3e84e640 1 -- 172.21.15.67:0/4203207658 >> 172.21.15.67:0/4203207658 conn(0x7fda380fdfe0 msgr2=0x7fda38106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:16.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.816+0000 7fda3e84e640 1 -- 172.21.15.67:0/4203207658 shutdown_connections 2024-09-21T06:28:16.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.816+0000 7fda3e84e640 1 -- 172.21.15.67:0/4203207658 wait complete. 2024-09-21T06:28:16.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.831+0000 7f56adeab640 1 -- 172.21.15.67:0/376490100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56a806c340 msgr2=0x7f56a81063e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:16.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.831+0000 7f56adeab640 1 --2- 172.21.15.67:0/376490100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56a806c340 0x7f56a81063e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5698009900 tx=0x7f569802a410 comp rx=0 tx=0).stop 2024-09-21T06:28:16.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.832+0000 7f56adeab640 1 -- 172.21.15.67:0/376490100 shutdown_connections 2024-09-21T06:28:16.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.832+0000 7f56adeab640 1 --2- 172.21.15.67:0/376490100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56a8106920 0x7f56a8108d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:16.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.832+0000 7f56adeab640 1 --2- 172.21.15.67:0/376490100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56a81094c0 0x7f56a810b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:16.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.832+0000 7f56adeab640 1 --2- 172.21.15.67:0/376490100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56a806c340 0x7f56a81063e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:16.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.832+0000 7f56adeab640 1 -- 172.21.15.67:0/376490100 >> 172.21.15.67:0/376490100 conn(0x7f56a80fe000 msgr2=0x7f56a81004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:16.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.832+0000 7f56adeab640 1 -- 172.21.15.67:0/376490100 shutdown_connections 2024-09-21T06:28:16.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.832+0000 7f56adeab640 1 -- 172.21.15.67:0/376490100 wait complete. 2024-09-21T06:28:16.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.832+0000 7f56adeab640 1 Processor -- start 2024-09-21T06:28:16.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.832+0000 7f56adeab640 1 -- start start 2024-09-21T06:28:16.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.833+0000 7f56adeab640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56a806c340 0x7f56a819c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:16.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.833+0000 7f56adeab640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56a8106920 0x7f56a819c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:16.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.833+0000 7f56adeab640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56a81094c0 0x7f56a819d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:16.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.833+0000 7f56adeab640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f56a810f260 con 0x7f56a806c340 2024-09-21T06:28:16.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.833+0000 7f56adeab640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f56a810f0e0 con 0x7f56a81094c0 2024-09-21T06:28:16.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.833+0000 7f56adeab640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f56a810f3e0 con 0x7f56a8106920 2024-09-21T06:28:16.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.833+0000 7f56a77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56a806c340 0x7f56a819c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:16.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.833+0000 7f56a77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56a806c340 0x7f56a819c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57278/0 (socket says 172.21.15.67:57278) 2024-09-21T06:28:16.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.833+0000 7f56a77fe640 1 -- 172.21.15.67:0/1723419278 learned_addr learned my addr 172.21.15.67:0/1723419278 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:28:16.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.833+0000 7f56a7fff640 1 --2- 172.21.15.67:0/1723419278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56a81094c0 0x7f56a819d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:16.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.833+0000 7f56a6ffd640 1 --2- 172.21.15.67:0/1723419278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56a8106920 0x7f56a819c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:16.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.833+0000 7f56a7fff640 1 -- 172.21.15.67:0/1723419278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56a8106920 msgr2=0x7f56a819c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:16.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.833+0000 7f56a7fff640 1 --2- 172.21.15.67:0/1723419278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56a8106920 0x7f56a819c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:16.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.833+0000 7f56a7fff640 1 -- 172.21.15.67:0/1723419278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56a806c340 msgr2=0x7f56a819c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:16.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.833+0000 7f56a7fff640 1 --2- 172.21.15.67:0/1723419278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56a806c340 0x7f56a819c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:16.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.833+0000 7f56a7fff640 1 -- 172.21.15.67:0/1723419278 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a819dad0 con 0x7f56a81094c0 2024-09-21T06:28:16.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.833+0000 7f56a77fe640 1 --2- 172.21.15.67:0/1723419278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56a806c340 0x7f56a819c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:28:16.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.834+0000 7f56a7fff640 1 --2- 172.21.15.67:0/1723419278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56a81094c0 0x7f56a819d340 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f56a000cab0 tx=0x7f56a000cf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:16.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.834+0000 7f56a4ff9640 1 -- 172.21.15.67:0/1723419278 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a000ae20 con 0x7f56a81094c0 2024-09-21T06:28:16.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.834+0000 7f56a4ff9640 1 -- 172.21.15.67:0/1723419278 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56a000f8e0 con 0x7f56a81094c0 2024-09-21T06:28:16.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.834+0000 7f56a4ff9640 1 -- 172.21.15.67:0/1723419278 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a00189a0 con 0x7f56a81094c0 2024-09-21T06:28:16.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.834+0000 7f56adeab640 1 -- 172.21.15.67:0/1723419278 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a819ddc0 con 0x7f56a81094c0 2024-09-21T06:28:16.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.835+0000 7f56adeab640 1 -- 172.21.15.67:0/1723419278 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a81a4380 con 0x7f56a81094c0 2024-09-21T06:28:16.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.836+0000 7f56adeab640 1 -- 172.21.15.67:0/1723419278 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56a810b0a0 con 0x7f56a81094c0 2024-09-21T06:28:16.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.837+0000 7f56a4ff9640 1 -- 172.21.15.67:0/1723419278 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f56a000f450 con 0x7f56a81094c0 2024-09-21T06:28:16.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.838+0000 7f56a4ff9640 1 --2- 172.21.15.67:0/1723419278 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f56900b6500 0x7f56900b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:16.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.838+0000 7f56a77fe640 1 --2- 172.21.15.67:0/1723419278 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f56900b6500 0x7f56900b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:16.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.838+0000 7f56a4ff9640 1 -- 172.21.15.67:0/1723419278 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f56a00e5980 con 0x7f56a81094c0 2024-09-21T06:28:16.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.838+0000 7f56a77fe640 1 --2- 172.21.15.67:0/1723419278 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f56900b6500 0x7f56900b8a10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5698005760 tx=0x7f5698031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:16.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:16.841+0000 7f56a4ff9640 1 -- 172.21.15.67:0/1723419278 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f56a0014030 con 0x7f56a81094c0 2024-09-21T06:28:16.883 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:16 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/4086538937' entity='client.admin' 2024-09-21T06:28:16.883 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:16 smithi203 ceph-mon[33735]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:16.970 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:16 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/4086538937' entity='client.admin' 2024-09-21T06:28:16.970 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:16 smithi100 ceph-mon[33401]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:17.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:17.031+0000 7f56adeab640 1 -- 172.21.15.67:0/1723419278 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f56a802e640 con 0x7f56a81094c0 2024-09-21T06:28:17.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:17.039+0000 7f56a4ff9640 1 -- 172.21.15.67:0/1723419278 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v29)=0 v29) ==== 122+0+0 (secure 0 0 0) 0x7f56a00adfd0 con 0x7f56a81094c0 2024-09-21T06:28:17.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:17.041+0000 7f56adeab640 1 -- 172.21.15.67:0/1723419278 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f56900b6500 msgr2=0x7f56900b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:17.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:17.041+0000 7f56adeab640 1 --2- 172.21.15.67:0/1723419278 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f56900b6500 0x7f56900b8a10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5698005760 tx=0x7f5698031400 comp rx=0 tx=0).stop 2024-09-21T06:28:17.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:17.041+0000 7f56adeab640 1 -- 172.21.15.67:0/1723419278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56a81094c0 msgr2=0x7f56a819d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:17.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:17.041+0000 7f56adeab640 1 --2- 172.21.15.67:0/1723419278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56a81094c0 0x7f56a819d340 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f56a000cab0 tx=0x7f56a000cf70 comp rx=0 tx=0).stop 2024-09-21T06:28:17.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:17.041+0000 7f56adeab640 1 -- 172.21.15.67:0/1723419278 shutdown_connections 2024-09-21T06:28:17.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:17.041+0000 7f56adeab640 1 --2- 172.21.15.67:0/1723419278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56a81094c0 0x7f56a819d340 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:17.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:17.041+0000 7f56adeab640 1 --2- 172.21.15.67:0/1723419278 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f56900b6500 0x7f56900b8a10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:17.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:17.041+0000 7f56adeab640 1 --2- 172.21.15.67:0/1723419278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56a8106920 0x7f56a819c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:17.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:17.041+0000 7f56adeab640 1 --2- 172.21.15.67:0/1723419278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56a806c340 0x7f56a819c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:17.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:17.041+0000 7f56adeab640 1 -- 172.21.15.67:0/1723419278 >> 172.21.15.67:0/1723419278 conn(0x7f56a80fe000 msgr2=0x7f56a8104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:17.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:17.041+0000 7f56adeab640 1 -- 172.21.15.67:0/1723419278 shutdown_connections 2024-09-21T06:28:17.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:17.041+0000 7f56adeab640 1 -- 172.21.15.67:0/1723419278 wait complete. 2024-09-21T06:28:17.443 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2a791768-77e2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi067') 2024-09-21T06:28:18.041 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:17 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/4203207658' entity='client.admin' 2024-09-21T06:28:18.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:17 smithi067 ceph-mon[31232]: from='client.? ' entity='client.admin' 2024-09-21T06:28:18.107 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:17 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/4203207658' entity='client.admin' 2024-09-21T06:28:18.107 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:17 smithi203 ceph-mon[33735]: from='client.? ' entity='client.admin' 2024-09-21T06:28:18.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:17 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/4203207658' entity='client.admin' 2024-09-21T06:28:18.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:17 smithi100 ceph-mon[33401]: from='client.? ' entity='client.admin' 2024-09-21T06:28:19.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.019+0000 7fbe2581c640 1 -- 172.21.15.67:0/4225833331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe20109490 msgr2=0x7fbe2010b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:19.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.019+0000 7fbe2581c640 1 --2- 172.21.15.67:0/4225833331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe20109490 0x7fbe2010b9a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbe18009ca0 tx=0x7fbe1802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.019+0000 7fbe2581c640 1 -- 172.21.15.67:0/4225833331 shutdown_connections 2024-09-21T06:28:19.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.019+0000 7fbe2581c640 1 --2- 172.21.15.67:0/4225833331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe20106950 0x7fbe20108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.019+0000 7fbe2581c640 1 --2- 172.21.15.67:0/4225833331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe20109490 0x7fbe2010b9a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.019+0000 7fbe2581c640 1 --2- 172.21.15.67:0/4225833331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe2006c2e0 0x7fbe20106410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.019+0000 7fbe2581c640 1 -- 172.21.15.67:0/4225833331 >> 172.21.15.67:0/4225833331 conn(0x7fbe200fdfd0 msgr2=0x7fbe20100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:19.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.019+0000 7fbe2581c640 1 -- 172.21.15.67:0/4225833331 shutdown_connections 2024-09-21T06:28:19.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.019+0000 7fbe2581c640 1 -- 172.21.15.67:0/4225833331 wait complete. 2024-09-21T06:28:19.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.020+0000 7fbe2581c640 1 Processor -- start 2024-09-21T06:28:19.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.020+0000 7fbe2581c640 1 -- start start 2024-09-21T06:28:19.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.020+0000 7fbe2581c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe2006c2e0 0x7fbe20199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:19.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.020+0000 7fbe2581c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe20106950 0x7fbe2019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:19.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.020+0000 7fbe2581c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe20109490 0x7fbe2019b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:19.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.020+0000 7fbe2581c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2010f280 con 0x7fbe20106950 2024-09-21T06:28:19.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.020+0000 7fbe2581c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2010f100 con 0x7fbe20109490 2024-09-21T06:28:19.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.020+0000 7fbe2581c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2010f400 con 0x7fbe2006c2e0 2024-09-21T06:28:19.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.020+0000 7fbe1e7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe20106950 0x7fbe2019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:19.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.021+0000 7fbe1e7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe20106950 0x7fbe2019a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59450/0 (socket says 172.21.15.67:59450) 2024-09-21T06:28:19.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.021+0000 7fbe1e7fc640 1 -- 172.21.15.67:0/3318939482 learned_addr learned my addr 172.21.15.67:0/3318939482 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:28:19.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.021+0000 7fbe1f7fe640 1 --2- 172.21.15.67:0/3318939482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe20109490 0x7fbe2019b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:19.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.021+0000 7fbe1effd640 1 --2- 172.21.15.67:0/3318939482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe2006c2e0 0x7fbe20199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:19.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.021+0000 7fbe1e7fc640 1 -- 172.21.15.67:0/3318939482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe2006c2e0 msgr2=0x7fbe20199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:19.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.021+0000 7fbe1e7fc640 1 --2- 172.21.15.67:0/3318939482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe2006c2e0 0x7fbe20199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.021+0000 7fbe1e7fc640 1 -- 172.21.15.67:0/3318939482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe20109490 msgr2=0x7fbe2019b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:19.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.021+0000 7fbe1e7fc640 1 --2- 172.21.15.67:0/3318939482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe20109490 0x7fbe2019b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.021+0000 7fbe1e7fc640 1 -- 172.21.15.67:0/3318939482 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2019b9d0 con 0x7fbe20106950 2024-09-21T06:28:19.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.021+0000 7fbe1effd640 1 --2- 172.21.15.67:0/3318939482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe2006c2e0 0x7fbe20199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:19.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.021+0000 7fbe1e7fc640 1 --2- 172.21.15.67:0/3318939482 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe20106950 0x7fbe2019a610 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400e9f0 tx=0x7fbe1400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:19.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.021+0000 7fbe2481a640 1 -- 172.21.15.67:0/3318939482 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1400cd80 con 0x7fbe20106950 2024-09-21T06:28:19.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.021+0000 7fbe2481a640 1 -- 172.21.15.67:0/3318939482 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe140108c0 con 0x7fbe20106950 2024-09-21T06:28:19.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.021+0000 7fbe2481a640 1 -- 172.21.15.67:0/3318939482 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe140187c0 con 0x7fbe20106950 2024-09-21T06:28:19.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.021+0000 7fbe2581c640 1 -- 172.21.15.67:0/3318939482 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe2019bcc0 con 0x7fbe20106950 2024-09-21T06:28:19.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.021+0000 7fbe2581c640 1 -- 172.21.15.67:0/3318939482 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe201a23c0 con 0x7fbe20106950 2024-09-21T06:28:19.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.022+0000 7fbe2581c640 1 -- 172.21.15.67:0/3318939482 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe20106d60 con 0x7fbe20106950 2024-09-21T06:28:19.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.024+0000 7fbe2481a640 1 -- 172.21.15.67:0/3318939482 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7fbe14010430 con 0x7fbe20106950 2024-09-21T06:28:19.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.025+0000 7fbe2481a640 1 --2- 172.21.15.67:0/3318939482 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fbe080b6500 0x7fbe080b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:19.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.025+0000 7fbe1effd640 1 --2- 172.21.15.67:0/3318939482 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fbe080b6500 0x7fbe080b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:19.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.025+0000 7fbe2481a640 1 -- 172.21.15.67:0/3318939482 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbe140e5e10 con 0x7fbe20106950 2024-09-21T06:28:19.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.026+0000 7fbe1effd640 1 --2- 172.21.15.67:0/3318939482 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fbe080b6500 0x7fbe080b8a10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbe100058c0 tx=0x7fbe10005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:19.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.032+0000 7fbe2481a640 1 -- 172.21.15.67:0/3318939482 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbe140ae370 con 0x7fbe20106950 2024-09-21T06:28:19.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.925+0000 7f79d8922640 1 -- 172.21.15.67:0/3639180521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79d4101b10 msgr2=0x7f79d4101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:19.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.925+0000 7f79d8922640 1 --2- 172.21.15.67:0/3639180521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79d4101b10 0x7f79d4101f20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f79c4009900 tx=0x7f79c402a410 comp rx=0 tx=0).stop 2024-09-21T06:28:19.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.925+0000 7f79d8922640 1 -- 172.21.15.67:0/3639180521 shutdown_connections 2024-09-21T06:28:19.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.925+0000 7f79d8922640 1 --2- 172.21.15.67:0/3639180521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79d4109510 0x7f79d410b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.925+0000 7f79d8922640 1 --2- 172.21.15.67:0/3639180521 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79d4106bb0 0x7f79d4108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.925+0000 7f79d8922640 1 --2- 172.21.15.67:0/3639180521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79d4101b10 0x7f79d4101f20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.925+0000 7f79d8922640 1 -- 172.21.15.67:0/3639180521 >> 172.21.15.67:0/3639180521 conn(0x7f79d40fd740 msgr2=0x7f79d40ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:19.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.925+0000 7f79d8922640 1 -- 172.21.15.67:0/3639180521 shutdown_connections 2024-09-21T06:28:19.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.925+0000 7f79d8922640 1 -- 172.21.15.67:0/3639180521 wait complete. 2024-09-21T06:28:19.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.926+0000 7f79d8922640 1 Processor -- start 2024-09-21T06:28:19.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.926+0000 7f79d8922640 1 -- start start 2024-09-21T06:28:19.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.926+0000 7f79d8922640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79d4101b10 0x7f79d419c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:19.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.926+0000 7f79d8922640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79d4106bb0 0x7f79d419c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:19.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.926+0000 7f79d8922640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79d4109510 0x7f79d419d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:19.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.926+0000 7f79d8922640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f79d410f220 con 0x7f79d4101b10 2024-09-21T06:28:19.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.926+0000 7f79d8922640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f79d410f0a0 con 0x7f79d4106bb0 2024-09-21T06:28:19.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.926+0000 7f79d8922640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f79d410f3a0 con 0x7f79d4109510 2024-09-21T06:28:19.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.926+0000 7f79d259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79d4101b10 0x7f79d419c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:19.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.926+0000 7f79d259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79d4101b10 0x7f79d419c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59428/0 (socket says 172.21.15.67:59428) 2024-09-21T06:28:19.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.926+0000 7f79d259c640 1 -- 172.21.15.67:0/1207877845 learned_addr learned my addr 172.21.15.67:0/1207877845 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:28:19.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.926+0000 7f79d1d9b640 1 --2- 172.21.15.67:0/1207877845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79d4106bb0 0x7f79d419c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:19.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.926+0000 7f79d2d9d640 1 --2- 172.21.15.67:0/1207877845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79d4109510 0x7f79d419d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:19.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.927+0000 7f79d259c640 1 -- 172.21.15.67:0/1207877845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79d4109510 msgr2=0x7f79d419d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:19.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.927+0000 7f79d259c640 1 --2- 172.21.15.67:0/1207877845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79d4109510 0x7f79d419d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.927+0000 7f79d259c640 1 -- 172.21.15.67:0/1207877845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79d4106bb0 msgr2=0x7f79d419c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:19.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.927+0000 7f79d259c640 1 --2- 172.21.15.67:0/1207877845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79d4106bb0 0x7f79d419c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.927+0000 7f79d259c640 1 -- 172.21.15.67:0/1207877845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d419db20 con 0x7f79d4101b10 2024-09-21T06:28:19.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.927+0000 7f79d1d9b640 1 --2- 172.21.15.67:0/1207877845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79d4106bb0 0x7f79d419c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:28:19.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.927+0000 7f79d259c640 1 --2- 172.21.15.67:0/1207877845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79d4101b10 0x7f79d419c0f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f79c4009230 tx=0x7f79c40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:19.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.927+0000 7f79c37fe640 1 -- 172.21.15.67:0/1207877845 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c403f030 con 0x7f79d4101b10 2024-09-21T06:28:19.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.927+0000 7f79c37fe640 1 -- 172.21.15.67:0/1207877845 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79c4003c90 con 0x7f79d4101b10 2024-09-21T06:28:19.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.927+0000 7f79d8922640 1 -- 172.21.15.67:0/1207877845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79d419ddb0 con 0x7f79d4101b10 2024-09-21T06:28:19.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.927+0000 7f79c37fe640 1 -- 172.21.15.67:0/1207877845 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c4035070 con 0x7f79d4101b10 2024-09-21T06:28:19.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.927+0000 7f79d8922640 1 -- 172.21.15.67:0/1207877845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79d41a4340 con 0x7f79d4101b10 2024-09-21T06:28:19.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.929+0000 7f79d8922640 1 -- 172.21.15.67:0/1207877845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79d4101f20 con 0x7f79d4101b10 2024-09-21T06:28:19.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.930+0000 7f79c37fe640 1 -- 172.21.15.67:0/1207877845 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f79c4042030 con 0x7f79d4101b10 2024-09-21T06:28:19.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.931+0000 7f79c37fe640 1 --2- 172.21.15.67:0/1207877845 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f79bc0b6480 0x7f79bc0b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:19.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.931+0000 7f79d1d9b640 1 --2- 172.21.15.67:0/1207877845 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f79bc0b6480 0x7f79bc0b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:19.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.931+0000 7f79c37fe640 1 -- 172.21.15.67:0/1207877845 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f79c4105fa0 con 0x7f79d4101b10 2024-09-21T06:28:19.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.932+0000 7f79d1d9b640 1 --2- 172.21.15.67:0/1207877845 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f79bc0b6480 0x7f79bc0b8990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f79c800bfd0 tx=0x7f79c800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:19.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:18.934+0000 7f79c37fe640 1 -- 172.21.15.67:0/1207877845 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f79c40d28e0 con 0x7f79d4101b10 2024-09-21T06:28:19.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.133+0000 7f79d8922640 1 -- 172.21.15.67:0/1207877845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f79d41a45a0 con 0x7f79d4101b10 2024-09-21T06:28:19.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.140+0000 7f79c37fe640 1 -- 172.21.15.67:0/1207877845 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v30)=0 v30) ==== 119+0+0 (secure 0 0 0) 0x7f79c40d7820 con 0x7f79d4101b10 2024-09-21T06:28:19.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.143+0000 7f79d8922640 1 -- 172.21.15.67:0/1207877845 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f79bc0b6480 msgr2=0x7f79bc0b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:19.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.143+0000 7f79d8922640 1 --2- 172.21.15.67:0/1207877845 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f79bc0b6480 0x7f79bc0b8990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f79c800bfd0 tx=0x7f79c800b390 comp rx=0 tx=0).stop 2024-09-21T06:28:19.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.143+0000 7f79d8922640 1 -- 172.21.15.67:0/1207877845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79d4101b10 msgr2=0x7f79d419c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:19.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.143+0000 7f79d8922640 1 --2- 172.21.15.67:0/1207877845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79d4101b10 0x7f79d419c0f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f79c4009230 tx=0x7f79c40042f0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.143+0000 7f79d8922640 1 -- 172.21.15.67:0/1207877845 shutdown_connections 2024-09-21T06:28:19.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.143+0000 7f79d8922640 1 --2- 172.21.15.67:0/1207877845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79d4106bb0 0x7f79d419c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.143+0000 7f79d8922640 1 --2- 172.21.15.67:0/1207877845 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f79bc0b6480 0x7f79bc0b8990 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.143+0000 7f79d8922640 1 --2- 172.21.15.67:0/1207877845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79d4109510 0x7f79d419d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.143+0000 7f79d8922640 1 --2- 172.21.15.67:0/1207877845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79d4101b10 0x7f79d419c0f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.143+0000 7f79d8922640 1 -- 172.21.15.67:0/1207877845 >> 172.21.15.67:0/1207877845 conn(0x7f79d40fd740 msgr2=0x7f79d4103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:19.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.143+0000 7f79d8922640 1 -- 172.21.15.67:0/1207877845 shutdown_connections 2024-09-21T06:28:19.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.143+0000 7f79d8922640 1 -- 172.21.15.67:0/1207877845 wait complete. 2024-09-21T06:28:19.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.228+0000 7fbe2581c640 1 -- 172.21.15.67:0/3318939482 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fbe200ffaf0 con 0x7fbe20106950 2024-09-21T06:28:19.230 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:19 smithi067 ceph-mon[31232]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:19.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.234+0000 7fbe2481a640 1 -- 172.21.15.67:0/3318939482 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v31)=0 v31) ==== 121+0+0 (secure 0 0 0) 0x7fbe140b2750 con 0x7fbe20106950 2024-09-21T06:28:19.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.237+0000 7fbe2581c640 1 -- 172.21.15.67:0/3318939482 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fbe080b6500 msgr2=0x7fbe080b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:19.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.237+0000 7fbe2581c640 1 --2- 172.21.15.67:0/3318939482 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fbe080b6500 0x7fbe080b8a10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbe100058c0 tx=0x7fbe10005850 comp rx=0 tx=0).stop 2024-09-21T06:28:19.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.237+0000 7fbe2581c640 1 -- 172.21.15.67:0/3318939482 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe20106950 msgr2=0x7fbe2019a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:19.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.237+0000 7fbe2581c640 1 --2- 172.21.15.67:0/3318939482 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe20106950 0x7fbe2019a610 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400e9f0 tx=0x7fbe1400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.237+0000 7fbe2581c640 1 -- 172.21.15.67:0/3318939482 shutdown_connections 2024-09-21T06:28:19.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.237+0000 7fbe2581c640 1 --2- 172.21.15.67:0/3318939482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe20109490 0x7fbe2019b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.237+0000 7fbe2581c640 1 --2- 172.21.15.67:0/3318939482 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fbe080b6500 0x7fbe080b8a10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.237+0000 7fbe2581c640 1 --2- 172.21.15.67:0/3318939482 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe20106950 0x7fbe2019a610 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.237+0000 7fbe2581c640 1 --2- 172.21.15.67:0/3318939482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe2006c2e0 0x7fbe20199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:19.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.237+0000 7fbe2581c640 1 -- 172.21.15.67:0/3318939482 >> 172.21.15.67:0/3318939482 conn(0x7fbe200fdfd0 msgr2=0x7fbe20104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:19.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.237+0000 7fbe2581c640 1 -- 172.21.15.67:0/3318939482 shutdown_connections 2024-09-21T06:28:19.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:19.237+0000 7fbe2581c640 1 -- 172.21.15.67:0/3318939482 wait complete. 2024-09-21T06:28:19.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:19 smithi100 ceph-mon[33401]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:19.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:19 smithi203 ceph-mon[33735]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:19.910 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2a791768-77e2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi067') 2024-09-21T06:28:20.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.343+0000 7fcd62a47640 1 -- 172.21.15.67:0/2779722041 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd5c103760 msgr2=0x7fcd5c103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:20.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.343+0000 7fcd62a47640 1 --2- 172.21.15.67:0/2779722041 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd5c103760 0x7fcd5c103bd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c009900 tx=0x7fcd4c02a480 comp rx=0 tx=0).stop 2024-09-21T06:28:20.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.343+0000 7fcd62a47640 1 -- 172.21.15.67:0/2779722041 shutdown_connections 2024-09-21T06:28:20.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.343+0000 7fcd62a47640 1 --2- 172.21.15.67:0/2779722041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd5c1047f0 0x7fcd5c104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.343+0000 7fcd62a47640 1 --2- 172.21.15.67:0/2779722041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd5c105190 0x7fcd5c10bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.343+0000 7fcd62a47640 1 --2- 172.21.15.67:0/2779722041 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd5c103760 0x7fcd5c103bd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.343+0000 7fcd62a47640 1 -- 172.21.15.67:0/2779722041 >> 172.21.15.67:0/2779722041 conn(0x7fcd5c0ff0b0 msgr2=0x7fcd5c101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:20.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.343+0000 7fcd62a47640 1 -- 172.21.15.67:0/2779722041 shutdown_connections 2024-09-21T06:28:20.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.343+0000 7fcd62a47640 1 -- 172.21.15.67:0/2779722041 wait complete. 2024-09-21T06:28:20.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.344+0000 7fcd62a47640 1 Processor -- start 2024-09-21T06:28:20.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.344+0000 7fcd62a47640 1 -- start start 2024-09-21T06:28:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.345+0000 7fcd62a47640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd5c103760 0x7fcd5c199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.345+0000 7fcd62a47640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd5c1047f0 0x7fcd5c19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.345+0000 7fcd62a47640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd5c105190 0x7fcd5c19b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.345+0000 7fcd62a47640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c10f230 con 0x7fcd5c1047f0 2024-09-21T06:28:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.345+0000 7fcd62a47640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c10f0b0 con 0x7fcd5c105190 2024-09-21T06:28:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.345+0000 7fcd62a47640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c10f3b0 con 0x7fcd5c103760 2024-09-21T06:28:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.345+0000 7fcd53fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd5c1047f0 0x7fcd5c19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.345+0000 7fcd53fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd5c1047f0 0x7fcd5c19a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59474/0 (socket says 172.21.15.67:59474) 2024-09-21T06:28:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.345+0000 7fcd53fff640 1 -- 172.21.15.67:0/1777935910 learned_addr learned my addr 172.21.15.67:0/1777935910 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:28:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.345+0000 7fcd5bfff640 1 --2- 172.21.15.67:0/1777935910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd5c103760 0x7fcd5c199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.345+0000 7fcd60fe4640 1 --2- 172.21.15.67:0/1777935910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd5c105190 0x7fcd5c19b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.345+0000 7fcd53fff640 1 -- 172.21.15.67:0/1777935910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd5c103760 msgr2=0x7fcd5c199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.345+0000 7fcd53fff640 1 --2- 172.21.15.67:0/1777935910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd5c103760 0x7fcd5c199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.345+0000 7fcd53fff640 1 -- 172.21.15.67:0/1777935910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd5c105190 msgr2=0x7fcd5c19b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:20.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.345+0000 7fcd53fff640 1 --2- 172.21.15.67:0/1777935910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd5c105190 0x7fcd5c19b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.345+0000 7fcd53fff640 1 -- 172.21.15.67:0/1777935910 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5c19b910 con 0x7fcd5c1047f0 2024-09-21T06:28:20.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.345+0000 7fcd60fe4640 1 --2- 172.21.15.67:0/1777935910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd5c105190 0x7fcd5c19b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:28:20.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.345+0000 7fcd53fff640 1 --2- 172.21.15.67:0/1777935910 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd5c1047f0 0x7fcd5c19a550 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400d950 tx=0x7fcd4400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:20.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.346+0000 7fcd59ffb640 1 -- 172.21.15.67:0/1777935910 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd44017070 con 0x7fcd5c1047f0 2024-09-21T06:28:20.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.346+0000 7fcd62a47640 1 -- 172.21.15.67:0/1777935910 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd5c19bc00 con 0x7fcd5c1047f0 2024-09-21T06:28:20.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.346+0000 7fcd59ffb640 1 -- 172.21.15.67:0/1777935910 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd44009c30 con 0x7fcd5c1047f0 2024-09-21T06:28:20.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.346+0000 7fcd59ffb640 1 -- 172.21.15.67:0/1777935910 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4400f620 con 0x7fcd5c1047f0 2024-09-21T06:28:20.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.346+0000 7fcd62a47640 1 -- 172.21.15.67:0/1777935910 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd5c1a2300 con 0x7fcd5c1047f0 2024-09-21T06:28:20.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.347+0000 7fcd62a47640 1 -- 172.21.15.67:0/1777935910 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd5c10a940 con 0x7fcd5c1047f0 2024-09-21T06:28:20.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.348+0000 7fcd59ffb640 1 -- 172.21.15.67:0/1777935910 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7fcd44010460 con 0x7fcd5c1047f0 2024-09-21T06:28:20.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.349+0000 7fcd59ffb640 1 --2- 172.21.15.67:0/1777935910 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fcd480b6480 0x7fcd480b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:20.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.349+0000 7fcd5bfff640 1 --2- 172.21.15.67:0/1777935910 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fcd480b6480 0x7fcd480b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:20.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.349+0000 7fcd59ffb640 1 -- 172.21.15.67:0/1777935910 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcd440e52f0 con 0x7fcd5c1047f0 2024-09-21T06:28:20.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.350+0000 7fcd5bfff640 1 --2- 172.21.15.67:0/1777935910 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fcd480b6480 0x7fcd480b8990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c000c00 tx=0x7fcd4c005720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:20.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.354+0000 7fcd59ffb640 1 -- 172.21.15.67:0/1777935910 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcd440ad850 con 0x7fcd5c1047f0 2024-09-21T06:28:20.426 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:20 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1207877845' entity='client.admin' 2024-09-21T06:28:20.427 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:20 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3318939482' entity='client.admin' 2024-09-21T06:28:20.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:20 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1207877845' entity='client.admin' 2024-09-21T06:28:20.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:20 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3318939482' entity='client.admin' 2024-09-21T06:28:20.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.522+0000 7fb063d2b640 1 -- 172.21.15.67:0/4008943833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb05c1094a0 msgr2=0x7fb05c10b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:20.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.522+0000 7fb063d2b640 1 --2- 172.21.15.67:0/4008943833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb05c1094a0 0x7fb05c10b9b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb054009ca0 tx=0x7fb05402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.522+0000 7fb063d2b640 1 -- 172.21.15.67:0/4008943833 shutdown_connections 2024-09-21T06:28:20.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.522+0000 7fb063d2b640 1 --2- 172.21.15.67:0/4008943833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb05c106900 0x7fb05c108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.522+0000 7fb063d2b640 1 --2- 172.21.15.67:0/4008943833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb05c1094a0 0x7fb05c10b9b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.522+0000 7fb063d2b640 1 --2- 172.21.15.67:0/4008943833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb05c06c340 0x7fb05c1063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.522+0000 7fb063d2b640 1 -- 172.21.15.67:0/4008943833 >> 172.21.15.67:0/4008943833 conn(0x7fb05c0fdfe0 msgr2=0x7fb05c100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:20.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.522+0000 7fb063d2b640 1 -- 172.21.15.67:0/4008943833 shutdown_connections 2024-09-21T06:28:20.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.522+0000 7fb063d2b640 1 -- 172.21.15.67:0/4008943833 wait complete. 2024-09-21T06:28:20.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.523+0000 7fb063d2b640 1 Processor -- start 2024-09-21T06:28:20.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.523+0000 7fb063d2b640 1 -- start start 2024-09-21T06:28:20.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.523+0000 7fb063d2b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb05c06c340 0x7fb05c199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:20.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.523+0000 7fb063d2b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb05c106900 0x7fb05c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:20.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.523+0000 7fb063d2b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb05c1094a0 0x7fb05c19b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:20.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.523+0000 7fb063d2b640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb05c10f2c0 con 0x7fb05c106900 2024-09-21T06:28:20.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.523+0000 7fb063d2b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb05c10f140 con 0x7fb05c1094a0 2024-09-21T06:28:20.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.523+0000 7fb063d2b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb05c10f440 con 0x7fb05c06c340 2024-09-21T06:28:20.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.523+0000 7fb0612c6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb05c106900 0x7fb05c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:20.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.523+0000 7fb0612c6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb05c106900 0x7fb05c19a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59496/0 (socket says 172.21.15.67:59496) 2024-09-21T06:28:20.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.523+0000 7fb0612c6640 1 -- 172.21.15.67:0/1086779903 learned_addr learned my addr 172.21.15.67:0/1086779903 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:28:20.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.523+0000 7fb061ac7640 1 --2- 172.21.15.67:0/1086779903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb05c06c340 0x7fb05c199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:20.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.524+0000 7fb0622c8640 1 --2- 172.21.15.67:0/1086779903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb05c1094a0 0x7fb05c19b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:20.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.524+0000 7fb0612c6640 1 -- 172.21.15.67:0/1086779903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb05c06c340 msgr2=0x7fb05c199e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:20.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.524+0000 7fb0612c6640 1 --2- 172.21.15.67:0/1086779903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb05c06c340 0x7fb05c199e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.524+0000 7fb0612c6640 1 -- 172.21.15.67:0/1086779903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb05c1094a0 msgr2=0x7fb05c19b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:20.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.524+0000 7fb0612c6640 1 --2- 172.21.15.67:0/1086779903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb05c1094a0 0x7fb05c19b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.524+0000 7fb0612c6640 1 -- 172.21.15.67:0/1086779903 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05c19b880 con 0x7fb05c106900 2024-09-21T06:28:20.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.524+0000 7fb0622c8640 1 --2- 172.21.15.67:0/1086779903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb05c1094a0 0x7fb05c19b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:20.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.524+0000 7fb0612c6640 1 --2- 172.21.15.67:0/1086779903 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb05c106900 0x7fb05c19a4c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00e9f0 tx=0x7fb04c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:20.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.524+0000 7fb052ffd640 1 -- 172.21.15.67:0/1086779903 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c00cd80 con 0x7fb05c106900 2024-09-21T06:28:20.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.524+0000 7fb063d2b640 1 -- 172.21.15.67:0/1086779903 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb05c19bb70 con 0x7fb05c106900 2024-09-21T06:28:20.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.524+0000 7fb052ffd640 1 -- 172.21.15.67:0/1086779903 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb04c0108c0 con 0x7fb05c106900 2024-09-21T06:28:20.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.524+0000 7fb052ffd640 1 -- 172.21.15.67:0/1086779903 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c0187c0 con 0x7fb05c106900 2024-09-21T06:28:20.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.524+0000 7fb063d2b640 1 -- 172.21.15.67:0/1086779903 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb05c1a2270 con 0x7fb05c106900 2024-09-21T06:28:20.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.525+0000 7fb063d2b640 1 -- 172.21.15.67:0/1086779903 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb05c0ffb00 con 0x7fb05c106900 2024-09-21T06:28:20.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.527+0000 7fb052ffd640 1 -- 172.21.15.67:0/1086779903 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7fb04c010430 con 0x7fb05c106900 2024-09-21T06:28:20.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.528+0000 7fb052ffd640 1 --2- 172.21.15.67:0/1086779903 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb0480b6500 0x7fb0480b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:20.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.528+0000 7fb061ac7640 1 --2- 172.21.15.67:0/1086779903 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb0480b6500 0x7fb0480b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:20.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.528+0000 7fb052ffd640 1 -- 172.21.15.67:0/1086779903 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb04c0e5e70 con 0x7fb05c106900 2024-09-21T06:28:20.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.528+0000 7fb061ac7640 1 --2- 172.21.15.67:0/1086779903 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb0480b6500 0x7fb0480b8a10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb058009bd0 tx=0x7fb058005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:20.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.531+0000 7fb052ffd640 1 -- 172.21.15.67:0/1086779903 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb04c0ae3d0 con 0x7fb05c106900 2024-09-21T06:28:20.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.548+0000 7fcd62a47640 1 -- 172.21.15.67:0/1777935910 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7fcd5c063fc0 con 0x7fcd5c1047f0 2024-09-21T06:28:20.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.549+0000 7fcd59ffb640 1 -- 172.21.15.67:0/1777935910 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v31)=0 v31) ==== 106+0+0 (secure 0 0 0) 0x7fcd44009de0 con 0x7fcd5c1047f0 2024-09-21T06:28:20.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.551+0000 7fcd62a47640 1 -- 172.21.15.67:0/1777935910 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fcd480b6480 msgr2=0x7fcd480b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:20.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.551+0000 7fcd62a47640 1 --2- 172.21.15.67:0/1777935910 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fcd480b6480 0x7fcd480b8990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c000c00 tx=0x7fcd4c005720 comp rx=0 tx=0).stop 2024-09-21T06:28:20.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.551+0000 7fcd62a47640 1 -- 172.21.15.67:0/1777935910 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd5c1047f0 msgr2=0x7fcd5c19a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:20.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.551+0000 7fcd62a47640 1 --2- 172.21.15.67:0/1777935910 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd5c1047f0 0x7fcd5c19a550 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400d950 tx=0x7fcd4400de10 comp rx=0 tx=0).stop 2024-09-21T06:28:20.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.551+0000 7fcd62a47640 1 -- 172.21.15.67:0/1777935910 shutdown_connections 2024-09-21T06:28:20.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.551+0000 7fcd62a47640 1 --2- 172.21.15.67:0/1777935910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd5c105190 0x7fcd5c19b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.551+0000 7fcd62a47640 1 --2- 172.21.15.67:0/1777935910 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fcd480b6480 0x7fcd480b8990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.551+0000 7fcd62a47640 1 --2- 172.21.15.67:0/1777935910 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd5c1047f0 0x7fcd5c19a550 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.551+0000 7fcd62a47640 1 --2- 172.21.15.67:0/1777935910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd5c103760 0x7fcd5c199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.551+0000 7fcd62a47640 1 -- 172.21.15.67:0/1777935910 >> 172.21.15.67:0/1777935910 conn(0x7fcd5c0ff0b0 msgr2=0x7fcd5c107b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:20.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.551+0000 7fcd62a47640 1 -- 172.21.15.67:0/1777935910 shutdown_connections 2024-09-21T06:28:20.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.551+0000 7fcd62a47640 1 -- 172.21.15.67:0/1777935910 wait complete. 2024-09-21T06:28:20.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:20 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1207877845' entity='client.admin' 2024-09-21T06:28:20.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:20 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3318939482' entity='client.admin' 2024-09-21T06:28:20.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.732+0000 7fb063d2b640 1 -- 172.21.15.67:0/1086779903 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7fb05c02e640 con 0x7fb05c106900 2024-09-21T06:28:20.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.738+0000 7fb052ffd640 1 -- 172.21.15.67:0/1086779903 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v32)=0 v32) ==== 108+0+0 (secure 0 0 0) 0x7fb04c0b27b0 con 0x7fb05c106900 2024-09-21T06:28:20.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.741+0000 7fb063d2b640 1 -- 172.21.15.67:0/1086779903 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb0480b6500 msgr2=0x7fb0480b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:20.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.741+0000 7fb063d2b640 1 --2- 172.21.15.67:0/1086779903 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb0480b6500 0x7fb0480b8a10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb058009bd0 tx=0x7fb058005f90 comp rx=0 tx=0).stop 2024-09-21T06:28:20.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.741+0000 7fb063d2b640 1 -- 172.21.15.67:0/1086779903 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb05c106900 msgr2=0x7fb05c19a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:20.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.741+0000 7fb063d2b640 1 --2- 172.21.15.67:0/1086779903 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb05c106900 0x7fb05c19a4c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00e9f0 tx=0x7fb04c00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.741+0000 7fb063d2b640 1 -- 172.21.15.67:0/1086779903 shutdown_connections 2024-09-21T06:28:20.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.741+0000 7fb063d2b640 1 --2- 172.21.15.67:0/1086779903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb05c1094a0 0x7fb05c19b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.741+0000 7fb063d2b640 1 --2- 172.21.15.67:0/1086779903 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb0480b6500 0x7fb0480b8a10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.741+0000 7fb063d2b640 1 --2- 172.21.15.67:0/1086779903 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb05c106900 0x7fb05c19a4c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.741+0000 7fb063d2b640 1 --2- 172.21.15.67:0/1086779903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb05c06c340 0x7fb05c199e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:20.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.741+0000 7fb063d2b640 1 -- 172.21.15.67:0/1086779903 >> 172.21.15.67:0/1086779903 conn(0x7fb05c0fdfe0 msgr2=0x7fb05c104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:20.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.741+0000 7fb063d2b640 1 -- 172.21.15.67:0/1086779903 shutdown_connections 2024-09-21T06:28:20.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:20.741+0000 7fb063d2b640 1 -- 172.21.15.67:0/1086779903 wait complete. 2024-09-21T06:28:21.382 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:21 smithi067 ceph-mon[31232]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:21.383 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:21 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1086779903' entity='client.admin' 2024-09-21T06:28:21.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:21 smithi100 ceph-mon[33401]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:21.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:21 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1086779903' entity='client.admin' 2024-09-21T06:28:21.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:21 smithi203 ceph-mon[33735]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:21.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:21 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1086779903' entity='client.admin' 2024-09-21T06:28:22.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.675+0000 7f93a07f8640 1 -- 172.21.15.67:0/1869943665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93981040c0 msgr2=0x7f939810aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:22.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.675+0000 7f93a07f8640 1 --2- 172.21.15.67:0/1869943665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93981040c0 0x7f939810aa60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9390009ca0 tx=0x7f939002a950 comp rx=0 tx=0).stop 2024-09-21T06:28:22.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.675+0000 7f93a07f8640 1 -- 172.21.15.67:0/1869943665 shutdown_connections 2024-09-21T06:28:22.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.675+0000 7f93a07f8640 1 --2- 172.21.15.67:0/1869943665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9398103720 0x7f9398103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:22.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.675+0000 7f93a07f8640 1 --2- 172.21.15.67:0/1869943665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93981040c0 0x7f939810aa60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:22.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.675+0000 7f93a07f8640 1 --2- 172.21.15.67:0/1869943665 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9398102690 0x7f9398102b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:22.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.675+0000 7f93a07f8640 1 -- 172.21.15.67:0/1869943665 >> 172.21.15.67:0/1869943665 conn(0x7f93980fe000 msgr2=0x7f93981004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:22.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.675+0000 7f93a07f8640 1 -- 172.21.15.67:0/1869943665 shutdown_connections 2024-09-21T06:28:22.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.675+0000 7f93a07f8640 1 -- 172.21.15.67:0/1869943665 wait complete. 2024-09-21T06:28:22.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.676+0000 7f93a07f8640 1 Processor -- start 2024-09-21T06:28:22.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.676+0000 7f93a07f8640 1 -- start start 2024-09-21T06:28:22.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.676+0000 7f93a07f8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9398102690 0x7f9398106280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:22.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.676+0000 7f93a07f8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9398103720 0x7f9398104b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:22.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.676+0000 7f93a07f8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93981040c0 0x7f9398105090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:22.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.676+0000 7f93a07f8640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f939810e170 con 0x7f9398103720 2024-09-21T06:28:22.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.676+0000 7f93a07f8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f939810dff0 con 0x7f93981040c0 2024-09-21T06:28:22.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.676+0000 7f93a07f8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f939810e2f0 con 0x7f9398102690 2024-09-21T06:28:22.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.676+0000 7f939dd93640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9398103720 0x7f9398104b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:22.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.676+0000 7f939dd93640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9398103720 0x7f9398104b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59524/0 (socket says 172.21.15.67:59524) 2024-09-21T06:28:22.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.676+0000 7f939dd93640 1 -- 172.21.15.67:0/4129252559 learned_addr learned my addr 172.21.15.67:0/4129252559 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:28:22.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.676+0000 7f939ed95640 1 --2- 172.21.15.67:0/4129252559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93981040c0 0x7f9398105090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:22.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.677+0000 7f939e594640 1 --2- 172.21.15.67:0/4129252559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9398102690 0x7f9398106280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:22.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.677+0000 7f939dd93640 1 -- 172.21.15.67:0/4129252559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9398102690 msgr2=0x7f9398106280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:22.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.677+0000 7f939dd93640 1 --2- 172.21.15.67:0/4129252559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9398102690 0x7f9398106280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:22.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.677+0000 7f939dd93640 1 -- 172.21.15.67:0/4129252559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93981040c0 msgr2=0x7f9398105090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:22.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.677+0000 7f939dd93640 1 --2- 172.21.15.67:0/4129252559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93981040c0 0x7f9398105090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:22.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.677+0000 7f939dd93640 1 -- 172.21.15.67:0/4129252559 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9398105790 con 0x7f9398103720 2024-09-21T06:28:22.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.677+0000 7f939e594640 1 --2- 172.21.15.67:0/4129252559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9398102690 0x7f9398106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:22.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.677+0000 7f939dd93640 1 --2- 172.21.15.67:0/4129252559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9398103720 0x7f9398104b50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f938800d950 tx=0x7f938800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:22.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.677+0000 7f938f7fe640 1 -- 172.21.15.67:0/4129252559 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388017070 con 0x7f9398103720 2024-09-21T06:28:22.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.677+0000 7f938f7fe640 1 -- 172.21.15.67:0/4129252559 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9388009c30 con 0x7f9398103720 2024-09-21T06:28:22.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.677+0000 7f93a07f8640 1 -- 172.21.15.67:0/4129252559 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93981a3f50 con 0x7f9398103720 2024-09-21T06:28:22.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.677+0000 7f938f7fe640 1 -- 172.21.15.67:0/4129252559 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938800f620 con 0x7f9398103720 2024-09-21T06:28:22.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.677+0000 7f93a07f8640 1 -- 172.21.15.67:0/4129252559 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93981a42f0 con 0x7f9398103720 2024-09-21T06:28:22.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.678+0000 7f93a07f8640 1 -- 172.21.15.67:0/4129252559 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9398102b00 con 0x7f9398103720 2024-09-21T06:28:22.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.680+0000 7f938f7fe640 1 -- 172.21.15.67:0/4129252559 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f9388010460 con 0x7f9398103720 2024-09-21T06:28:22.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.681+0000 7f938f7fe640 1 --2- 172.21.15.67:0/4129252559 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f93840b6480 0x7f93840b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:22.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.681+0000 7f939e594640 1 --2- 172.21.15.67:0/4129252559 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f93840b6480 0x7f93840b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:22.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.681+0000 7f938f7fe640 1 -- 172.21.15.67:0/4129252559 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f93880e51d0 con 0x7f9398103720 2024-09-21T06:28:22.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.682+0000 7f939e594640 1 --2- 172.21.15.67:0/4129252559 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f93840b6480 0x7f93840b8990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9394005d00 tx=0x7f9394005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:22.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.684+0000 7f938f7fe640 1 -- 172.21.15.67:0/4129252559 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f93880ad730 con 0x7f9398103720 2024-09-21T06:28:22.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.873+0000 7f93a07f8640 1 -- 172.21.15.67:0/4129252559 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f93981a4550 con 0x7f9398103720 2024-09-21T06:28:22.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.877+0000 7f938f7fe640 1 -- 172.21.15.67:0/4129252559 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v33)=0 v33) ==== 116+0+0 (secure 0 0 0) 0x7f93981a4550 con 0x7f9398103720 2024-09-21T06:28:22.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.880+0000 7f93a07f8640 1 -- 172.21.15.67:0/4129252559 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f93840b6480 msgr2=0x7f93840b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:22.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.880+0000 7f93a07f8640 1 --2- 172.21.15.67:0/4129252559 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f93840b6480 0x7f93840b8990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9394005d00 tx=0x7f9394005c70 comp rx=0 tx=0).stop 2024-09-21T06:28:22.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.880+0000 7f93a07f8640 1 -- 172.21.15.67:0/4129252559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9398103720 msgr2=0x7f9398104b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:22.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.880+0000 7f93a07f8640 1 --2- 172.21.15.67:0/4129252559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9398103720 0x7f9398104b50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f938800d950 tx=0x7f938800de10 comp rx=0 tx=0).stop 2024-09-21T06:28:22.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.880+0000 7f93a07f8640 1 -- 172.21.15.67:0/4129252559 shutdown_connections 2024-09-21T06:28:22.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.880+0000 7f93a07f8640 1 --2- 172.21.15.67:0/4129252559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93981040c0 0x7f9398105090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:22.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.880+0000 7f93a07f8640 1 --2- 172.21.15.67:0/4129252559 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f93840b6480 0x7f93840b8990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:22.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.880+0000 7f93a07f8640 1 --2- 172.21.15.67:0/4129252559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9398103720 0x7f9398104b50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:22.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.880+0000 7f93a07f8640 1 --2- 172.21.15.67:0/4129252559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9398102690 0x7f9398106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:22.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.880+0000 7f93a07f8640 1 -- 172.21.15.67:0/4129252559 >> 172.21.15.67:0/4129252559 conn(0x7f93980fe000 msgr2=0x7f93980ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:22.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.880+0000 7f93a07f8640 1 -- 172.21.15.67:0/4129252559 shutdown_connections 2024-09-21T06:28:22.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:22.880+0000 7f93a07f8640 1 -- 172.21.15.67:0/4129252559 wait complete. 2024-09-21T06:28:23.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:23 smithi067 ceph-mon[31232]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:23.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:23 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/4129252559' entity='client.admin' 2024-09-21T06:28:23.475 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:28:23.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:23 smithi100 ceph-mon[33401]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:23.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:23 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/4129252559' entity='client.admin' 2024-09-21T06:28:23.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:23 smithi203 ceph-mon[33735]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:23.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:23 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/4129252559' entity='client.admin' 2024-09-21T06:28:25.258 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2a791768-77e2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi067') 2024-09-21T06:28:25.258 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2a791768-77e2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi067') 2024-09-21T06:28:25.258 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2a791768-77e2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi067') 2024-09-21T06:28:25.258 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2a791768-77e2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi067') 2024-09-21T06:28:25.258 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2a791768-77e2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi067') 2024-09-21T06:28:25.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:25 smithi067 ceph-mon[31232]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:25.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:25 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:25.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:25 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:25.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:25 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:28:25.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:25 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:25.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:25 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:28:25.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:25 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:25.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:25 smithi100 ceph-mon[33401]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:25.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:25 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:25.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:25 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:25.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:25 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:28:25.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:25 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:25.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:25 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:28:25.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:25 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:25.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:25 smithi203 ceph-mon[33735]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:25.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:25 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:25.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:25 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:25.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:25 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:28:25.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:25 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:25.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:25 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:28:25.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:25 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:26.309 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2a791768-77e2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi067') 2024-09-21T06:28:26.310 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2a791768-77e2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi067') 2024-09-21T06:28:26.310 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2a791768-77e2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi067') 2024-09-21T06:28:26.310 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2a791768-77e2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi067') 2024-09-21T06:28:26.310 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2a791768-77e2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi067') 2024-09-21T06:28:26.310 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2a791768-77e2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi067') 2024-09-21T06:28:26.310 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2a791768-77e2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi067') 2024-09-21T06:28:26.310 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2a791768-77e2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi067') 2024-09-21T06:28:26.310 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2a791768-77e2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi067') 2024-09-21T06:28:26.311 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2a791768-77e2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi067') 2024-09-21T06:28:26.311 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2a791768-77e2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi067') 2024-09-21T06:28:26.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.525+0000 7f5db93c6640 1 -- 172.21.15.67:0/797178637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5db41047e0 msgr2=0x7f5db4104bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:26.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.525+0000 7f5db93c6640 1 --2- 172.21.15.67:0/797178637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5db41047e0 0x7f5db4104bf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009900 tx=0x7f5d9c02a480 comp rx=0 tx=0).stop 2024-09-21T06:28:26.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.525+0000 7f5db93c6640 1 -- 172.21.15.67:0/797178637 shutdown_connections 2024-09-21T06:28:26.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.525+0000 7f5db93c6640 1 --2- 172.21.15.67:0/797178637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5db4105180 0x7f5db410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.525+0000 7f5db93c6640 1 --2- 172.21.15.67:0/797178637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5db41047e0 0x7f5db4104bf0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.525+0000 7f5db93c6640 1 --2- 172.21.15.67:0/797178637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5db4103750 0x7f5db4103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.525+0000 7f5db93c6640 1 -- 172.21.15.67:0/797178637 >> 172.21.15.67:0/797178637 conn(0x7f5db40ff0c0 msgr2=0x7f5db4101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:26.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.525+0000 7f5db93c6640 1 -- 172.21.15.67:0/797178637 shutdown_connections 2024-09-21T06:28:26.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.525+0000 7f5db93c6640 1 -- 172.21.15.67:0/797178637 wait complete. 2024-09-21T06:28:26.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.526+0000 7f5db93c6640 1 Processor -- start 2024-09-21T06:28:26.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.526+0000 7f5db93c6640 1 -- start start 2024-09-21T06:28:26.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.527+0000 7f5db93c6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5db4103750 0x7f5db4199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:26.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.527+0000 7f5db93c6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5db41047e0 0x7f5db419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:26.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.527+0000 7f5db93c6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5db4105180 0x7f5db419b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:26.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.527+0000 7f5db93c6640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5db410f250 con 0x7f5db41047e0 2024-09-21T06:28:26.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.527+0000 7f5db93c6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5db410f0d0 con 0x7f5db4105180 2024-09-21T06:28:26.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.527+0000 7f5db93c6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5db410f3d0 con 0x7f5db4103750 2024-09-21T06:28:26.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.527+0000 7f5dabfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5db41047e0 0x7f5db419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:26.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.527+0000 7f5dabfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5db41047e0 0x7f5db419a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59534/0 (socket says 172.21.15.67:59534) 2024-09-21T06:28:26.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.527+0000 7f5dabfff640 1 -- 172.21.15.67:0/647099566 learned_addr learned my addr 172.21.15.67:0/647099566 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:28:26.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.527+0000 7f5db2ffd640 1 --2- 172.21.15.67:0/647099566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5db4103750 0x7f5db4199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:26.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.527+0000 7f5db37fe640 1 --2- 172.21.15.67:0/647099566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5db4105180 0x7f5db419b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:26.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.527+0000 7f5dabfff640 1 -- 172.21.15.67:0/647099566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5db4103750 msgr2=0x7f5db4199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:26.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.527+0000 7f5dabfff640 1 --2- 172.21.15.67:0/647099566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5db4103750 0x7f5db4199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.527+0000 7f5dabfff640 1 -- 172.21.15.67:0/647099566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5db4105180 msgr2=0x7f5db419b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:26.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.527+0000 7f5dabfff640 1 --2- 172.21.15.67:0/647099566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5db4105180 0x7f5db419b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.527+0000 7f5dabfff640 1 -- 172.21.15.67:0/647099566 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db419b930 con 0x7f5db41047e0 2024-09-21T06:28:26.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.527+0000 7f5db2ffd640 1 --2- 172.21.15.67:0/647099566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5db4103750 0x7f5db4199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:28:26.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.527+0000 7f5dabfff640 1 --2- 172.21.15.67:0/647099566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5db41047e0 0x7f5db419a570 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009230 tx=0x7f5d9c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:26.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.527+0000 7f5db0ff9640 1 -- 172.21.15.67:0/647099566 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c03f030 con 0x7f5db41047e0 2024-09-21T06:28:26.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.528+0000 7f5db93c6640 1 -- 172.21.15.67:0/647099566 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db419bbc0 con 0x7f5db41047e0 2024-09-21T06:28:26.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.528+0000 7f5db0ff9640 1 -- 172.21.15.67:0/647099566 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d9c003c90 con 0x7f5db41047e0 2024-09-21T06:28:26.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.528+0000 7f5db0ff9640 1 -- 172.21.15.67:0/647099566 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c035070 con 0x7f5db41047e0 2024-09-21T06:28:26.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.528+0000 7f5db93c6640 1 -- 172.21.15.67:0/647099566 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db41a2320 con 0x7f5db41047e0 2024-09-21T06:28:26.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.529+0000 7f5db93c6640 1 -- 172.21.15.67:0/647099566 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5db410a910 con 0x7f5db41047e0 2024-09-21T06:28:26.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.531+0000 7f5db0ff9640 1 -- 172.21.15.67:0/647099566 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f5d9c042030 con 0x7f5db41047e0 2024-09-21T06:28:26.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.531+0000 7f5db0ff9640 1 --2- 172.21.15.67:0/647099566 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5da00b6480 0x7f5da00b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:26.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.532+0000 7f5db2ffd640 1 --2- 172.21.15.67:0/647099566 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5da00b6480 0x7f5da00b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:26.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.532+0000 7f5db0ff9640 1 -- 172.21.15.67:0/647099566 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5d9c106110 con 0x7f5db41047e0 2024-09-21T06:28:26.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.532+0000 7f5db2ffd640 1 --2- 172.21.15.67:0/647099566 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5da00b6480 0x7f5da00b8990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009b40 tx=0x7f5da4005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:26.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.534+0000 7f5db0ff9640 1 -- 172.21.15.67:0/647099566 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5d9c0ce670 con 0x7f5db41047e0 2024-09-21T06:28:26.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.723+0000 7f5db93c6640 1 -- 172.21.15.67:0/647099566 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7f5db402e640 con 0x7f5db41047e0 2024-09-21T06:28:26.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.723+0000 7f5db0ff9640 1 -- 172.21.15.67:0/647099566 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v33)=0 v33) ==== 108+0+0 (secure 0 0 0) 0x7f5d9c0d2a50 con 0x7f5db41047e0 2024-09-21T06:28:26.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.725+0000 7f5db93c6640 1 -- 172.21.15.67:0/647099566 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5da00b6480 msgr2=0x7f5da00b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:26.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.725+0000 7f5db93c6640 1 --2- 172.21.15.67:0/647099566 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5da00b6480 0x7f5da00b8990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009b40 tx=0x7f5da4005e50 comp rx=0 tx=0).stop 2024-09-21T06:28:26.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.725+0000 7f5db93c6640 1 -- 172.21.15.67:0/647099566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5db41047e0 msgr2=0x7f5db419a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:26.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.725+0000 7f5db93c6640 1 --2- 172.21.15.67:0/647099566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5db41047e0 0x7f5db419a570 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009230 tx=0x7f5d9c0042f0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.725+0000 7f5db93c6640 1 -- 172.21.15.67:0/647099566 shutdown_connections 2024-09-21T06:28:26.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.725+0000 7f5db93c6640 1 --2- 172.21.15.67:0/647099566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5db4105180 0x7f5db419b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.726+0000 7f5db93c6640 1 --2- 172.21.15.67:0/647099566 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5da00b6480 0x7f5da00b8990 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.726+0000 7f5db93c6640 1 --2- 172.21.15.67:0/647099566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5db41047e0 0x7f5db419a570 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.726+0000 7f5db93c6640 1 --2- 172.21.15.67:0/647099566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5db4103750 0x7f5db4199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:26.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.726+0000 7f5db93c6640 1 -- 172.21.15.67:0/647099566 >> 172.21.15.67:0/647099566 conn(0x7f5db40ff0c0 msgr2=0x7f5db4107ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:26.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.726+0000 7f5db93c6640 1 -- 172.21.15.67:0/647099566 shutdown_connections 2024-09-21T06:28:26.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:26.726+0000 7f5db93c6640 1 -- 172.21.15.67:0/647099566 wait complete. 2024-09-21T06:28:27.276 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:27 smithi067 ceph-mon[31232]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:27.305 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2a791768-77e2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi067') 2024-09-21T06:28:27.305 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2a791768-77e2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi067') 2024-09-21T06:28:27.305 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2a791768-77e2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi067') 2024-09-21T06:28:27.305 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2a791768-77e2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi067') 2024-09-21T06:28:27.306 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph config dump 2024-09-21T06:28:27.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:27 smithi100 ceph-mon[33401]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:27.556 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:28:27.624 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:27 smithi203 ceph-mon[33735]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:27.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.994+0000 7f54cff58640 1 -- 172.21.15.67:0/3763766930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54c8109470 msgr2=0x7f54c810b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:27.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.994+0000 7f54cff58640 1 --2- 172.21.15.67:0/3763766930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54c8109470 0x7f54c810b980 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f54b8009900 tx=0x7f54b802a410 comp rx=0 tx=0).stop 2024-09-21T06:28:27.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.994+0000 7f54cff58640 1 -- 172.21.15.67:0/3763766930 shutdown_connections 2024-09-21T06:28:27.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.994+0000 7f54cff58640 1 --2- 172.21.15.67:0/3763766930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54c8109470 0x7f54c810b980 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:27.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.994+0000 7f54cff58640 1 --2- 172.21.15.67:0/3763766930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54c8106930 0x7f54c8108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:27.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.994+0000 7f54cff58640 1 --2- 172.21.15.67:0/3763766930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54c806b920 0x7f54c81063f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:27.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.994+0000 7f54cff58640 1 -- 172.21.15.67:0/3763766930 >> 172.21.15.67:0/3763766930 conn(0x7f54c80fdfb0 msgr2=0x7f54c8100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:27.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.994+0000 7f54cff58640 1 -- 172.21.15.67:0/3763766930 shutdown_connections 2024-09-21T06:28:27.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.994+0000 7f54cff58640 1 -- 172.21.15.67:0/3763766930 wait complete. 2024-09-21T06:28:27.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.995+0000 7f54cff58640 1 Processor -- start 2024-09-21T06:28:27.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.995+0000 7f54cff58640 1 -- start start 2024-09-21T06:28:27.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.995+0000 7f54cff58640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54c806b920 0x7f54c8199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:27.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.995+0000 7f54cff58640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54c8106930 0x7f54c819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:28.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.995+0000 7f54cff58640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54c8109470 0x7f54c819b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:28.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.995+0000 7f54cff58640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f54c810f230 con 0x7f54c806b920 2024-09-21T06:28:28.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.995+0000 7f54cff58640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f54c810f0b0 con 0x7f54c8106930 2024-09-21T06:28:28.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.995+0000 7f54cff58640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f54c810f3b0 con 0x7f54c8109470 2024-09-21T06:28:28.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.995+0000 7f54cdcf4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54c806b920 0x7f54c8199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:28.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.995+0000 7f54cdcf4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54c806b920 0x7f54c8199e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41096/0 (socket says 172.21.15.67:41096) 2024-09-21T06:28:28.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.995+0000 7f54cdcf4640 1 -- 172.21.15.67:0/1546136519 learned_addr learned my addr 172.21.15.67:0/1546136519 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:28:28.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.995+0000 7f54cd4f3640 1 --2- 172.21.15.67:0/1546136519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54c8106930 0x7f54c819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:28.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.995+0000 7f54ce4f5640 1 --2- 172.21.15.67:0/1546136519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54c8109470 0x7f54c819b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:28.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.996+0000 7f54cd4f3640 1 -- 172.21.15.67:0/1546136519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54c8109470 msgr2=0x7f54c819b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:28.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.996+0000 7f54cd4f3640 1 --2- 172.21.15.67:0/1546136519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54c8109470 0x7f54c819b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:28.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.996+0000 7f54cd4f3640 1 -- 172.21.15.67:0/1546136519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54c806b920 msgr2=0x7f54c8199e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:28.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.996+0000 7f54cd4f3640 1 --2- 172.21.15.67:0/1546136519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54c806b920 0x7f54c8199e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:28.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.996+0000 7f54cd4f3640 1 -- 172.21.15.67:0/1546136519 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c819b880 con 0x7f54c8106930 2024-09-21T06:28:28.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.996+0000 7f54ce4f5640 1 --2- 172.21.15.67:0/1546136519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54c8109470 0x7f54c819b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:28:28.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.996+0000 7f54cdcf4640 1 --2- 172.21.15.67:0/1546136519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54c806b920 0x7f54c8199e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:28:28.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.996+0000 7f54cd4f3640 1 --2- 172.21.15.67:0/1546136519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54c8106930 0x7f54c819a4c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f54bc009fd0 tx=0x7f54bc00ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:28.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.996+0000 7f54c6ffd640 1 -- 172.21.15.67:0/1546136519 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54bc004b20 con 0x7f54c8106930 2024-09-21T06:28:28.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.996+0000 7f54cff58640 1 -- 172.21.15.67:0/1546136519 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c819bb70 con 0x7f54c8106930 2024-09-21T06:28:28.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.996+0000 7f54c6ffd640 1 -- 172.21.15.67:0/1546136519 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f54bc004cc0 con 0x7f54c8106930 2024-09-21T06:28:28.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.996+0000 7f54c6ffd640 1 -- 172.21.15.67:0/1546136519 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54bc010690 con 0x7f54c8106930 2024-09-21T06:28:28.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.996+0000 7f54cff58640 1 -- 172.21.15.67:0/1546136519 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c81a2270 con 0x7f54c8106930 2024-09-21T06:28:28.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.997+0000 7f54cff58640 1 -- 172.21.15.67:0/1546136519 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54c80ffad0 con 0x7f54c8106930 2024-09-21T06:28:28.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.999+0000 7f54c6ffd640 1 -- 172.21.15.67:0/1546136519 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f54bc005020 con 0x7f54c8106930 2024-09-21T06:28:28.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.999+0000 7f54c6ffd640 1 --2- 172.21.15.67:0/1546136519 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f54b00b6480 0x7f54b00b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:28.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:27.999+0000 7f54cdcf4640 1 --2- 172.21.15.67:0/1546136519 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f54b00b6480 0x7f54b00b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:28.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:28.000+0000 7f54c6ffd640 1 -- 172.21.15.67:0/1546136519 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f54bc0e5e50 con 0x7f54c8106930 2024-09-21T06:28:28.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:28.000+0000 7f54cdcf4640 1 --2- 172.21.15.67:0/1546136519 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f54b00b6480 0x7f54b00b8990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f54c0011550 tx=0x7f54c0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:28.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:28.003+0000 7f54c6ffd640 1 -- 172.21.15.67:0/1546136519 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f54bc0ae390 con 0x7f54c8106930 2024-09-21T06:28:28.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:28.193+0000 7f54cff58640 1 -- 172.21.15.67:0/1546136519 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f54c802e640 con 0x7f54c8106930 2024-09-21T06:28:28.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:28.194+0000 7f54c6ffd640 1 -- 172.21.15.67:0/1546136519 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v33) ==== 59+0+20909 (secure 0 0 0) 0x7f54bc0b2770 con 0x7f54c8106930 2024-09-21T06:28:28.197 INFO:teuthology.orchestra.run.smithi067.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T06:28:28.197 INFO:teuthology.orchestra.run.smithi067.stdout:global dev auth_debug true 2024-09-21T06:28:28.197 INFO:teuthology.orchestra.run.smithi067.stdout:global basic container_image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 * 2024-09-21T06:28:28.197 INFO:teuthology.orchestra.run.smithi067.stdout:global dev debug_asserts_on_shutdown true 2024-09-21T06:28:28.197 INFO:teuthology.orchestra.run.smithi067.stdout:global basic log_to_file true 2024-09-21T06:28:28.197 INFO:teuthology.orchestra.run.smithi067.stdout:global basic log_to_journald false 2024-09-21T06:28:28.198 INFO:teuthology.orchestra.run.smithi067.stdout:global basic log_to_stderr false 2024-09-21T06:28:28.198 INFO:teuthology.orchestra.run.smithi067.stdout:global advanced mon_allow_pool_delete true 2024-09-21T06:28:28.198 INFO:teuthology.orchestra.run.smithi067.stdout:global advanced mon_clock_drift_allowed 1.000000 2024-09-21T06:28:28.198 INFO:teuthology.orchestra.run.smithi067.stdout:global advanced mon_cluster_log_to_file true 2024-09-21T06:28:28.198 INFO:teuthology.orchestra.run.smithi067.stdout:global advanced mon_max_pg_per_osd 10000 2024-09-21T06:28:28.198 INFO:teuthology.orchestra.run.smithi067.stdout:global advanced mon_pg_warn_max_object_skew 0.000000 2024-09-21T06:28:28.198 INFO:teuthology.orchestra.run.smithi067.stdout:global advanced mon_warn_on_crush_straw_calc_version_zero false 2024-09-21T06:28:28.198 INFO:teuthology.orchestra.run.smithi067.stdout:global advanced mon_warn_on_legacy_crush_tunables false 2024-09-21T06:28:28.198 INFO:teuthology.orchestra.run.smithi067.stdout:global advanced mon_warn_on_osd_down_out_interval_zero false 2024-09-21T06:28:28.198 INFO:teuthology.orchestra.run.smithi067.stdout:global dev mon_warn_on_pool_pg_num_not_power_of_two false 2024-09-21T06:28:28.198 INFO:teuthology.orchestra.run.smithi067.stdout:global advanced mon_warn_on_too_few_osds false 2024-09-21T06:28:28.198 INFO:teuthology.orchestra.run.smithi067.stdout:global dev ms_die_on_bug true 2024-09-21T06:28:28.198 INFO:teuthology.orchestra.run.smithi067.stdout:global dev ms_die_on_old_message true 2024-09-21T06:28:28.199 INFO:teuthology.orchestra.run.smithi067.stdout:global advanced osd_pool_default_erasure_code_profile plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T06:28:28.199 INFO:teuthology.orchestra.run.smithi067.stdout:global advanced osd_pool_default_pg_autoscale_mode off 2024-09-21T06:28:28.199 INFO:teuthology.orchestra.run.smithi067.stdout:mon advanced auth_allow_insecure_global_id_reclaim false 2024-09-21T06:28:28.199 INFO:teuthology.orchestra.run.smithi067.stdout:mon advanced auth_mon_ticket_ttl 660.000000 2024-09-21T06:28:28.199 INFO:teuthology.orchestra.run.smithi067.stdout:mon advanced auth_service_ticket_ttl 240.000000 2024-09-21T06:28:28.199 INFO:teuthology.orchestra.run.smithi067.stdout:mon advanced debug_mon 20/20 2024-09-21T06:28:28.199 INFO:teuthology.orchestra.run.smithi067.stdout:mon advanced debug_ms 1/1 2024-09-21T06:28:28.199 INFO:teuthology.orchestra.run.smithi067.stdout:mon advanced debug_paxos 20/20 2024-09-21T06:28:28.199 INFO:teuthology.orchestra.run.smithi067.stdout:mon advanced mon_data_avail_warn 5 2024-09-21T06:28:28.199 INFO:teuthology.orchestra.run.smithi067.stdout:mon advanced mon_down_mkfs_grace 300 2024-09-21T06:28:28.199 INFO:teuthology.orchestra.run.smithi067.stdout:mon advanced mon_mgr_mkfs_grace 240 2024-09-21T06:28:28.199 INFO:teuthology.orchestra.run.smithi067.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T06:28:28.199 INFO:teuthology.orchestra.run.smithi067.stdout:mon dev mon_osd_prime_pg_temp true 2024-09-21T06:28:28.199 INFO:teuthology.orchestra.run.smithi067.stdout:mon advanced mon_osd_reporter_subtree_level osd 2024-09-21T06:28:28.200 INFO:teuthology.orchestra.run.smithi067.stdout:mon advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T06:28:28.200 INFO:teuthology.orchestra.run.smithi067.stdout:mon advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T06:28:28.200 INFO:teuthology.orchestra.run.smithi067.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim false 2024-09-21T06:28:28.200 INFO:teuthology.orchestra.run.smithi067.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim_allowed false 2024-09-21T06:28:28.200 INFO:teuthology.orchestra.run.smithi067.stdout:mon advanced public_network 172.21.0.0/20 * 2024-09-21T06:28:28.200 INFO:teuthology.orchestra.run.smithi067.stdout:mgr advanced cephsqlite_lock_renewal_timeout 900000 2024-09-21T06:28:28.200 INFO:teuthology.orchestra.run.smithi067.stdout:mgr advanced client_mount_timeout 30 2024-09-21T06:28:28.200 INFO:teuthology.orchestra.run.smithi067.stdout:mgr advanced debug_client 20/20 2024-09-21T06:28:28.200 INFO:teuthology.orchestra.run.smithi067.stdout:mgr advanced debug_mgr 20/20 2024-09-21T06:28:28.200 INFO:teuthology.orchestra.run.smithi067.stdout:mgr advanced debug_ms 1/1 2024-09-21T06:28:28.200 INFO:teuthology.orchestra.run.smithi067.stdout:mgr advanced mgr/cephadm/allow_ptrace true * 2024-09-21T06:28:28.200 INFO:teuthology.orchestra.run.smithi067.stdout:mgr advanced mgr/cephadm/container_init True * 2024-09-21T06:28:28.200 INFO:teuthology.orchestra.run.smithi067.stdout:mgr advanced mgr/cephadm/migration_current 7 * 2024-09-21T06:28:28.201 INFO:teuthology.orchestra.run.smithi067.stdout:mgr advanced mgr/dashboard/ssl_server_port 8443 * 2024-09-21T06:28:28.201 INFO:teuthology.orchestra.run.smithi067.stdout:mgr advanced mgr/orchestrator/orchestrator cephadm 2024-09-21T06:28:28.201 INFO:teuthology.orchestra.run.smithi067.stdout:mgr advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T06:28:28.201 INFO:teuthology.orchestra.run.smithi067.stdout:mgr advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T06:28:28.201 INFO:teuthology.orchestra.run.smithi067.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T06:28:28.201 INFO:teuthology.orchestra.run.smithi067.stdout:osd dev bdev_debug_aio true 2024-09-21T06:28:28.201 INFO:teuthology.orchestra.run.smithi067.stdout:osd dev bluestore_block_size 96636764160 * 2024-09-21T06:28:28.201 INFO:teuthology.orchestra.run.smithi067.stdout:osd advanced bluestore_compression_mode aggressive 2024-09-21T06:28:28.201 INFO:teuthology.orchestra.run.smithi067.stdout:osd dev bluestore_fsck_on_mount true 2024-09-21T06:28:28.201 INFO:teuthology.orchestra.run.smithi067.stdout:osd advanced debug_bluefs 20/20 2024-09-21T06:28:28.201 INFO:teuthology.orchestra.run.smithi067.stdout:osd advanced debug_bluestore 20/20 2024-09-21T06:28:28.201 INFO:teuthology.orchestra.run.smithi067.stdout:osd advanced debug_ms 1/1 2024-09-21T06:28:28.201 INFO:teuthology.orchestra.run.smithi067.stdout:osd advanced debug_osd 20/20 2024-09-21T06:28:28.201 INFO:teuthology.orchestra.run.smithi067.stdout:osd advanced debug_rocksdb 10/10 2024-09-21T06:28:28.202 INFO:teuthology.orchestra.run.smithi067.stdout:osd dev osd_debug_misdirected_ops true 2024-09-21T06:28:28.202 INFO:teuthology.orchestra.run.smithi067.stdout:osd dev osd_debug_op_order true 2024-09-21T06:28:28.202 INFO:teuthology.orchestra.run.smithi067.stdout:osd dev osd_debug_pg_log_writeout true 2024-09-21T06:28:28.202 INFO:teuthology.orchestra.run.smithi067.stdout:osd dev osd_debug_shutdown true 2024-09-21T06:28:28.202 INFO:teuthology.orchestra.run.smithi067.stdout:osd dev osd_debug_verify_cached_snaps true 2024-09-21T06:28:28.202 INFO:teuthology.orchestra.run.smithi067.stdout:osd dev osd_debug_verify_missing_on_start true 2024-09-21T06:28:28.202 INFO:teuthology.orchestra.run.smithi067.stdout:osd dev osd_debug_verify_stray_on_activate true 2024-09-21T06:28:28.202 INFO:teuthology.orchestra.run.smithi067.stdout:osd advanced osd_deep_scrub_update_digest_min_age 30 2024-09-21T06:28:28.202 INFO:teuthology.orchestra.run.smithi067.stdout:osd advanced osd_failsafe_full_ratio 0.950000 2024-09-21T06:28:28.202 INFO:teuthology.orchestra.run.smithi067.stdout:osd advanced osd_max_omap_entries_per_request 10000 2024-09-21T06:28:28.202 INFO:teuthology.orchestra.run.smithi067.stdout:osd advanced osd_mclock_profile high_recovery_ops 2024-09-21T06:28:28.202 INFO:teuthology.orchestra.run.smithi067.stdout:osd advanced osd_memory_target_autotune true 2024-09-21T06:28:28.202 INFO:teuthology.orchestra.run.smithi067.stdout:osd advanced osd_objectstore bluestore * 2024-09-21T06:28:28.202 INFO:teuthology.orchestra.run.smithi067.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T06:28:28.203 INFO:teuthology.orchestra.run.smithi067.stdout:osd advanced osd_op_queue debug_random * 2024-09-21T06:28:28.203 INFO:teuthology.orchestra.run.smithi067.stdout:osd advanced osd_op_queue_cut_off debug_random * 2024-09-21T06:28:28.203 INFO:teuthology.orchestra.run.smithi067.stdout:osd advanced osd_recover_clone_overlap true 2024-09-21T06:28:28.203 INFO:teuthology.orchestra.run.smithi067.stdout:osd advanced osd_recovery_max_chunk 1048576 2024-09-21T06:28:28.203 INFO:teuthology.orchestra.run.smithi067.stdout:osd advanced osd_scrub_load_threshold 5.000000 2024-09-21T06:28:28.203 INFO:teuthology.orchestra.run.smithi067.stdout:osd advanced osd_scrub_max_interval 600.000000 2024-09-21T06:28:28.203 INFO:teuthology.orchestra.run.smithi067.stdout:osd advanced osd_shutdown_pgref_assert true 2024-09-21T06:28:28.203 INFO:teuthology.orchestra.run.smithi067.stdout:mds advanced debug_mds 20/20 2024-09-21T06:28:28.203 INFO:teuthology.orchestra.run.smithi067.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T06:28:28.203 INFO:teuthology.orchestra.run.smithi067.stdout:mds advanced debug_ms 2/2 2024-09-21T06:28:28.203 INFO:teuthology.orchestra.run.smithi067.stdout:mds advanced mds_bal_fragment_size_max 10000 2024-09-21T06:28:28.203 INFO:teuthology.orchestra.run.smithi067.stdout:mds advanced mds_bal_merge_size 5 2024-09-21T06:28:28.203 INFO:teuthology.orchestra.run.smithi067.stdout:mds advanced mds_bal_split_bits 3 2024-09-21T06:28:28.204 INFO:teuthology.orchestra.run.smithi067.stdout:mds advanced mds_bal_split_size 100 2024-09-21T06:28:28.204 INFO:teuthology.orchestra.run.smithi067.stdout:mds dev mds_debug_frag true 2024-09-21T06:28:28.204 INFO:teuthology.orchestra.run.smithi067.stdout:mds dev mds_debug_scatterstat true 2024-09-21T06:28:28.204 INFO:teuthology.orchestra.run.smithi067.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T06:28:28.204 INFO:teuthology.orchestra.run.smithi067.stdout:mds dev mds_verify_scatter true 2024-09-21T06:28:28.204 INFO:teuthology.orchestra.run.smithi067.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T06:28:28.204 INFO:teuthology.orchestra.run.smithi067.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T06:28:28.204 INFO:teuthology.orchestra.run.smithi067.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T06:28:28.204 INFO:teuthology.orchestra.run.smithi067.stdout:client advanced client_mount_timeout 600 2024-09-21T06:28:28.204 INFO:teuthology.orchestra.run.smithi067.stdout:client advanced debug_client 20/20 2024-09-21T06:28:28.204 INFO:teuthology.orchestra.run.smithi067.stdout:client advanced debug_ms 1/1 2024-09-21T06:28:28.204 INFO:teuthology.orchestra.run.smithi067.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T06:28:28.204 INFO:teuthology.orchestra.run.smithi067.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T06:28:28.204 INFO:teuthology.orchestra.run.smithi067.stdout:client.rgw advanced rgw_cache_enabled true 2024-09-21T06:28:28.205 INFO:teuthology.orchestra.run.smithi067.stdout:client.rgw advanced rgw_enable_ops_log true 2024-09-21T06:28:28.205 INFO:teuthology.orchestra.run.smithi067.stdout:client.rgw advanced rgw_enable_usage_log true 2024-09-21T06:28:28.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:28.198+0000 7f54cff58640 1 -- 172.21.15.67:0/1546136519 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f54b00b6480 msgr2=0x7f54b00b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:28.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:28.198+0000 7f54cff58640 1 --2- 172.21.15.67:0/1546136519 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f54b00b6480 0x7f54b00b8990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f54c0011550 tx=0x7f54c0005f90 comp rx=0 tx=0).stop 2024-09-21T06:28:28.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:28.198+0000 7f54cff58640 1 -- 172.21.15.67:0/1546136519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54c8106930 msgr2=0x7f54c819a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:28.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:28.198+0000 7f54cff58640 1 --2- 172.21.15.67:0/1546136519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54c8106930 0x7f54c819a4c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f54bc009fd0 tx=0x7f54bc00ef90 comp rx=0 tx=0).stop 2024-09-21T06:28:28.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:28.198+0000 7f54cff58640 1 -- 172.21.15.67:0/1546136519 shutdown_connections 2024-09-21T06:28:28.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:28.198+0000 7f54cff58640 1 --2- 172.21.15.67:0/1546136519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54c8106930 0x7f54c819a4c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:28.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:28.198+0000 7f54cff58640 1 --2- 172.21.15.67:0/1546136519 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f54b00b6480 0x7f54b00b8990 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:28.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:28.198+0000 7f54cff58640 1 --2- 172.21.15.67:0/1546136519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54c8109470 0x7f54c819b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:28.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:28.198+0000 7f54cff58640 1 --2- 172.21.15.67:0/1546136519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54c806b920 0x7f54c8199e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:28.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:28.198+0000 7f54cff58640 1 -- 172.21.15.67:0/1546136519 >> 172.21.15.67:0/1546136519 conn(0x7f54c80fdfb0 msgr2=0x7f54c8104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:28.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:28.198+0000 7f54cff58640 1 -- 172.21.15.67:0/1546136519 shutdown_connections 2024-09-21T06:28:28.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:28.198+0000 7f54cff58640 1 -- 172.21.15.67:0/1546136519 wait complete. 2024-09-21T06:28:28.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:28 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1546136519' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T06:28:28.496 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:28 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1546136519' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T06:28:28.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:28 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1546136519' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T06:28:28.964 INFO:tasks.cephadm:Deploying OSDs... 2024-09-21T06:28:28.964 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-21T06:28:28.964 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T06:28:28.981 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T06:28:28.981 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_1 2024-09-21T06:28:29.037 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T06:28:29.037 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:28:29.037 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-21T06:28:29.038 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:28:29.038 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:28:29.038 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-09-21 06:27:16.958445787 +0000 2024-09-21T06:28:29.038 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-09-21 06:22:30.335514511 +0000 2024-09-21T06:28:29.038 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-09-21 06:22:30.335514511 +0000 2024-09-21T06:28:29.038 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-09-21 06:22:30.335514511 +0000 2024-09-21T06:28:29.039 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T06:28:29.103 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-09-21T06:28:29.103 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-09-21T06:28:29.103 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000253689 s, 2.0 MB/s 2024-09-21T06:28:29.105 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T06:28:29.162 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_2 2024-09-21T06:28:29.218 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T06:28:29.218 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:28:29.218 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-21T06:28:29.218 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:28:29.218 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:28:29.218 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-09-21 06:27:16.984445223 +0000 2024-09-21T06:28:29.218 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-09-21 06:22:30.665507418 +0000 2024-09-21T06:28:29.218 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-09-21 06:22:30.665507418 +0000 2024-09-21T06:28:29.218 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-09-21 06:22:30.665507418 +0000 2024-09-21T06:28:29.218 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T06:28:29.283 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-09-21T06:28:29.283 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-09-21T06:28:29.283 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000215986 s, 2.4 MB/s 2024-09-21T06:28:29.284 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T06:28:29.341 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_3 2024-09-21T06:28:29.396 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T06:28:29.397 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:28:29.397 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-21T06:28:29.397 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:28:29.397 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:28:29.397 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-09-21 06:27:17.011444637 +0000 2024-09-21T06:28:29.397 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-09-21 06:22:30.997500281 +0000 2024-09-21T06:28:29.397 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-09-21 06:22:30.997500281 +0000 2024-09-21T06:28:29.397 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-09-21 06:22:30.997500281 +0000 2024-09-21T06:28:29.397 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T06:28:29.461 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-09-21T06:28:29.461 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-09-21T06:28:29.461 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000196936 s, 2.6 MB/s 2024-09-21T06:28:29.463 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T06:28:29.519 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_4 2024-09-21T06:28:29.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:29 smithi067 ceph-mon[31232]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:29.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:29 smithi100 ceph-mon[33401]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:29.557 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T06:28:29.557 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:28:29.557 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-21T06:28:29.557 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:28:29.557 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:28:29.557 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-09-21 06:27:17.037444072 +0000 2024-09-21T06:28:29.557 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-09-21 06:22:31.343492844 +0000 2024-09-21T06:28:29.557 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-09-21 06:22:31.343492844 +0000 2024-09-21T06:28:29.558 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-09-21 06:22:31.343492844 +0000 2024-09-21T06:28:29.558 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T06:28:29.622 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-09-21T06:28:29.622 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-09-21T06:28:29.622 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000195015 s, 2.6 MB/s 2024-09-21T06:28:29.623 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T06:28:29.680 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-21T06:28:29.680 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T06:28:29.700 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T06:28:29.700 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_1 2024-09-21T06:28:29.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:29 smithi203 ceph-mon[33735]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:29.758 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T06:28:29.758 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:28:29.758 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-21T06:28:29.758 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:28:29.758 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:28:29.759 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-09-21 06:28:04.370956618 +0000 2024-09-21T06:28:29.759 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-09-21 06:22:25.952753385 +0000 2024-09-21T06:28:29.759 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-09-21 06:22:25.952753385 +0000 2024-09-21T06:28:29.759 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-09-21 06:22:25.952753385 +0000 2024-09-21T06:28:29.759 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T06:28:29.828 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-21T06:28:29.828 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-21T06:28:29.829 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000231124 s, 2.2 MB/s 2024-09-21T06:28:29.832 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T06:28:29.892 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_2 2024-09-21T06:28:29.950 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T06:28:29.950 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:28:29.950 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-21T06:28:29.951 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:28:29.951 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:28:29.951 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-09-21 06:28:04.370956618 +0000 2024-09-21T06:28:29.951 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-09-21 06:22:26.287744816 +0000 2024-09-21T06:28:29.951 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-09-21 06:22:26.287744816 +0000 2024-09-21T06:28:29.951 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-09-21 06:22:26.287744816 +0000 2024-09-21T06:28:29.951 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T06:28:30.017 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-21T06:28:30.017 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-21T06:28:30.017 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000205277 s, 2.5 MB/s 2024-09-21T06:28:30.018 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T06:28:30.076 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_3 2024-09-21T06:28:30.133 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T06:28:30.133 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:28:30.134 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-21T06:28:30.134 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:28:30.134 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:28:30.134 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-09-21 06:28:04.371956592 +0000 2024-09-21T06:28:30.134 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-09-21 06:22:26.615736426 +0000 2024-09-21T06:28:30.134 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-09-21 06:22:26.615736426 +0000 2024-09-21T06:28:30.134 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-09-21 06:22:26.615736426 +0000 2024-09-21T06:28:30.134 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T06:28:30.199 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-21T06:28:30.199 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-21T06:28:30.199 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000190089 s, 2.7 MB/s 2024-09-21T06:28:30.200 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T06:28:30.257 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_4 2024-09-21T06:28:30.313 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T06:28:30.313 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:28:30.314 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-21T06:28:30.314 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:28:30.314 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:28:30.314 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-09-21 06:28:04.371956592 +0000 2024-09-21T06:28:30.314 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-09-21 06:22:26.930728369 +0000 2024-09-21T06:28:30.314 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-09-21 06:22:26.930728369 +0000 2024-09-21T06:28:30.314 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-09-21 06:22:26.930728369 +0000 2024-09-21T06:28:30.314 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T06:28:30.377 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-21T06:28:30.378 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-21T06:28:30.378 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000233035 s, 2.2 MB/s 2024-09-21T06:28:30.379 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T06:28:30.435 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-21T06:28:30.435 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T06:28:30.455 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T06:28:30.455 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_1 2024-09-21T06:28:30.514 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T06:28:30.514 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:28:30.514 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-21T06:28:30.514 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:28:30.514 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:28:30.514 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-09-21 06:28:02.122535297 +0000 2024-09-21T06:28:30.515 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-09-21 06:22:26.320025695 +0000 2024-09-21T06:28:30.515 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-09-21 06:22:26.320025695 +0000 2024-09-21T06:28:30.515 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-09-21 06:22:26.320025695 +0000 2024-09-21T06:28:30.515 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T06:28:30.586 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-09-21T06:28:30.586 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-09-21T06:28:30.586 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.00028833 s, 1.8 MB/s 2024-09-21T06:28:30.587 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T06:28:30.646 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_2 2024-09-21T06:28:30.703 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T06:28:30.704 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:28:30.704 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-21T06:28:30.704 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:28:30.704 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:28:30.704 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-09-21 06:28:02.123535272 +0000 2024-09-21T06:28:30.704 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-09-21 06:22:26.633017883 +0000 2024-09-21T06:28:30.704 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-09-21 06:22:26.633017883 +0000 2024-09-21T06:28:30.704 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-09-21 06:22:26.633017883 +0000 2024-09-21T06:28:30.704 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T06:28:30.770 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-09-21T06:28:30.770 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-09-21T06:28:30.770 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000211098 s, 2.4 MB/s 2024-09-21T06:28:30.771 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T06:28:30.829 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_3 2024-09-21T06:28:30.886 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T06:28:30.886 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:28:30.886 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-21T06:28:30.886 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:28:30.886 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:28:30.886 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-09-21 06:28:02.123535272 +0000 2024-09-21T06:28:30.886 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-09-21 06:22:26.957009795 +0000 2024-09-21T06:28:30.887 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-09-21 06:22:26.957009795 +0000 2024-09-21T06:28:30.887 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-09-21 06:22:26.957009795 +0000 2024-09-21T06:28:30.887 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T06:28:30.951 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-09-21T06:28:30.951 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-09-21T06:28:30.951 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000202019 s, 2.5 MB/s 2024-09-21T06:28:30.952 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T06:28:31.009 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_4 2024-09-21T06:28:31.065 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T06:28:31.066 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:28:31.066 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-21T06:28:31.066 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:28:31.066 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:28:31.066 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-09-21 06:28:02.124535247 +0000 2024-09-21T06:28:31.066 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-09-21 06:22:27.293001408 +0000 2024-09-21T06:28:31.066 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-09-21 06:22:27.293001408 +0000 2024-09-21T06:28:31.066 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-09-21 06:22:27.293001408 +0000 2024-09-21T06:28:31.066 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T06:28:31.129 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-09-21T06:28:31.129 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-09-21T06:28:31.129 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000206577 s, 2.5 MB/s 2024-09-21T06:28:31.130 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T06:28:31.186 INFO:tasks.cephadm:Deploying osd.0 on smithi067 with /dev/vg_nvme/lv_4... 2024-09-21T06:28:31.187 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T06:28:31.449 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:28:31.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:31 smithi067 ceph-mon[31232]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:31.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:31 smithi100 ceph-mon[33401]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:31.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:31 smithi203 ceph-mon[33735]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:33.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:33 smithi067 ceph-mon[31232]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:33.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:33 smithi100 ceph-mon[33401]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:33.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:33 smithi203 ceph-mon[33735]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:33.858 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:28:33.887 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi067:vg_nvme/lv_4 2024-09-21T06:28:34.140 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:28:34.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.610+0000 7f4460455640 1 -- 172.21.15.67:0/2257001893 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f445806c660 msgr2=0x7f44581063c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:34.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.610+0000 7f4460455640 1 --2- 172.21.15.67:0/2257001893 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f445806c660 0x7f44581063c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4454009900 tx=0x7f445402a410 comp rx=0 tx=0).stop 2024-09-21T06:28:34.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.610+0000 7f4460455640 1 -- 172.21.15.67:0/2257001893 shutdown_connections 2024-09-21T06:28:34.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.610+0000 7f4460455640 1 --2- 172.21.15.67:0/2257001893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4458106900 0x7f4458108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:34.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.610+0000 7f4460455640 1 --2- 172.21.15.67:0/2257001893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44581094a0 0x7f445810b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:34.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.610+0000 7f4460455640 1 --2- 172.21.15.67:0/2257001893 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f445806c660 0x7f44581063c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:34.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.610+0000 7f4460455640 1 -- 172.21.15.67:0/2257001893 >> 172.21.15.67:0/2257001893 conn(0x7f44580fdfe0 msgr2=0x7f4458100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:34.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.610+0000 7f4460455640 1 -- 172.21.15.67:0/2257001893 shutdown_connections 2024-09-21T06:28:34.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.610+0000 7f4460455640 1 -- 172.21.15.67:0/2257001893 wait complete. 2024-09-21T06:28:34.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.611+0000 7f4460455640 1 Processor -- start 2024-09-21T06:28:34.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.611+0000 7f4460455640 1 -- start start 2024-09-21T06:28:34.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.611+0000 7f4460455640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f445806c660 0x7f4458199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:34.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.611+0000 7f4460455640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4458106900 0x7f445819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:34.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.611+0000 7f4460455640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44581094a0 0x7f445819b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:34.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.611+0000 7f4460455640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f445810f310 con 0x7f44581094a0 2024-09-21T06:28:34.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.611+0000 7f4460455640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f445810f190 con 0x7f4458106900 2024-09-21T06:28:34.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.611+0000 7f4460455640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f445810f490 con 0x7f445806c660 2024-09-21T06:28:34.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.611+0000 7f445e9f2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44581094a0 0x7f445819b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:34.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.611+0000 7f445e9f2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44581094a0 0x7f445819b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41122/0 (socket says 172.21.15.67:41122) 2024-09-21T06:28:34.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.611+0000 7f445e9f2640 1 -- 172.21.15.67:0/94176330 learned_addr learned my addr 172.21.15.67:0/94176330 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:28:34.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.611+0000 7f445e1f1640 1 --2- 172.21.15.67:0/94176330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f445806c660 0x7f4458199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:34.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.612+0000 7f445d9f0640 1 --2- 172.21.15.67:0/94176330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4458106900 0x7f445819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:34.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.612+0000 7f445e9f2640 1 -- 172.21.15.67:0/94176330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f445806c660 msgr2=0x7f4458199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:34.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.612+0000 7f445e9f2640 1 --2- 172.21.15.67:0/94176330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f445806c660 0x7f4458199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:34.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.612+0000 7f445e9f2640 1 -- 172.21.15.67:0/94176330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4458106900 msgr2=0x7f445819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:34.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.612+0000 7f445e9f2640 1 --2- 172.21.15.67:0/94176330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4458106900 0x7f445819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:34.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.612+0000 7f445e9f2640 1 -- 172.21.15.67:0/94176330 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f445819b970 con 0x7f44581094a0 2024-09-21T06:28:34.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.612+0000 7f445e9f2640 1 --2- 172.21.15.67:0/94176330 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44581094a0 0x7f445819b150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f445000b970 tx=0x7f445000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:34.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.612+0000 7f444f7fe640 1 -- 172.21.15.67:0/94176330 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4450019070 con 0x7f44581094a0 2024-09-21T06:28:34.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.612+0000 7f444f7fe640 1 -- 172.21.15.67:0/94176330 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4450014440 con 0x7f44581094a0 2024-09-21T06:28:34.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.612+0000 7f444f7fe640 1 -- 172.21.15.67:0/94176330 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44500075d0 con 0x7f44581094a0 2024-09-21T06:28:34.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.612+0000 7f4460455640 1 -- 172.21.15.67:0/94176330 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f445819bc60 con 0x7f44581094a0 2024-09-21T06:28:34.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.612+0000 7f4460455640 1 -- 172.21.15.67:0/94176330 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44581a2360 con 0x7f44581094a0 2024-09-21T06:28:34.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.613+0000 7f4460455640 1 -- 172.21.15.67:0/94176330 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44580ffb00 con 0x7f44581094a0 2024-09-21T06:28:34.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.616+0000 7f444f7fe640 1 -- 172.21.15.67:0/94176330 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f445001c070 con 0x7f44581094a0 2024-09-21T06:28:34.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.616+0000 7f444f7fe640 1 --2- 172.21.15.67:0/94176330 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f44440b6500 0x7f44440b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:34.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.617+0000 7f445e1f1640 1 --2- 172.21.15.67:0/94176330 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f44440b6500 0x7f44440b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:34.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.617+0000 7f444f7fe640 1 -- 172.21.15.67:0/94176330 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f44500e4ec0 con 0x7f44581094a0 2024-09-21T06:28:34.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.617+0000 7f445e1f1640 1 --2- 172.21.15.67:0/94176330 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f44440b6500 0x7f44440b8a10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4454005760 tx=0x7f4454031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:34.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.620+0000 7f444f7fe640 1 -- 172.21.15.67:0/94176330 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f44500b1800 con 0x7f44581094a0 2024-09-21T06:28:34.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:34.819+0000 7f4460455640 1 -- 172.21.15.67:0/94176330 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f4458063800 con 0x7f44440b6500 2024-09-21T06:28:35.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:35 smithi067 ceph-mon[31232]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:35.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:28:35.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:28:35.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:35.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:35 smithi100 ceph-mon[33401]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:35.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:35 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:28:35.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:35 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:28:35.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:35 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:35.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:35 smithi203 ceph-mon[33735]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:35.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:35 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:28:35.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:35 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:28:35.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:35 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:36.421 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:36 smithi067 ceph-mon[31232]: from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:28:36.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:36 smithi100 ceph-mon[33401]: from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:28:36.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:36 smithi203 ceph-mon[33735]: from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:28:37.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:37 smithi100 ceph-mon[33401]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:37.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:37 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2637229778' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "18778364-82ed-41c0-92da-ce44cdad7358"} : dispatch 2024-09-21T06:28:37.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:37 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2637229778' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18778364-82ed-41c0-92da-ce44cdad7358"}]': finished 2024-09-21T06:28:37.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:37 smithi100 ceph-mon[33401]: osdmap e5: 1 total, 0 up, 1 in 2024-09-21T06:28:37.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:37 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:28:37.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:37 smithi203 ceph-mon[33735]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:37.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:37 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2637229778' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "18778364-82ed-41c0-92da-ce44cdad7358"} : dispatch 2024-09-21T06:28:37.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:37 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2637229778' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18778364-82ed-41c0-92da-ce44cdad7358"}]': finished 2024-09-21T06:28:37.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:37 smithi203 ceph-mon[33735]: osdmap e5: 1 total, 0 up, 1 in 2024-09-21T06:28:37.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:37 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:28:37.781 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:37 smithi067 ceph-mon[31232]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:37.781 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:37 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2637229778' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "18778364-82ed-41c0-92da-ce44cdad7358"} : dispatch 2024-09-21T06:28:37.781 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:37 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2637229778' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18778364-82ed-41c0-92da-ce44cdad7358"}]': finished 2024-09-21T06:28:37.781 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:37 smithi067 ceph-mon[31232]: osdmap e5: 1 total, 0 up, 1 in 2024-09-21T06:28:37.781 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:37 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:28:38.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:38 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2519050789' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:28:38.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:38 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2519050789' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:28:38.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:38 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2519050789' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:28:39.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:39 smithi203 ceph-mon[33735]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:39.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:39 smithi067 ceph-mon[31232]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:39.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:39 smithi100 ceph-mon[33401]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:41.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:41 smithi203 ceph-mon[33735]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:41.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:41 smithi067 ceph-mon[31232]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:41.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:41 smithi100 ceph-mon[33401]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:43.517 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:43 smithi067 ceph-mon[31232]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:43.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:43 smithi203 ceph-mon[33735]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:43.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:43 smithi100 ceph-mon[33401]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:45.541 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:45 smithi067 ceph-mon[31232]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:45.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:45 smithi203 ceph-mon[33735]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:45.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:45 smithi100 ceph-mon[33401]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:46.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:46 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T06:28:46.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:46 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:46.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:46 smithi067 ceph-mon[31232]: Deploying daemon osd.0 on smithi067 2024-09-21T06:28:46.633 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:46 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T06:28:46.633 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:46 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:46.633 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:46 smithi203 ceph-mon[33735]: Deploying daemon osd.0 on smithi067 2024-09-21T06:28:46.705 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:46 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T06:28:46.705 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:46 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:46.705 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:46 smithi100 ceph-mon[33401]: Deploying daemon osd.0 on smithi067 2024-09-21T06:28:47.337 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:47 smithi067 ceph-mon[31232]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:47.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:47 smithi203 ceph-mon[33735]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:47.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:47 smithi100 ceph-mon[33401]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:49.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:49 smithi067 ceph-mon[31232]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:49.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:49 smithi203 ceph-mon[33735]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:49.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:49 smithi100 ceph-mon[33401]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:51.645 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:51 smithi067 ceph-mon[31232]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:51.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:51 smithi203 ceph-mon[33735]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:51.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:51 smithi100 ceph-mon[33401]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:52.617 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:52 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:28:52.617 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:52 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:52.618 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:52 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:52.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:52 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:28:52.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:52 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:52.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:52 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:52.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:52 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:28:52.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:52 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:52.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:52 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:53.540 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:53 smithi067 ceph-mon[31232]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:53.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:53 smithi203 ceph-mon[33735]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:53.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:53 smithi100 ceph-mon[33401]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:55.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:55 smithi067 ceph-mon[31232]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:55.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:55 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:55.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:55 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:55.506 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:55 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:55.506 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:55 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:28:55.506 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:55 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:55.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:55 smithi203 ceph-mon[33735]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:55.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:55 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:55.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:55 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:55.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:55 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:55.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:55 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:28:55.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:55 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:55.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:55 smithi100 ceph-mon[33401]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:55.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:55 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:55.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:55 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:55.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:55 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:55.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:55 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:28:55.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:55 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:55.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:55.836+0000 7f444f7fe640 1 -- 172.21.15.67:0/94176330 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4458063800 con 0x7f44440b6500 2024-09-21T06:28:55.837 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 0 on host 'smithi067' 2024-09-21T06:28:55.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:55.838+0000 7f4460455640 1 -- 172.21.15.67:0/94176330 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f44440b6500 msgr2=0x7f44440b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:55.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:55.838+0000 7f4460455640 1 --2- 172.21.15.67:0/94176330 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f44440b6500 0x7f44440b8a10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4454005760 tx=0x7f4454031400 comp rx=0 tx=0).stop 2024-09-21T06:28:55.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:55.838+0000 7f4460455640 1 -- 172.21.15.67:0/94176330 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44581094a0 msgr2=0x7f445819b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:55.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:55.838+0000 7f4460455640 1 --2- 172.21.15.67:0/94176330 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44581094a0 0x7f445819b150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f445000b970 tx=0x7f445000be30 comp rx=0 tx=0).stop 2024-09-21T06:28:55.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:55.838+0000 7f4460455640 1 -- 172.21.15.67:0/94176330 shutdown_connections 2024-09-21T06:28:55.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:55.838+0000 7f4460455640 1 --2- 172.21.15.67:0/94176330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4458106900 0x7f445819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:55.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:55.838+0000 7f4460455640 1 --2- 172.21.15.67:0/94176330 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f44440b6500 0x7f44440b8a10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:55.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:55.838+0000 7f4460455640 1 --2- 172.21.15.67:0/94176330 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44581094a0 0x7f445819b150 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:55.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:55.838+0000 7f4460455640 1 --2- 172.21.15.67:0/94176330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f445806c660 0x7f4458199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:55.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:55.838+0000 7f4460455640 1 -- 172.21.15.67:0/94176330 >> 172.21.15.67:0/94176330 conn(0x7f44580fdfe0 msgr2=0x7f4458104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:55.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:55.839+0000 7f4460455640 1 -- 172.21.15.67:0/94176330 shutdown_connections 2024-09-21T06:28:55.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:28:55.839+0000 7f4460455640 1 -- 172.21.15.67:0/94176330 wait complete. 2024-09-21T06:28:56.426 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:56 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:28:56.426 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:56 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:56.426 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:56 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:56.455 DEBUG:teuthology.orchestra.run.smithi067:osd.0> sudo journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.0.service 2024-09-21T06:28:56.458 INFO:tasks.cephadm:Deploying osd.1 on smithi100 with /dev/vg_nvme/lv_4... 2024-09-21T06:28:56.458 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T06:28:56.705 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:28:56.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:56 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:28:56.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:56 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:56.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:56 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:56.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:56 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:28:56.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:56 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:56.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:56 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:57.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:57 smithi100 ceph-mon[33401]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:57.658 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:57 smithi067 ceph-mon[31232]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:57.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:57 smithi203 ceph-mon[33735]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:58.292 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 21 06:28:57 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-0[46350]: 2024-09-21T06:28:57.923+0000 7f17d35fb640 -1 osd.0 0 log_to_monitors true 2024-09-21T06:28:58.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:58 smithi203 ceph-mon[33735]: from='osd.0 [v2:172.21.15.67:6802/2310759946,v1:172.21.15.67:6803/2310759946]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T06:28:58.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:58 smithi203 ceph-mon[33735]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T06:28:58.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:58 smithi067 ceph-mon[31232]: from='osd.0 [v2:172.21.15.67:6802/2310759946,v1:172.21.15.67:6803/2310759946]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T06:28:58.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:58 smithi067 ceph-mon[31232]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T06:28:58.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:58 smithi100 ceph-mon[33401]: from='osd.0 [v2:172.21.15.67:6802/2310759946,v1:172.21.15.67:6803/2310759946]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T06:28:58.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:58 smithi100 ceph-mon[33401]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T06:28:59.104 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-21T06:28:59.128 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi100:vg_nvme/lv_4 2024-09-21T06:28:59.377 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:28:59.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:59 smithi067 ceph-mon[31232]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:59.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:59 smithi067 ceph-mon[31232]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T06:28:59.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:59 smithi067 ceph-mon[31232]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T06:28:59.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:59 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:28:59.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:59 smithi067 ceph-mon[31232]: from='osd.0 [v2:172.21.15.67:6802/2310759946,v1:172.21.15.67:6803/2310759946]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]} : dispatch 2024-09-21T06:28:59.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:59 smithi067 ceph-mon[31232]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]} : dispatch 2024-09-21T06:28:59.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:59 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:59.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:59 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:59.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:59 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:28:59.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:59 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:59.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:59 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:59.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:59 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:28:59.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:28:59 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:59.612 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:59 smithi100 ceph-mon[33401]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:59.612 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:59 smithi100 ceph-mon[33401]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T06:28:59.612 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:59 smithi100 ceph-mon[33401]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T06:28:59.612 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:59 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:28:59.612 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:59 smithi100 ceph-mon[33401]: from='osd.0 [v2:172.21.15.67:6802/2310759946,v1:172.21.15.67:6803/2310759946]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]} : dispatch 2024-09-21T06:28:59.612 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:59 smithi100 ceph-mon[33401]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]} : dispatch 2024-09-21T06:28:59.612 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:59 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:59.612 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:59 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:59.613 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:59 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:28:59.613 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:59 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:59.613 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:59 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:59.613 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:59 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:28:59.613 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:28:59 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:59.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:59 smithi203 ceph-mon[33735]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:28:59.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:59 smithi203 ceph-mon[33735]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T06:28:59.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:59 smithi203 ceph-mon[33735]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T06:28:59.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:59 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:28:59.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:59 smithi203 ceph-mon[33735]: from='osd.0 [v2:172.21.15.67:6802/2310759946,v1:172.21.15.67:6803/2310759946]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]} : dispatch 2024-09-21T06:28:59.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:59 smithi203 ceph-mon[33735]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]} : dispatch 2024-09-21T06:28:59.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:59 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:59.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:59 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:59.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:59 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:28:59.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:59 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:59.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:59 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:28:59.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:59 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:28:59.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:28:59 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:28:59.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.824+0000 7f8223dc5640 1 -- 172.21.15.100:0/3606383515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f821c1036d0 msgr2=0x7f821c103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:59.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.824+0000 7f8223dc5640 1 --2- 172.21.15.100:0/3606383515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f821c1036d0 0x7f821c103ae0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f820c009900 tx=0x7f820c02a480 comp rx=0 tx=0).stop 2024-09-21T06:28:59.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.824+0000 7f8223dc5640 1 -- 172.21.15.100:0/3606383515 shutdown_connections 2024-09-21T06:28:59.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.824+0000 7f8223dc5640 1 --2- 172.21.15.100:0/3606383515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f821c1036d0 0x7f821c103ae0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:59.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.824+0000 7f8223dc5640 1 --2- 172.21.15.100:0/3606383515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f821c104070 0x7f821c10aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:59.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.824+0000 7f8223dc5640 1 --2- 172.21.15.100:0/3606383515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f821c102640 0x7f821c102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:59.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.824+0000 7f8223dc5640 1 -- 172.21.15.100:0/3606383515 >> 172.21.15.100:0/3606383515 conn(0x7f821c0fdfb0 msgr2=0x7f821c100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:28:59.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.824+0000 7f8223dc5640 1 -- 172.21.15.100:0/3606383515 shutdown_connections 2024-09-21T06:28:59.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.824+0000 7f8223dc5640 1 -- 172.21.15.100:0/3606383515 wait complete. 2024-09-21T06:28:59.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.824+0000 7f8223dc5640 1 Processor -- start 2024-09-21T06:28:59.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.825+0000 7f8223dc5640 1 -- start start 2024-09-21T06:28:59.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.825+0000 7f8223dc5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f821c102640 0x7f821c06b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:59.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.825+0000 7f8223dc5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f821c1036d0 0x7f821c19ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:59.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.825+0000 7f8223dc5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f821c104070 0x7f821c1a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:59.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.825+0000 7f8223dc5640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f821c10e120 con 0x7f821c1036d0 2024-09-21T06:28:59.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.825+0000 7f8223dc5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f821c10dfa0 con 0x7f821c102640 2024-09-21T06:28:59.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.825+0000 7f8223dc5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f821c10e2a0 con 0x7f821c104070 2024-09-21T06:28:59.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.825+0000 7f8221b61640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f821c102640 0x7f821c06b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:59.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.825+0000 7f8221b61640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f821c102640 0x7f821c06b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59744/0 (socket says 172.21.15.100:59744) 2024-09-21T06:28:59.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.825+0000 7f8221b61640 1 -- 172.21.15.100:0/3167320907 learned_addr learned my addr 172.21.15.100:0/3167320907 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-21T06:28:59.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.825+0000 7f8221360640 1 --2- 172.21.15.100:0/3167320907 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f821c1036d0 0x7f821c19ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:59.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.825+0000 7f8222362640 1 --2- 172.21.15.100:0/3167320907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f821c104070 0x7f821c1a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:59.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.825+0000 7f8221b61640 1 -- 172.21.15.100:0/3167320907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f821c104070 msgr2=0x7f821c1a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:59.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.825+0000 7f8221b61640 1 --2- 172.21.15.100:0/3167320907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f821c104070 0x7f821c1a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:59.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.825+0000 7f8221b61640 1 -- 172.21.15.100:0/3167320907 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f821c1036d0 msgr2=0x7f821c19ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:28:59.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.825+0000 7f8221b61640 1 --2- 172.21.15.100:0/3167320907 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f821c1036d0 0x7f821c19ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:28:59.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.825+0000 7f8221b61640 1 -- 172.21.15.100:0/3167320907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f821c1a1980 con 0x7f821c102640 2024-09-21T06:28:59.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.826+0000 7f8221b61640 1 --2- 172.21.15.100:0/3167320907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f821c102640 0x7f821c06b6a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f821800d950 tx=0x7f821800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:59.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.832+0000 7f8212ffd640 1 -- 172.21.15.100:0/3167320907 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218017030 con 0x7f821c102640 2024-09-21T06:28:59.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.832+0000 7f8223dc5640 1 -- 172.21.15.100:0/3167320907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f821c1a1c70 con 0x7f821c102640 2024-09-21T06:28:59.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.832+0000 7f8212ffd640 1 -- 172.21.15.100:0/3167320907 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8218004020 con 0x7f821c102640 2024-09-21T06:28:59.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.832+0000 7f8212ffd640 1 -- 172.21.15.100:0/3167320907 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82180055d0 con 0x7f821c102640 2024-09-21T06:28:59.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.832+0000 7f8223dc5640 1 -- 172.21.15.100:0/3167320907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f821c1a21b0 con 0x7f821c102640 2024-09-21T06:28:59.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.833+0000 7f8223dc5640 1 -- 172.21.15.100:0/3167320907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f821c109820 con 0x7f821c102640 2024-09-21T06:28:59.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.835+0000 7f8212ffd640 1 -- 172.21.15.100:0/3167320907 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f8218009a30 con 0x7f821c102640 2024-09-21T06:28:59.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.836+0000 7f8212ffd640 1 --2- 172.21.15.100:0/3167320907 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f82080b6480 0x7f82080b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:28:59.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.836+0000 7f8212ffd640 1 -- 172.21.15.100:0/3167320907 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f82180e57c0 con 0x7f821c102640 2024-09-21T06:28:59.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.836+0000 7f8221360640 1 --2- 172.21.15.100:0/3167320907 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f82080b6480 0x7f82080b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:28:59.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.836+0000 7f8221360640 1 --2- 172.21.15.100:0/3167320907 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f82080b6480 0x7f82080b8990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f820c0057c0 tx=0x7f820c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:28:59.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:28:59.839+0000 7f8212ffd640 1 -- 172.21.15.100:0/3167320907 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f82180adbe0 con 0x7f821c102640 2024-09-21T06:29:00.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:29:00.032+0000 7f8223dc5640 1 -- 172.21.15.100:0/3167320907 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f821c063800 con 0x7f82080b6480 2024-09-21T06:29:00.292 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 21 06:28:59 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-0[46350]: 2024-09-21T06:28:59.823+0000 7f17cf57d640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T06:29:00.292 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 21 06:28:59 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-0[46350]: 2024-09-21T06:28:59.826+0000 7f17c7373640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:29:00.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:00 smithi100 ceph-mon[33401]: Detected new or changed devices on smithi067 2024-09-21T06:29:00.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:00 smithi100 ceph-mon[33401]: Adjusting osd_memory_target on smithi067 to 17088M 2024-09-21T06:29:00.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:00 smithi100 ceph-mon[33401]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-21T06:29:00.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:00 smithi100 ceph-mon[33401]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T06:29:00.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:00 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:29:00.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:00 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:29:00.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:00 smithi100 ceph-mon[33401]: from='osd.0 ' entity='osd.0' 2024-09-21T06:29:00.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:00 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:29:00.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:00 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:29:00.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:00 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:29:00.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:00 smithi203 ceph-mon[33735]: Detected new or changed devices on smithi067 2024-09-21T06:29:00.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:00 smithi203 ceph-mon[33735]: Adjusting osd_memory_target on smithi067 to 17088M 2024-09-21T06:29:00.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:00 smithi203 ceph-mon[33735]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-21T06:29:00.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:00 smithi203 ceph-mon[33735]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T06:29:00.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:00 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:29:00.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:00 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:29:00.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:00 smithi203 ceph-mon[33735]: from='osd.0 ' entity='osd.0' 2024-09-21T06:29:00.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:00 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:29:00.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:00 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:29:00.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:00 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:29:00.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:00 smithi067 ceph-mon[31232]: Detected new or changed devices on smithi067 2024-09-21T06:29:00.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:00 smithi067 ceph-mon[31232]: Adjusting osd_memory_target on smithi067 to 17088M 2024-09-21T06:29:00.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:00 smithi067 ceph-mon[31232]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-21T06:29:00.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:00 smithi067 ceph-mon[31232]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T06:29:00.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:00 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:29:00.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:00 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:29:00.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:00 smithi067 ceph-mon[31232]: from='osd.0 ' entity='osd.0' 2024-09-21T06:29:00.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:00 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:29:00.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:00 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:29:00.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:00 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:29:01.509 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:01 smithi100 ceph-mon[33401]: purged_snaps scrub starts 2024-09-21T06:29:01.509 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:01 smithi100 ceph-mon[33401]: purged_snaps scrub ok 2024-09-21T06:29:01.509 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:01 smithi100 ceph-mon[33401]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:29:01.509 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:01 smithi100 ceph-mon[33401]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:29:01.509 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:01 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:29:01.509 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:01 smithi100 ceph-mon[33401]: osd.0 [v2:172.21.15.67:6802/2310759946,v1:172.21.15.67:6803/2310759946] boot 2024-09-21T06:29:01.509 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:01 smithi100 ceph-mon[33401]: osdmap e8: 1 total, 1 up, 1 in 2024-09-21T06:29:01.509 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:01 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:29:01.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:01 smithi203 ceph-mon[33735]: purged_snaps scrub starts 2024-09-21T06:29:01.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:01 smithi203 ceph-mon[33735]: purged_snaps scrub ok 2024-09-21T06:29:01.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:01 smithi203 ceph-mon[33735]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:29:01.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:01 smithi203 ceph-mon[33735]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:29:01.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:01 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:29:01.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:01 smithi203 ceph-mon[33735]: osd.0 [v2:172.21.15.67:6802/2310759946,v1:172.21.15.67:6803/2310759946] boot 2024-09-21T06:29:01.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:01 smithi203 ceph-mon[33735]: osdmap e8: 1 total, 1 up, 1 in 2024-09-21T06:29:01.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:01 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:29:01.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:01 smithi067 ceph-mon[31232]: purged_snaps scrub starts 2024-09-21T06:29:01.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:01 smithi067 ceph-mon[31232]: purged_snaps scrub ok 2024-09-21T06:29:01.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:01 smithi067 ceph-mon[31232]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:29:01.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:01 smithi067 ceph-mon[31232]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:29:01.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:01 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:29:01.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:01 smithi067 ceph-mon[31232]: osd.0 [v2:172.21.15.67:6802/2310759946,v1:172.21.15.67:6803/2310759946] boot 2024-09-21T06:29:01.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:01 smithi067 ceph-mon[31232]: osdmap e8: 1 total, 1 up, 1 in 2024-09-21T06:29:01.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:01 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:29:03.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:02 smithi100 ceph-mon[33401]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:03.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:02 smithi100 ceph-mon[33401]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T06:29:03.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:02 smithi100 ceph-mon[33401]: from='client.? 172.21.15.100:0/1083002470' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "84e8d0c2-1c45-4673-ad49-f61df667d7f7"} : dispatch 2024-09-21T06:29:03.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:02 smithi100 ceph-mon[33401]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "84e8d0c2-1c45-4673-ad49-f61df667d7f7"} : dispatch 2024-09-21T06:29:03.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:02 smithi100 ceph-mon[33401]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84e8d0c2-1c45-4673-ad49-f61df667d7f7"}]': finished 2024-09-21T06:29:03.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:02 smithi100 ceph-mon[33401]: osdmap e10: 2 total, 1 up, 2 in 2024-09-21T06:29:03.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:02 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:29:03.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:02 smithi100 ceph-mon[33401]: from='client.? 172.21.15.100:0/2897313704' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:29:03.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:02 smithi203 ceph-mon[33735]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:03.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:02 smithi203 ceph-mon[33735]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T06:29:03.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:02 smithi203 ceph-mon[33735]: from='client.? 172.21.15.100:0/1083002470' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "84e8d0c2-1c45-4673-ad49-f61df667d7f7"} : dispatch 2024-09-21T06:29:03.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:02 smithi203 ceph-mon[33735]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "84e8d0c2-1c45-4673-ad49-f61df667d7f7"} : dispatch 2024-09-21T06:29:03.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:02 smithi203 ceph-mon[33735]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84e8d0c2-1c45-4673-ad49-f61df667d7f7"}]': finished 2024-09-21T06:29:03.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:02 smithi203 ceph-mon[33735]: osdmap e10: 2 total, 1 up, 2 in 2024-09-21T06:29:03.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:02 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:29:03.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:02 smithi203 ceph-mon[33735]: from='client.? 172.21.15.100:0/2897313704' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:29:03.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:02 smithi067 ceph-mon[31232]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:03.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:02 smithi067 ceph-mon[31232]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T06:29:03.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:02 smithi067 ceph-mon[31232]: from='client.? 172.21.15.100:0/1083002470' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "84e8d0c2-1c45-4673-ad49-f61df667d7f7"} : dispatch 2024-09-21T06:29:03.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:02 smithi067 ceph-mon[31232]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "84e8d0c2-1c45-4673-ad49-f61df667d7f7"} : dispatch 2024-09-21T06:29:03.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:02 smithi067 ceph-mon[31232]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84e8d0c2-1c45-4673-ad49-f61df667d7f7"}]': finished 2024-09-21T06:29:03.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:02 smithi067 ceph-mon[31232]: osdmap e10: 2 total, 1 up, 2 in 2024-09-21T06:29:03.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:02 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:29:03.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:02 smithi067 ceph-mon[31232]: from='client.? 172.21.15.100:0/2897313704' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:29:05.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:04 smithi203 ceph-mon[33735]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:05.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:04 smithi067 ceph-mon[31232]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:05.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:04 smithi100 ceph-mon[33401]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:07.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:06 smithi100 ceph-mon[33401]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:07.094 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:06 smithi067 ceph-mon[31232]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:07.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:06 smithi203 ceph-mon[33735]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:09.033 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:08 smithi100 ceph-mon[33401]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:09.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:08 smithi203 ceph-mon[33735]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:09.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:08 smithi067 ceph-mon[31232]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:10.951 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:10 smithi100 ceph-mon[33401]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:10.951 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:10 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T06:29:10.951 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:10 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:29:10.951 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:10 smithi100 ceph-mon[33401]: Deploying daemon osd.1 on smithi100 2024-09-21T06:29:11.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:10 smithi203 ceph-mon[33735]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:11.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:10 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T06:29:11.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:10 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:29:11.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:10 smithi203 ceph-mon[33735]: Deploying daemon osd.1 on smithi100 2024-09-21T06:29:11.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:10 smithi067 ceph-mon[31232]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:11.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:10 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T06:29:11.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:10 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:29:11.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:10 smithi067 ceph-mon[31232]: Deploying daemon osd.1 on smithi100 2024-09-21T06:29:12.972 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:12 smithi100 ceph-mon[33401]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:13.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:12 smithi203 ceph-mon[33735]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:13.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:12 smithi067 ceph-mon[31232]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:15.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:14 smithi203 ceph-mon[33735]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:15.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:14 smithi067 ceph-mon[31232]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:15.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:14 smithi100 ceph-mon[33401]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:17.110 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:16 smithi100 ceph-mon[33401]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:17.110 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:16 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:29:17.110 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:16 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:17.111 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:16 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:17.157 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:16 smithi067 ceph-mon[31232]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:17.157 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:16 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:29:17.157 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:16 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:17.157 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:16 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:17.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:16 smithi203 ceph-mon[33735]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:17.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:16 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:29:17.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:16 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:17.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:16 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:19.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:18 smithi100 ceph-mon[33401]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:19.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:18 smithi203 ceph-mon[33735]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:19.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:18 smithi067 ceph-mon[31232]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:20.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:20 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:20.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:20 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:20.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:20 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:29:20.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:20 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:29:20.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:20 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:20.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:29:20.360+0000 7f8212ffd640 1 -- 172.21.15.100:0/3167320907 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f821c063800 con 0x7f82080b6480 2024-09-21T06:29:20.362 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 1 on host 'smithi100' 2024-09-21T06:29:20.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:29:20.362+0000 7f8223dc5640 1 -- 172.21.15.100:0/3167320907 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f82080b6480 msgr2=0x7f82080b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:20.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:29:20.362+0000 7f8223dc5640 1 --2- 172.21.15.100:0/3167320907 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f82080b6480 0x7f82080b8990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f820c0057c0 tx=0x7f820c000bc0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:29:20.362+0000 7f8223dc5640 1 -- 172.21.15.100:0/3167320907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f821c102640 msgr2=0x7f821c06b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:20.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:29:20.362+0000 7f8223dc5640 1 --2- 172.21.15.100:0/3167320907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f821c102640 0x7f821c06b6a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f821800d950 tx=0x7f821800de10 comp rx=0 tx=0).stop 2024-09-21T06:29:20.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:29:20.362+0000 7f8223dc5640 1 -- 172.21.15.100:0/3167320907 shutdown_connections 2024-09-21T06:29:20.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:29:20.362+0000 7f8223dc5640 1 --2- 172.21.15.100:0/3167320907 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f82080b6480 0x7f82080b8990 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:29:20.362+0000 7f8223dc5640 1 --2- 172.21.15.100:0/3167320907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f821c104070 0x7f821c1a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:29:20.362+0000 7f8223dc5640 1 --2- 172.21.15.100:0/3167320907 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f821c1036d0 0x7f821c19ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:29:20.362+0000 7f8223dc5640 1 --2- 172.21.15.100:0/3167320907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f821c102640 0x7f821c06b6a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:20.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:29:20.362+0000 7f8223dc5640 1 -- 172.21.15.100:0/3167320907 >> 172.21.15.100:0/3167320907 conn(0x7f821c0fdfb0 msgr2=0x7f821c1069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:20.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:29:20.362+0000 7f8223dc5640 1 -- 172.21.15.100:0/3167320907 shutdown_connections 2024-09-21T06:29:20.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:29:20.362+0000 7f8223dc5640 1 -- 172.21.15.100:0/3167320907 wait complete. 2024-09-21T06:29:20.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:20 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:20.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:20 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:20.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:20 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:29:20.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:20 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:29:20.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:20 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:20.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:20 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:20.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:20 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:20.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:20 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:29:20.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:20 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:29:20.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:20 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:21.065 DEBUG:teuthology.orchestra.run.smithi100:osd.1> sudo journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.1.service 2024-09-21T06:29:21.067 INFO:tasks.cephadm:Deploying osd.2 on smithi203 with /dev/vg_nvme/lv_4... 2024-09-21T06:29:21.067 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T06:29:21.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:21 smithi100 ceph-mon[33401]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:21.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:21 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:29:21.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:21 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:21.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:21 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:21.322 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:29:21.442 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:21 smithi203 ceph-mon[33735]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:21.442 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:21 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:29:21.442 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:21 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:21.442 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:21 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:21.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:21 smithi067 ceph-mon[31232]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:21.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:21 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:29:21.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:21 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:21.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:21 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:22.932 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 21 06:29:22 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-1[37274]: 2024-09-21T06:29:22.632+0000 7f0b17d1f640 -1 osd.1 0 log_to_monitors true 2024-09-21T06:29:23.212 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:23 smithi100 ceph-mon[33401]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:23.212 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:23 smithi100 ceph-mon[33401]: from='osd.1 [v2:172.21.15.100:6800/161807364,v1:172.21.15.100:6801/161807364]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T06:29:23.212 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:23 smithi100 ceph-mon[33401]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T06:29:23.462 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:23 smithi203 ceph-mon[33735]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:23.462 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:23 smithi203 ceph-mon[33735]: from='osd.1 [v2:172.21.15.100:6800/161807364,v1:172.21.15.100:6801/161807364]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T06:29:23.463 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:23 smithi203 ceph-mon[33735]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T06:29:23.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:23 smithi067 ceph-mon[31232]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:23.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:23 smithi067 ceph-mon[31232]: from='osd.1 [v2:172.21.15.100:6800/161807364,v1:172.21.15.100:6801/161807364]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T06:29:23.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:23 smithi067 ceph-mon[31232]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T06:29:23.768 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T06:29:23.794 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi203:vg_nvme/lv_4 2024-09-21T06:29:24.048 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:29:24.386 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:24 smithi203 ceph-mon[33735]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T06:29:24.386 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:24 smithi203 ceph-mon[33735]: from='osd.1 [v2:172.21.15.100:6800/161807364,v1:172.21.15.100:6801/161807364]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]} : dispatch 2024-09-21T06:29:24.386 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:24 smithi203 ceph-mon[33735]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T06:29:24.386 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:24 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:29:24.386 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:24 smithi203 ceph-mon[33735]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]} : dispatch 2024-09-21T06:29:24.386 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:24 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:24.386 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:24 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:24.386 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:24 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:29:24.387 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:24 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:24.387 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:24 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:29:24.387 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:24 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:29:24.387 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:24 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:24.387 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:24 smithi203 ceph-mon[33735]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-21T06:29:24.387 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:24 smithi203 ceph-mon[33735]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T06:29:24.387 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:24 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:29:24.482 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.478+0000 7f9d9cff7640 1 -- 172.21.15.203:0/2127735567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d98104070 msgr2=0x7f9d9810aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:24.482 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.478+0000 7f9d9cff7640 1 --2- 172.21.15.203:0/2127735567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d98104070 0x7f9d9810aa10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9d90009ca0 tx=0x7f9d9002a950 comp rx=0 tx=0).stop 2024-09-21T06:29:24.482 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.478+0000 7f9d9cff7640 1 -- 172.21.15.203:0/2127735567 shutdown_connections 2024-09-21T06:29:24.482 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.478+0000 7f9d9cff7640 1 --2- 172.21.15.203:0/2127735567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d98104070 0x7f9d9810aa10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:24.482 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.478+0000 7f9d9cff7640 1 --2- 172.21.15.203:0/2127735567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d981036d0 0x7f9d98103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:24.482 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.478+0000 7f9d9cff7640 1 --2- 172.21.15.203:0/2127735567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d98102640 0x7f9d98102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:24.482 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.478+0000 7f9d9cff7640 1 -- 172.21.15.203:0/2127735567 >> 172.21.15.203:0/2127735567 conn(0x7f9d980fdfb0 msgr2=0x7f9d98100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:24.483 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.478+0000 7f9d9cff7640 1 -- 172.21.15.203:0/2127735567 shutdown_connections 2024-09-21T06:29:24.483 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.478+0000 7f9d9cff7640 1 -- 172.21.15.203:0/2127735567 wait complete. 2024-09-21T06:29:24.483 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.479+0000 7f9d9cff7640 1 Processor -- start 2024-09-21T06:29:24.483 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.479+0000 7f9d9cff7640 1 -- start start 2024-09-21T06:29:24.483 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.479+0000 7f9d9cff7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d98102640 0x7f9d9806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:24.483 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.479+0000 7f9d9cff7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d981036d0 0x7f9d9819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:24.483 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.479+0000 7f9d9cff7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d98104070 0x7f9d981a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:24.483 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.479+0000 7f9d9cff7640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9810e120 con 0x7f9d981036d0 2024-09-21T06:29:24.483 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.479+0000 7f9d9cff7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9810dfa0 con 0x7f9d98104070 2024-09-21T06:29:24.483 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.479+0000 7f9d9cff7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9810e2a0 con 0x7f9d98102640 2024-09-21T06:29:24.483 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.479+0000 7f9d9659c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d98102640 0x7f9d9806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:24.483 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.479+0000 7f9d9659c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d98102640 0x7f9d9806b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:59466/0 (socket says 172.21.15.203:59466) 2024-09-21T06:29:24.483 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.479+0000 7f9d9659c640 1 -- 172.21.15.203:0/199315450 learned_addr learned my addr 172.21.15.203:0/199315450 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-09-21T06:29:24.484 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.480+0000 7f9d95d9b640 1 --2- 172.21.15.203:0/199315450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d981036d0 0x7f9d9819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:24.484 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.480+0000 7f9d96d9d640 1 --2- 172.21.15.203:0/199315450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d98104070 0x7f9d981a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:24.484 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.480+0000 7f9d9659c640 1 -- 172.21.15.203:0/199315450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d98104070 msgr2=0x7f9d981a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:24.484 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.480+0000 7f9d9659c640 1 --2- 172.21.15.203:0/199315450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d98104070 0x7f9d981a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:24.484 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.480+0000 7f9d9659c640 1 -- 172.21.15.203:0/199315450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d981036d0 msgr2=0x7f9d9819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:24.484 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.480+0000 7f9d9659c640 1 --2- 172.21.15.203:0/199315450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d981036d0 0x7f9d9819eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:24.484 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.480+0000 7f9d9659c640 1 -- 172.21.15.203:0/199315450 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d981a1820 con 0x7f9d98102640 2024-09-21T06:29:24.484 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.480+0000 7f9d96d9d640 1 --2- 172.21.15.203:0/199315450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d98104070 0x7f9d981a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:29:24.484 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.480+0000 7f9d9659c640 1 --2- 172.21.15.203:0/199315450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d98102640 0x7f9d9806b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9d8800c920 tx=0x7f9d8800cd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:24.484 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.480+0000 7f9d8f7fe640 1 -- 172.21.15.203:0/199315450 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d88007bb0 con 0x7f9d98102640 2024-09-21T06:29:24.484 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.480+0000 7f9d8f7fe640 1 -- 172.21.15.203:0/199315450 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d88007d50 con 0x7f9d98102640 2024-09-21T06:29:24.484 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.480+0000 7f9d9cff7640 1 -- 172.21.15.203:0/199315450 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d981a1b10 con 0x7f9d98102640 2024-09-21T06:29:24.484 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.480+0000 7f9d8f7fe640 1 -- 172.21.15.203:0/199315450 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d88018640 con 0x7f9d98102640 2024-09-21T06:29:24.484 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.480+0000 7f9d9cff7640 1 -- 172.21.15.203:0/199315450 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d981a20a0 con 0x7f9d98102640 2024-09-21T06:29:24.485 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.481+0000 7f9d9cff7640 1 -- 172.21.15.203:0/199315450 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d98109800 con 0x7f9d98102640 2024-09-21T06:29:24.485 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.483+0000 7f9d8f7fe640 1 -- 172.21.15.203:0/199315450 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f9d8800f450 con 0x7f9d98102640 2024-09-21T06:29:24.485 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.484+0000 7f9d8f7fe640 1 --2- 172.21.15.203:0/199315450 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9d840b6500 0x7f9d840b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:24.486 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.484+0000 7f9d8f7fe640 1 -- 172.21.15.203:0/199315450 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7f9d880e5ca0 con 0x7f9d98102640 2024-09-21T06:29:24.486 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.484+0000 7f9d95d9b640 1 --2- 172.21.15.203:0/199315450 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9d840b6500 0x7f9d840b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:24.486 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.484+0000 7f9d95d9b640 1 --2- 172.21.15.203:0/199315450 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9d840b6500 0x7f9d840b8a10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000abe0 tx=0x7f9d8000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:24.488 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.487+0000 7f9d8f7fe640 1 -- 172.21.15.203:0/199315450 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9d88014030 con 0x7f9d98102640 2024-09-21T06:29:24.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:24 smithi067 ceph-mon[31232]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T06:29:24.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:24 smithi067 ceph-mon[31232]: from='osd.1 [v2:172.21.15.100:6800/161807364,v1:172.21.15.100:6801/161807364]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]} : dispatch 2024-09-21T06:29:24.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:24 smithi067 ceph-mon[31232]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T06:29:24.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:24 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:29:24.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:24 smithi067 ceph-mon[31232]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]} : dispatch 2024-09-21T06:29:24.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:24 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:24.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:24 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:24.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:24 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:29:24.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:24 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:24.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:24 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:29:24.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:24 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:29:24.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:24 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:24.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:24 smithi067 ceph-mon[31232]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-21T06:29:24.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:24 smithi067 ceph-mon[31232]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T06:29:24.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:24 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:29:24.553 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 21 06:29:24 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-1[37274]: 2024-09-21T06:29:24.104+0000 7f0b13ca1640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T06:29:24.553 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 21 06:29:24 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-1[37274]: 2024-09-21T06:29:24.108+0000 7f0b0fa9f640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:29:24.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:24 smithi100 ceph-mon[33401]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T06:29:24.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:24 smithi100 ceph-mon[33401]: from='osd.1 [v2:172.21.15.100:6800/161807364,v1:172.21.15.100:6801/161807364]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]} : dispatch 2024-09-21T06:29:24.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:24 smithi100 ceph-mon[33401]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T06:29:24.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:24 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:29:24.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:24 smithi100 ceph-mon[33401]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]} : dispatch 2024-09-21T06:29:24.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:24 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:24.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:24 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:24.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:24 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:29:24.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:24 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:24.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:24 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:29:24.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:24 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:29:24.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:24 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:24.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:24 smithi100 ceph-mon[33401]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-21T06:29:24.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:24 smithi100 ceph-mon[33401]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T06:29:24.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:24 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:29:24.674 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:24.672+0000 7f9d9cff7640 1 -- 172.21.15.203:0/199315450 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f9d98063800 con 0x7f9d840b6500 2024-09-21T06:29:25.355 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:25 smithi203 ceph-mon[33735]: Detected new or changed devices on smithi100 2024-09-21T06:29:25.356 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:25 smithi203 ceph-mon[33735]: Adjusting osd_memory_target on smithi100 to 17087M 2024-09-21T06:29:25.356 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:25 smithi203 ceph-mon[33735]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:25.356 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:25 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:29:25.356 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:25 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:29:25.356 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:25 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:29:25.356 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:25 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:29:25.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:25 smithi067 ceph-mon[31232]: Detected new or changed devices on smithi100 2024-09-21T06:29:25.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:25 smithi067 ceph-mon[31232]: Adjusting osd_memory_target on smithi100 to 17087M 2024-09-21T06:29:25.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:25 smithi067 ceph-mon[31232]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:25.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:25 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:29:25.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:25 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:29:25.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:25 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:29:25.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:25 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:29:25.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:25 smithi100 ceph-mon[33401]: Detected new or changed devices on smithi100 2024-09-21T06:29:25.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:25 smithi100 ceph-mon[33401]: Adjusting osd_memory_target on smithi100 to 17087M 2024-09-21T06:29:25.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:25 smithi100 ceph-mon[33401]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:25.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:25 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:29:25.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:25 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:29:25.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:25 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:29:25.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:25 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:29:26.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:26 smithi203 ceph-mon[33735]: purged_snaps scrub starts 2024-09-21T06:29:26.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:26 smithi203 ceph-mon[33735]: purged_snaps scrub ok 2024-09-21T06:29:26.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:26 smithi203 ceph-mon[33735]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:29:26.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:26 smithi203 ceph-mon[33735]: osd.1 [v2:172.21.15.100:6800/161807364,v1:172.21.15.100:6801/161807364] boot 2024-09-21T06:29:26.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:26 smithi203 ceph-mon[33735]: osdmap e13: 2 total, 2 up, 2 in 2024-09-21T06:29:26.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:26 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:29:26.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:26 smithi067 ceph-mon[31232]: purged_snaps scrub starts 2024-09-21T06:29:26.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:26 smithi067 ceph-mon[31232]: purged_snaps scrub ok 2024-09-21T06:29:26.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:26 smithi067 ceph-mon[31232]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:29:26.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:26 smithi067 ceph-mon[31232]: osd.1 [v2:172.21.15.100:6800/161807364,v1:172.21.15.100:6801/161807364] boot 2024-09-21T06:29:26.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:26 smithi067 ceph-mon[31232]: osdmap e13: 2 total, 2 up, 2 in 2024-09-21T06:29:26.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:26 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:29:26.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:26 smithi100 ceph-mon[33401]: purged_snaps scrub starts 2024-09-21T06:29:26.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:26 smithi100 ceph-mon[33401]: purged_snaps scrub ok 2024-09-21T06:29:26.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:26 smithi100 ceph-mon[33401]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:29:26.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:26 smithi100 ceph-mon[33401]: osd.1 [v2:172.21.15.100:6800/161807364,v1:172.21.15.100:6801/161807364] boot 2024-09-21T06:29:26.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:26 smithi100 ceph-mon[33401]: osdmap e13: 2 total, 2 up, 2 in 2024-09-21T06:29:26.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:26 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:29:27.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:27 smithi203 ceph-mon[33735]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:27.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:27 smithi203 ceph-mon[33735]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T06:29:27.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:27 smithi203 ceph-mon[33735]: from='client.? 172.21.15.203:0/2009752120' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e9a4d919-a779-4a1b-9b5b-d03a65f28ddf"} : dispatch 2024-09-21T06:29:27.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:27 smithi203 ceph-mon[33735]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e9a4d919-a779-4a1b-9b5b-d03a65f28ddf"} : dispatch 2024-09-21T06:29:27.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:27 smithi203 ceph-mon[33735]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9a4d919-a779-4a1b-9b5b-d03a65f28ddf"}]': finished 2024-09-21T06:29:27.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:27 smithi203 ceph-mon[33735]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T06:29:27.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:27 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:29:27.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:27 smithi067 ceph-mon[31232]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:27.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:27 smithi067 ceph-mon[31232]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T06:29:27.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:27 smithi067 ceph-mon[31232]: from='client.? 172.21.15.203:0/2009752120' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e9a4d919-a779-4a1b-9b5b-d03a65f28ddf"} : dispatch 2024-09-21T06:29:27.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:27 smithi067 ceph-mon[31232]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e9a4d919-a779-4a1b-9b5b-d03a65f28ddf"} : dispatch 2024-09-21T06:29:27.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:27 smithi067 ceph-mon[31232]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9a4d919-a779-4a1b-9b5b-d03a65f28ddf"}]': finished 2024-09-21T06:29:27.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:27 smithi067 ceph-mon[31232]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T06:29:27.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:27 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:29:27.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:27 smithi100 ceph-mon[33401]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:29:27.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:27 smithi100 ceph-mon[33401]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T06:29:27.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:27 smithi100 ceph-mon[33401]: from='client.? 172.21.15.203:0/2009752120' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e9a4d919-a779-4a1b-9b5b-d03a65f28ddf"} : dispatch 2024-09-21T06:29:27.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:27 smithi100 ceph-mon[33401]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e9a4d919-a779-4a1b-9b5b-d03a65f28ddf"} : dispatch 2024-09-21T06:29:27.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:27 smithi100 ceph-mon[33401]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9a4d919-a779-4a1b-9b5b-d03a65f28ddf"}]': finished 2024-09-21T06:29:27.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:27 smithi100 ceph-mon[33401]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T06:29:27.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:27 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:29:28.418 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:28 smithi100 ceph-mon[33401]: from='client.? 172.21.15.203:0/1486074271' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:29:28.484 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:28 smithi203 ceph-mon[33735]: from='client.? 172.21.15.203:0/1486074271' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:29:28.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:28 smithi067 ceph-mon[31232]: from='client.? 172.21.15.203:0/1486074271' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:29:29.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:29 smithi203 ceph-mon[33735]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:29.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:29 smithi067 ceph-mon[31232]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:29.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:29 smithi100 ceph-mon[33401]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:31.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:31 smithi203 ceph-mon[33735]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:31.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:31 smithi067 ceph-mon[31232]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:31.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:31 smithi100 ceph-mon[33401]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:33.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:33 smithi203 ceph-mon[33735]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:33.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:33 smithi067 ceph-mon[31232]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:33.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:33 smithi100 ceph-mon[33401]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:35.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:35 smithi203 ceph-mon[33735]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:35.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:35 smithi067 ceph-mon[31232]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:35.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:35 smithi100 ceph-mon[33401]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:36.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:36 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T06:29:36.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:36 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:29:36.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:36 smithi203 ceph-mon[33735]: Deploying daemon osd.2 on smithi203 2024-09-21T06:29:36.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:36 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T06:29:36.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:36 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:29:36.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:36 smithi067 ceph-mon[31232]: Deploying daemon osd.2 on smithi203 2024-09-21T06:29:36.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:36 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T06:29:36.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:36 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:29:36.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:36 smithi100 ceph-mon[33401]: Deploying daemon osd.2 on smithi203 2024-09-21T06:29:37.408 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:37 smithi100 ceph-mon[33401]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:37.443 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:37 smithi067 ceph-mon[31232]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:37.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:37 smithi203 ceph-mon[33735]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:39.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:39 smithi203 ceph-mon[33735]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:39.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:39 smithi067 ceph-mon[31232]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:39.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:39 smithi100 ceph-mon[33401]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:41.429 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:41 smithi203 ceph-mon[33735]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:41.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:41 smithi067 ceph-mon[31232]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:41.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:41 smithi100 ceph-mon[33401]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:42.370 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:42 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:29:42.370 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:42 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:42.370 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:42 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:42.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:42 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:29:42.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:42 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:42.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:42 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:42.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:42 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:29:42.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:42 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:42.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:42 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:43.351 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:43 smithi203 ceph-mon[33735]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:43.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:43 smithi067 ceph-mon[31232]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:43.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:43 smithi100 ceph-mon[33401]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:45.209 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:45.206+0000 7f9d8f7fe640 1 -- 172.21.15.203:0/199315450 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9d98063800 con 0x7f9d840b6500 2024-09-21T06:29:45.209 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 2 on host 'smithi203' 2024-09-21T06:29:45.212 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:45.209+0000 7f9d9cff7640 1 -- 172.21.15.203:0/199315450 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9d840b6500 msgr2=0x7f9d840b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:45.212 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:45.209+0000 7f9d9cff7640 1 --2- 172.21.15.203:0/199315450 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9d840b6500 0x7f9d840b8a10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000abe0 tx=0x7f9d8000a2d0 comp rx=0 tx=0).stop 2024-09-21T06:29:45.212 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:45.209+0000 7f9d9cff7640 1 -- 172.21.15.203:0/199315450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d98102640 msgr2=0x7f9d9806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:45.212 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:45.209+0000 7f9d9cff7640 1 --2- 172.21.15.203:0/199315450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d98102640 0x7f9d9806b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9d8800c920 tx=0x7f9d8800cd20 comp rx=0 tx=0).stop 2024-09-21T06:29:45.212 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:45.209+0000 7f9d9cff7640 1 -- 172.21.15.203:0/199315450 shutdown_connections 2024-09-21T06:29:45.212 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:45.209+0000 7f9d9cff7640 1 --2- 172.21.15.203:0/199315450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d98104070 0x7f9d981a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:45.212 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:45.209+0000 7f9d9cff7640 1 --2- 172.21.15.203:0/199315450 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9d840b6500 0x7f9d840b8a10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:45.212 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:45.209+0000 7f9d9cff7640 1 --2- 172.21.15.203:0/199315450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d981036d0 0x7f9d9819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:45.212 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:45.209+0000 7f9d9cff7640 1 --2- 172.21.15.203:0/199315450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d98102640 0x7f9d9806b6c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:45.212 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:45.209+0000 7f9d9cff7640 1 -- 172.21.15.203:0/199315450 >> 172.21.15.203:0/199315450 conn(0x7f9d980fdfb0 msgr2=0x7f9d981069d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:45.212 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:45.209+0000 7f9d9cff7640 1 -- 172.21.15.203:0/199315450 shutdown_connections 2024-09-21T06:29:45.212 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:29:45.209+0000 7f9d9cff7640 1 -- 172.21.15.203:0/199315450 wait complete. 2024-09-21T06:29:45.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:45 smithi203 ceph-mon[33735]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:45.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:45 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:45.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:45 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:45.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:45 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:29:45.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:45 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:29:45.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:45 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:45.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:45 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:29:45.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:45 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:45.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:45 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:45.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:45 smithi067 ceph-mon[31232]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:45.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:45 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:45.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:45 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:45.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:45 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:29:45.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:45 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:29:45.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:45 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:45.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:45 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:29:45.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:45 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:45.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:45 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:45.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:45 smithi100 ceph-mon[33401]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:45.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:45 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:45.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:45 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:45.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:45 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:29:45.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:45 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:29:45.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:45 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:45.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:45 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:29:45.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:45 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:45.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:45 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:45.865 DEBUG:teuthology.orchestra.run.smithi203:osd.2> sudo journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.2.service 2024-09-21T06:29:45.867 INFO:tasks.cephadm:Deploying osd.3 on smithi067 with /dev/vg_nvme/lv_3... 2024-09-21T06:29:45.867 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T06:29:46.133 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:29:47.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:47 smithi203 ceph-mon[33735]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:47.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:47 smithi203 ceph-mon[33735]: from='osd.2 [v2:172.21.15.203:6800/2754787169,v1:172.21.15.203:6801/2754787169]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T06:29:47.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:47 smithi203 ceph-mon[33735]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T06:29:47.483 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 06:29:47 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-2[38108]: 2024-09-21T06:29:47.253+0000 7f0b77eae640 -1 osd.2 0 log_to_monitors true 2024-09-21T06:29:47.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:47 smithi067 ceph-mon[31232]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:47.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:47 smithi067 ceph-mon[31232]: from='osd.2 [v2:172.21.15.203:6800/2754787169,v1:172.21.15.203:6801/2754787169]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T06:29:47.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:47 smithi067 ceph-mon[31232]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T06:29:47.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:47 smithi100 ceph-mon[33401]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:47.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:47 smithi100 ceph-mon[33401]: from='osd.2 [v2:172.21.15.203:6800/2754787169,v1:172.21.15.203:6801/2754787169]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T06:29:47.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:47 smithi100 ceph-mon[33401]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T06:29:48.454 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 06:29:48 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-2[38108]: 2024-09-21T06:29:48.279+0000 7f0b73e30640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T06:29:48.454 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 06:29:48 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-2[38108]: 2024-09-21T06:29:48.283+0000 7f0b6fc2e640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:29:48.455 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:48 smithi203 ceph-mon[33735]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T06:29:48.455 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:48 smithi203 ceph-mon[33735]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T06:29:48.455 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:48 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:29:48.455 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:48 smithi203 ceph-mon[33735]: from='osd.2 [v2:172.21.15.203:6800/2754787169,v1:172.21.15.203:6801/2754787169]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T06:29:48.455 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:48 smithi203 ceph-mon[33735]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T06:29:48.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:48 smithi067 ceph-mon[31232]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T06:29:48.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:48 smithi067 ceph-mon[31232]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T06:29:48.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:48 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:29:48.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:48 smithi067 ceph-mon[31232]: from='osd.2 [v2:172.21.15.203:6800/2754787169,v1:172.21.15.203:6801/2754787169]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T06:29:48.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:48 smithi067 ceph-mon[31232]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T06:29:48.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:48 smithi100 ceph-mon[33401]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T06:29:48.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:48 smithi100 ceph-mon[33401]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T06:29:48.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:48 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:29:48.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:48 smithi100 ceph-mon[33401]: from='osd.2 [v2:172.21.15.203:6800/2754787169,v1:172.21.15.203:6801/2754787169]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T06:29:48.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:48 smithi100 ceph-mon[33401]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T06:29:48.732 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:29:48.758 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi067:vg_nvme/lv_3 2024-09-21T06:29:49.021 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:29:49.407 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:49 smithi067 ceph-mon[31232]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:49.408 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:49 smithi067 ceph-mon[31232]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-21T06:29:49.408 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:49 smithi067 ceph-mon[31232]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T06:29:49.408 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:49 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:29:49.408 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:49 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:29:49.408 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:49 smithi067 ceph-mon[31232]: Detected new or changed devices on smithi203 2024-09-21T06:29:49.408 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:49 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:49.408 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:49 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:49.408 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:49 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:29:49.408 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:49 smithi067 ceph-mon[31232]: Adjusting osd_memory_target on smithi203 to 17088M 2024-09-21T06:29:49.408 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:49 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:49.408 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:49 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:29:49.408 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:49 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:29:49.408 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:49 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:49.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:49 smithi100 ceph-mon[33401]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:49.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:49 smithi100 ceph-mon[33401]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-21T06:29:49.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:49 smithi100 ceph-mon[33401]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T06:29:49.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:49 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:29:49.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:49 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:29:49.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:49 smithi100 ceph-mon[33401]: Detected new or changed devices on smithi203 2024-09-21T06:29:49.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:49 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:49.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:49 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:49.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:49 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:29:49.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:49 smithi100 ceph-mon[33401]: Adjusting osd_memory_target on smithi203 to 17088M 2024-09-21T06:29:49.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:49 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:49.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:49 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:29:49.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:49 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:29:49.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:49 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:49.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.504+0000 7f89e6d4e640 1 -- 172.21.15.67:0/3074099805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89e01036f0 msgr2=0x7f89e0103b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:49.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.504+0000 7f89e6d4e640 1 --2- 172.21.15.67:0/3074099805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89e01036f0 0x7f89e0103b00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f89cc009900 tx=0x7f89cc02a480 comp rx=0 tx=0).stop 2024-09-21T06:29:49.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.504+0000 7f89e6d4e640 1 -- 172.21.15.67:0/3074099805 shutdown_connections 2024-09-21T06:29:49.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.504+0000 7f89e6d4e640 1 --2- 172.21.15.67:0/3074099805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89e0104090 0x7f89e010aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:49.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.504+0000 7f89e6d4e640 1 --2- 172.21.15.67:0/3074099805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89e01036f0 0x7f89e0103b00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:49.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.504+0000 7f89e6d4e640 1 --2- 172.21.15.67:0/3074099805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89e0102660 0x7f89e0102ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:49.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.504+0000 7f89e6d4e640 1 -- 172.21.15.67:0/3074099805 >> 172.21.15.67:0/3074099805 conn(0x7f89e00fdfd0 msgr2=0x7f89e0100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:29:49.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.504+0000 7f89e6d4e640 1 -- 172.21.15.67:0/3074099805 shutdown_connections 2024-09-21T06:29:49.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.504+0000 7f89e6d4e640 1 -- 172.21.15.67:0/3074099805 wait complete. 2024-09-21T06:29:49.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.505+0000 7f89e6d4e640 1 Processor -- start 2024-09-21T06:29:49.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.505+0000 7f89e6d4e640 1 -- start start 2024-09-21T06:29:49.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.505+0000 7f89e6d4e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89e0102660 0x7f89e0199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:49.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.505+0000 7f89e6d4e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89e01036f0 0x7f89e019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:49.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.505+0000 7f89e6d4e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89e0104090 0x7f89e019b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:49.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.505+0000 7f89e6d4e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f89e010e160 con 0x7f89e0104090 2024-09-21T06:29:49.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.505+0000 7f89e6d4e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f89e010dfe0 con 0x7f89e0102660 2024-09-21T06:29:49.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.505+0000 7f89e6d4e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f89e010e2e0 con 0x7f89e01036f0 2024-09-21T06:29:49.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.505+0000 7f89e52eb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89e0104090 0x7f89e019b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:49.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.505+0000 7f89e52eb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89e0104090 0x7f89e019b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60254/0 (socket says 172.21.15.67:60254) 2024-09-21T06:29:49.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.505+0000 7f89e52eb640 1 -- 172.21.15.67:0/703250571 learned_addr learned my addr 172.21.15.67:0/703250571 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:29:49.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.505+0000 7f89dffff640 1 --2- 172.21.15.67:0/703250571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89e01036f0 0x7f89e019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:49.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.505+0000 7f89e4aea640 1 --2- 172.21.15.67:0/703250571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89e0102660 0x7f89e0199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:49.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.505+0000 7f89e52eb640 1 -- 172.21.15.67:0/703250571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89e01036f0 msgr2=0x7f89e019a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:49.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.505+0000 7f89e52eb640 1 --2- 172.21.15.67:0/703250571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89e01036f0 0x7f89e019a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:49.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.505+0000 7f89e52eb640 1 -- 172.21.15.67:0/703250571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89e0102660 msgr2=0x7f89e0199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:29:49.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.505+0000 7f89e52eb640 1 --2- 172.21.15.67:0/703250571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89e0102660 0x7f89e0199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:29:49.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.505+0000 7f89e52eb640 1 -- 172.21.15.67:0/703250571 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89e019b9b0 con 0x7f89e0104090 2024-09-21T06:29:49.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.505+0000 7f89e4aea640 1 --2- 172.21.15.67:0/703250571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89e0102660 0x7f89e0199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:29:49.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.505+0000 7f89e52eb640 1 --2- 172.21.15.67:0/703250571 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89e0104090 0x7f89e019b190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f89d800eb10 tx=0x7f89d800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:49.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.506+0000 7f89ddffb640 1 -- 172.21.15.67:0/703250571 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d8004b20 con 0x7f89e0104090 2024-09-21T06:29:49.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.506+0000 7f89e6d4e640 1 -- 172.21.15.67:0/703250571 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89e019bca0 con 0x7f89e0104090 2024-09-21T06:29:49.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.506+0000 7f89ddffb640 1 -- 172.21.15.67:0/703250571 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f89d8004cc0 con 0x7f89e0104090 2024-09-21T06:29:49.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.506+0000 7f89ddffb640 1 -- 172.21.15.67:0/703250571 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d8010690 con 0x7f89e0104090 2024-09-21T06:29:49.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.506+0000 7f89e6d4e640 1 -- 172.21.15.67:0/703250571 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89e01a23a0 con 0x7f89e0104090 2024-09-21T06:29:49.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.507+0000 7f89e6d4e640 1 -- 172.21.15.67:0/703250571 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89e01a25d0 con 0x7f89e0104090 2024-09-21T06:29:49.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.508+0000 7f89ddffb640 1 -- 172.21.15.67:0/703250571 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f89d8005020 con 0x7f89e0104090 2024-09-21T06:29:49.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.509+0000 7f89ddffb640 1 --2- 172.21.15.67:0/703250571 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f89d00b6480 0x7f89d00b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:29:49.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.509+0000 7f89e4aea640 1 --2- 172.21.15.67:0/703250571 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f89d00b6480 0x7f89d00b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:29:49.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.509+0000 7f89ddffb640 1 -- 172.21.15.67:0/703250571 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2851+0+0 (secure 0 0 0) 0x7f89d80e62f0 con 0x7f89e0104090 2024-09-21T06:29:49.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.510+0000 7f89e4aea640 1 --2- 172.21.15.67:0/703250571 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f89d00b6480 0x7f89d00b8990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f89d40058c0 tx=0x7f89d4005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:29:49.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.514+0000 7f89ddffb640 1 -- 172.21.15.67:0/703250571 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f89d80ae1b0 con 0x7f89e0104090 2024-09-21T06:29:49.706 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:49 smithi203 ceph-mon[33735]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:49.706 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:49 smithi203 ceph-mon[33735]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-21T06:29:49.706 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:49 smithi203 ceph-mon[33735]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T06:29:49.706 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:49 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:29:49.706 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:49 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:29:49.707 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:49 smithi203 ceph-mon[33735]: Detected new or changed devices on smithi203 2024-09-21T06:29:49.707 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:49 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:49.707 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:49 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:49.707 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:49 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:29:49.707 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:49 smithi203 ceph-mon[33735]: Adjusting osd_memory_target on smithi203 to 17088M 2024-09-21T06:29:49.707 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:49 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:49.707 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:49 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:29:49.707 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:49 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:29:49.707 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:49 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:49.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:49.713+0000 7f89e6d4e640 1 -- 172.21.15.67:0/703250571 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f89e006c290 con 0x7f89d00b6480 2024-09-21T06:29:50.419 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:50 smithi067 ceph-mon[31232]: purged_snaps scrub starts 2024-09-21T06:29:50.419 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:50 smithi067 ceph-mon[31232]: purged_snaps scrub ok 2024-09-21T06:29:50.419 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:50 smithi067 ceph-mon[31232]: osd.2 [v2:172.21.15.203:6800/2754787169,v1:172.21.15.203:6801/2754787169] boot 2024-09-21T06:29:50.419 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:50 smithi067 ceph-mon[31232]: osdmap e18: 3 total, 3 up, 3 in 2024-09-21T06:29:50.419 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:50 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:29:50.419 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:50 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:29:50.419 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:50 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:29:50.419 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:50 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:29:50.419 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:50 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:29:50.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:50 smithi100 ceph-mon[33401]: purged_snaps scrub starts 2024-09-21T06:29:50.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:50 smithi100 ceph-mon[33401]: purged_snaps scrub ok 2024-09-21T06:29:50.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:50 smithi100 ceph-mon[33401]: osd.2 [v2:172.21.15.203:6800/2754787169,v1:172.21.15.203:6801/2754787169] boot 2024-09-21T06:29:50.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:50 smithi100 ceph-mon[33401]: osdmap e18: 3 total, 3 up, 3 in 2024-09-21T06:29:50.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:50 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:29:50.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:50 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:29:50.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:50 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:29:50.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:50 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:29:50.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:50 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:29:50.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:50 smithi203 ceph-mon[33735]: purged_snaps scrub starts 2024-09-21T06:29:50.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:50 smithi203 ceph-mon[33735]: purged_snaps scrub ok 2024-09-21T06:29:50.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:50 smithi203 ceph-mon[33735]: osd.2 [v2:172.21.15.203:6800/2754787169,v1:172.21.15.203:6801/2754787169] boot 2024-09-21T06:29:50.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:50 smithi203 ceph-mon[33735]: osdmap e18: 3 total, 3 up, 3 in 2024-09-21T06:29:50.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:50 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:29:50.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:50 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:29:50.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:50 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:29:50.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:50 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:29:50.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:50 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:29:51.341 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:51 smithi067 ceph-mon[31232]: from='client.14463 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:29:51.342 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:51 smithi067 ceph-mon[31232]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:51.342 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:51 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T06:29:51.342 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:51 smithi067 ceph-mon[31232]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T06:29:51.342 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:51 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:29:51.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:51 smithi100 ceph-mon[33401]: from='client.14463 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:29:51.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:51 smithi100 ceph-mon[33401]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:51.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:51 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T06:29:51.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:51 smithi100 ceph-mon[33401]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T06:29:51.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:51 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:29:51.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:51 smithi203 ceph-mon[33735]: from='client.14463 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:29:51.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:51 smithi203 ceph-mon[33735]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:29:51.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:51 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T06:29:51.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:51 smithi203 ceph-mon[33735]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T06:29:51.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:51 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:29:52.354 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:52 smithi067 sudo[50821]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T06:29:52.354 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:52 smithi067 sudo[50821]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:29:52.354 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:52 smithi067 sudo[50821]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:29:52.354 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:52 smithi067 sudo[50821]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:29:52.354 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:52 smithi067 sudo[50824]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T06:29:52.354 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:52 smithi067 sudo[50824]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:29:52.354 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:52 smithi067 sudo[50824]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:29:52.354 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:52 smithi067 sudo[50824]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:29:52.354 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 21 06:29:52 smithi067 sudo[50809]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T06:29:52.355 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 21 06:29:52 smithi067 sudo[50809]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:29:52.355 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 21 06:29:52 smithi067 sudo[50809]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:29:52.355 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 21 06:29:52 smithi067 sudo[50809]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:29:52.355 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 21 06:29:52 smithi067 sudo[50812]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T06:29:52.355 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 21 06:29:52 smithi067 sudo[50812]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:29:52.355 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 21 06:29:52 smithi067 sudo[50812]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:29:52.355 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 21 06:29:52 smithi067 sudo[50812]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:29:52.482 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 06:29:52 smithi203 sudo[41578]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T06:29:52.482 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 06:29:52 smithi203 sudo[41578]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:29:52.482 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 06:29:52 smithi203 sudo[41578]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:29:52.483 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 06:29:52 smithi203 sudo[41578]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:29:52.483 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 06:29:52 smithi203 sudo[41581]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T06:29:52.483 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 06:29:52 smithi203 sudo[41581]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:29:52.483 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 06:29:52 smithi203 sudo[41581]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:29:52.483 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 06:29:52 smithi203 sudo[41581]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:29:52.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:52 smithi203 sudo[41584]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T06:29:52.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:52 smithi203 sudo[41584]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:29:52.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:52 smithi203 sudo[41584]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:29:52.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:52 smithi203 sudo[41584]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:29:52.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:52 smithi203 sudo[41587]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T06:29:52.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:52 smithi203 sudo[41587]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:29:52.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:52 smithi203 sudo[41587]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:29:52.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:52 smithi203 sudo[41587]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:29:52.484 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:52 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T06:29:52.484 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:52 smithi203 ceph-mon[33735]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T06:29:52.484 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:52 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3801808879' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3f7d9abd-587c-47ad-bf43-11f7dcdf71c9"} : dispatch 2024-09-21T06:29:52.484 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:52 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3801808879' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f7d9abd-587c-47ad-bf43-11f7dcdf71c9"}]': finished 2024-09-21T06:29:52.484 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:52 smithi203 ceph-mon[33735]: osdmap e21: 4 total, 3 up, 4 in 2024-09-21T06:29:52.484 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:52 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:29:52.484 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:52 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:52.484 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:52 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:52.484 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:52 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:52.484 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:52 smithi203 ceph-mon[33735]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:29:52.553 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 21 06:29:52 smithi100 sudo[40958]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T06:29:52.554 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 21 06:29:52 smithi100 sudo[40958]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:29:52.554 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 21 06:29:52 smithi100 sudo[40958]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:29:52.554 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 21 06:29:52 smithi100 sudo[40958]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:29:52.554 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 21 06:29:52 smithi100 sudo[40961]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T06:29:52.554 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 21 06:29:52 smithi100 sudo[40961]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:29:52.554 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 21 06:29:52 smithi100 sudo[40961]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:29:52.554 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 21 06:29:52 smithi100 sudo[40961]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:29:52.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:52 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T06:29:52.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:52 smithi100 ceph-mon[33401]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T06:29:52.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:52 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3801808879' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3f7d9abd-587c-47ad-bf43-11f7dcdf71c9"} : dispatch 2024-09-21T06:29:52.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:52 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3801808879' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f7d9abd-587c-47ad-bf43-11f7dcdf71c9"}]': finished 2024-09-21T06:29:52.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:52 smithi100 ceph-mon[33401]: osdmap e21: 4 total, 3 up, 4 in 2024-09-21T06:29:52.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:52 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:29:52.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:52 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:52.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:52 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:52.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:52 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:52.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:52 smithi100 ceph-mon[33401]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:29:52.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:52 smithi100 sudo[40964]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T06:29:52.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:52 smithi100 sudo[40964]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:29:52.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:52 smithi100 sudo[40964]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:29:52.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:52 smithi100 sudo[40964]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:29:52.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:52 smithi100 sudo[40967]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T06:29:52.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:52 smithi100 sudo[40967]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:29:52.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:52 smithi100 sudo[40967]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:29:52.555 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:52 smithi100 sudo[40967]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:29:52.745 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:52 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T06:29:52.745 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:52 smithi067 ceph-mon[31232]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T06:29:52.745 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:52 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3801808879' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3f7d9abd-587c-47ad-bf43-11f7dcdf71c9"} : dispatch 2024-09-21T06:29:52.745 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:52 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3801808879' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f7d9abd-587c-47ad-bf43-11f7dcdf71c9"}]': finished 2024-09-21T06:29:52.746 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:52 smithi067 ceph-mon[31232]: osdmap e21: 4 total, 3 up, 4 in 2024-09-21T06:29:52.746 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:52 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:29:52.746 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:52 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:52.746 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:52 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:52.746 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:52 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:29:52.746 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:52 smithi067 ceph-mon[31232]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:29:53.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:53 smithi203 ceph-mon[33735]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:29:53.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:53 smithi203 ceph-mon[33735]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:29:53.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:53 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:29:53.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:53 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:29:53.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:53 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:29:53.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:53 smithi203 ceph-mon[33735]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:29:53.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:53 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:29:53.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:53 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:29:53.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:53 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:29:53.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:53 smithi203 ceph-mon[33735]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:29:53.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:53 smithi203 ceph-mon[33735]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:29:53.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:53 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:29:53.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:53 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:29:53.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:53 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:29:53.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:53 smithi203 ceph-mon[33735]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:29:53.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:53 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/507911418' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:29:53.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:53 smithi067 ceph-mon[31232]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:29:53.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:53 smithi067 ceph-mon[31232]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:29:53.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:53 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:29:53.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:53 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:29:53.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:53 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:29:53.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:53 smithi067 ceph-mon[31232]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:29:53.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:53 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:29:53.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:53 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:29:53.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:53 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:29:53.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:53 smithi067 ceph-mon[31232]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:29:53.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:53 smithi067 ceph-mon[31232]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:29:53.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:53 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:29:53.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:53 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:29:53.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:53 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:29:53.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:53 smithi067 ceph-mon[31232]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:29:53.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:53 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/507911418' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:29:53.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:53 smithi100 ceph-mon[33401]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:29:53.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:53 smithi100 ceph-mon[33401]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:29:53.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:53 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:29:53.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:53 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:29:53.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:53 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:29:53.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:53 smithi100 ceph-mon[33401]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:29:53.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:53 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:29:53.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:53 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:29:53.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:53 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:29:53.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:53 smithi100 ceph-mon[33401]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:29:53.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:53 smithi100 ceph-mon[33401]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:29:53.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:53 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:29:53.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:53 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:29:53.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:53 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:29:53.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:53 smithi100 ceph-mon[33401]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:29:53.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:53 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/507911418' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:29:54.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:29:54.365+0000 7f89ddffb640 1 -- 172.21.15.67:0/703250571 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f89d80a9d20 con 0x7f89e0104090 2024-09-21T06:29:55.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:55 smithi203 ceph-mon[33735]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:29:55.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:55 smithi203 ceph-mon[33735]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-21T06:29:55.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:55 smithi067 ceph-mon[31232]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:29:55.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:55 smithi067 ceph-mon[31232]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-21T06:29:55.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:55 smithi100 ceph-mon[33401]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:29:55.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:55 smithi100 ceph-mon[33401]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-21T06:29:57.481 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:57 smithi067 ceph-mon[31232]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:29:57.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:57 smithi203 ceph-mon[33735]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:29:57.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:57 smithi100 ceph-mon[33401]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:29:59.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:29:59 smithi203 ceph-mon[33735]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:29:59.484 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:29:59 smithi067 ceph-mon[31232]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:29:59.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:29:59 smithi100 ceph-mon[33401]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:30:00.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:00 smithi067 ceph-mon[31232]: overall HEALTH_OK 2024-09-21T06:30:00.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:00 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T06:30:00.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:00 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:30:00.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:00 smithi203 ceph-mon[33735]: overall HEALTH_OK 2024-09-21T06:30:00.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:00 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T06:30:00.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:00 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:30:00.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:00 smithi100 ceph-mon[33401]: overall HEALTH_OK 2024-09-21T06:30:00.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:00 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T06:30:00.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:00 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:30:01.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:01 smithi067 ceph-mon[31232]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:30:01.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:01 smithi067 ceph-mon[31232]: Deploying daemon osd.3 on smithi067 2024-09-21T06:30:01.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:01 smithi203 ceph-mon[33735]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:30:01.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:01 smithi203 ceph-mon[33735]: Deploying daemon osd.3 on smithi067 2024-09-21T06:30:01.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:01 smithi100 ceph-mon[33401]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:30:01.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:01 smithi100 ceph-mon[33401]: Deploying daemon osd.3 on smithi067 2024-09-21T06:30:03.701 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:03 smithi067 ceph-mon[31232]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:30:03.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:03 smithi203 ceph-mon[33735]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:30:03.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:03 smithi100 ceph-mon[33401]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:30:05.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:05 smithi067 ceph-mon[31232]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:30:05.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:05 smithi203 ceph-mon[33735]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:30:05.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:05 smithi100 ceph-mon[33401]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:30:07.649 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:07 smithi067 ceph-mon[31232]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:30:07.649 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:07 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:30:07.649 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:07 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:07.649 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:07 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:07.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:07 smithi203 ceph-mon[33735]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:30:07.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:07 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:30:07.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:07 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:07.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:07 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:07.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:07 smithi100 ceph-mon[33401]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:30:07.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:07 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:30:07.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:07 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:07.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:07 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:09.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:09 smithi203 ceph-mon[33735]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:30:09.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:09 smithi067 ceph-mon[31232]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:30:09.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:09 smithi100 ceph-mon[33401]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:30:10.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:10 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:10.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:10 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:10.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:10 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:30:10.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:10 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:30:10.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:10 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:10.788 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:10 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:10.788 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:10 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:10.788 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:10 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:30:10.788 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:10 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:30:10.788 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:10 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:10.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:10 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:10.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:10 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:10.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:10 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:30:10.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:10 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:30:10.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:10 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:11.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:30:11.130+0000 7f89ddffb640 1 -- 172.21.15.67:0/703250571 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f89e006c290 con 0x7f89d00b6480 2024-09-21T06:30:11.133 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 3 on host 'smithi067' 2024-09-21T06:30:11.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:30:11.133+0000 7f89e6d4e640 1 -- 172.21.15.67:0/703250571 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f89d00b6480 msgr2=0x7f89d00b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:11.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:30:11.133+0000 7f89e6d4e640 1 --2- 172.21.15.67:0/703250571 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f89d00b6480 0x7f89d00b8990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f89d40058c0 tx=0x7f89d4005d30 comp rx=0 tx=0).stop 2024-09-21T06:30:11.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:30:11.133+0000 7f89e6d4e640 1 -- 172.21.15.67:0/703250571 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89e0104090 msgr2=0x7f89e019b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:11.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:30:11.133+0000 7f89e6d4e640 1 --2- 172.21.15.67:0/703250571 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89e0104090 0x7f89e019b190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f89d800eb10 tx=0x7f89d800c2d0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:30:11.133+0000 7f89e6d4e640 1 -- 172.21.15.67:0/703250571 shutdown_connections 2024-09-21T06:30:11.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:30:11.133+0000 7f89e6d4e640 1 --2- 172.21.15.67:0/703250571 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f89d00b6480 0x7f89d00b8990 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:30:11.133+0000 7f89e6d4e640 1 --2- 172.21.15.67:0/703250571 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89e0104090 0x7f89e019b190 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:30:11.133+0000 7f89e6d4e640 1 --2- 172.21.15.67:0/703250571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89e01036f0 0x7f89e019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:30:11.133+0000 7f89e6d4e640 1 --2- 172.21.15.67:0/703250571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89e0102660 0x7f89e0199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:11.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:30:11.133+0000 7f89e6d4e640 1 -- 172.21.15.67:0/703250571 >> 172.21.15.67:0/703250571 conn(0x7f89e00fdfd0 msgr2=0x7f89e006bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:11.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:30:11.133+0000 7f89e6d4e640 1 -- 172.21.15.67:0/703250571 shutdown_connections 2024-09-21T06:30:11.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:30:11.133+0000 7f89e6d4e640 1 -- 172.21.15.67:0/703250571 wait complete. 2024-09-21T06:30:11.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:11 smithi203 ceph-mon[33735]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:30:11.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:11 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:30:11.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:11 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:11.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:11 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:11.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:11 smithi067 ceph-mon[31232]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:30:11.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:11 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:30:11.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:11 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:11.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:11 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:11.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:11 smithi100 ceph-mon[33401]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:30:11.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:11 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:30:11.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:11 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:11.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:11 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:11.845 DEBUG:teuthology.orchestra.run.smithi067:osd.3> sudo journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.3.service 2024-09-21T06:30:11.848 INFO:tasks.cephadm:Deploying osd.4 on smithi100 with /dev/vg_nvme/lv_3... 2024-09-21T06:30:11.848 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T06:30:12.103 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:30:13.443 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 21 06:30:13 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-3[52794]: 2024-09-21T06:30:13.111+0000 7efd7fa1a640 -1 osd.3 0 log_to_monitors true 2024-09-21T06:30:13.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:13 smithi203 ceph-mon[33735]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:30:13.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:13 smithi203 ceph-mon[33735]: from='osd.3 [v2:172.21.15.67:6810/4007517951,v1:172.21.15.67:6811/4007517951]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T06:30:13.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:13 smithi067 ceph-mon[31232]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:30:13.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:13 smithi067 ceph-mon[31232]: from='osd.3 [v2:172.21.15.67:6810/4007517951,v1:172.21.15.67:6811/4007517951]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T06:30:13.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:13 smithi100 ceph-mon[33401]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:30:13.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:13 smithi100 ceph-mon[33401]: from='osd.3 [v2:172.21.15.67:6810/4007517951,v1:172.21.15.67:6811/4007517951]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T06:30:14.525 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:14 smithi100 ceph-mon[33401]: from='osd.3 [v2:172.21.15.67:6810/4007517951,v1:172.21.15.67:6811/4007517951]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T06:30:14.526 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:14 smithi100 ceph-mon[33401]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T06:30:14.526 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:14 smithi100 ceph-mon[33401]: from='osd.3 [v2:172.21.15.67:6810/4007517951,v1:172.21.15.67:6811/4007517951]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]} : dispatch 2024-09-21T06:30:14.526 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:14 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:30:14.529 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-21T06:30:14.556 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi100:vg_nvme/lv_3 2024-09-21T06:30:14.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:14 smithi067 ceph-mon[31232]: from='osd.3 [v2:172.21.15.67:6810/4007517951,v1:172.21.15.67:6811/4007517951]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T06:30:14.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:14 smithi067 ceph-mon[31232]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T06:30:14.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:14 smithi067 ceph-mon[31232]: from='osd.3 [v2:172.21.15.67:6810/4007517951,v1:172.21.15.67:6811/4007517951]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]} : dispatch 2024-09-21T06:30:14.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:14 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:30:14.825 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:30:14.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:14 smithi203 ceph-mon[33735]: from='osd.3 [v2:172.21.15.67:6810/4007517951,v1:172.21.15.67:6811/4007517951]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T06:30:14.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:14 smithi203 ceph-mon[33735]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T06:30:14.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:14 smithi203 ceph-mon[33735]: from='osd.3 [v2:172.21.15.67:6810/4007517951,v1:172.21.15.67:6811/4007517951]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]} : dispatch 2024-09-21T06:30:14.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:14 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:30:15.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.248+0000 7fda14f64640 1 -- 172.21.15.100:0/2102260780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda101036d0 msgr2=0x7fda10103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:15.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.248+0000 7fda14f64640 1 --2- 172.21.15.100:0/2102260780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda101036d0 0x7fda10103ae0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009900 tx=0x7fd9f802a480 comp rx=0 tx=0).stop 2024-09-21T06:30:15.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.248+0000 7fda14f64640 1 -- 172.21.15.100:0/2102260780 shutdown_connections 2024-09-21T06:30:15.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.248+0000 7fda14f64640 1 --2- 172.21.15.100:0/2102260780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda101036d0 0x7fda10103ae0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:15.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.248+0000 7fda14f64640 1 --2- 172.21.15.100:0/2102260780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda10104070 0x7fda1010aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:15.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.248+0000 7fda14f64640 1 --2- 172.21.15.100:0/2102260780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda10102640 0x7fda10102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:15.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.248+0000 7fda14f64640 1 -- 172.21.15.100:0/2102260780 >> 172.21.15.100:0/2102260780 conn(0x7fda100fdfb0 msgr2=0x7fda10100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:15.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.248+0000 7fda14f64640 1 -- 172.21.15.100:0/2102260780 shutdown_connections 2024-09-21T06:30:15.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.248+0000 7fda14f64640 1 -- 172.21.15.100:0/2102260780 wait complete. 2024-09-21T06:30:15.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.249+0000 7fda14f64640 1 Processor -- start 2024-09-21T06:30:15.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.249+0000 7fda14f64640 1 -- start start 2024-09-21T06:30:15.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.250+0000 7fda14f64640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda10102640 0x7fda10198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:15.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.250+0000 7fda14f64640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda101036d0 0x7fda10199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:15.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.250+0000 7fda14f64640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda10104070 0x7fda10199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:15.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.250+0000 7fda14f64640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fda1010e140 con 0x7fda10104070 2024-09-21T06:30:15.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.250+0000 7fda14f64640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fda1010dfc0 con 0x7fda10102640 2024-09-21T06:30:15.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.250+0000 7fda14f64640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fda1010e2c0 con 0x7fda101036d0 2024-09-21T06:30:15.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.250+0000 7fda0e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda10102640 0x7fda10198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:15.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.250+0000 7fda0e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda10102640 0x7fda10198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39888/0 (socket says 172.21.15.100:39888) 2024-09-21T06:30:15.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.250+0000 7fda0e59c640 1 -- 172.21.15.100:0/3646906813 learned_addr learned my addr 172.21.15.100:0/3646906813 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-21T06:30:15.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.250+0000 7fda0ed9d640 1 --2- 172.21.15.100:0/3646906813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda10104070 0x7fda10199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:15.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.250+0000 7fda0dd9b640 1 --2- 172.21.15.100:0/3646906813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda101036d0 0x7fda10199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:15.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.250+0000 7fda0e59c640 1 -- 172.21.15.100:0/3646906813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda101036d0 msgr2=0x7fda10199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:15.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.250+0000 7fda0e59c640 1 --2- 172.21.15.100:0/3646906813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda101036d0 0x7fda10199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:15.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.250+0000 7fda0e59c640 1 -- 172.21.15.100:0/3646906813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda10104070 msgr2=0x7fda10199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:15.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.250+0000 7fda0e59c640 1 --2- 172.21.15.100:0/3646906813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda10104070 0x7fda10199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:15.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.250+0000 7fda0e59c640 1 -- 172.21.15.100:0/3646906813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1019a7d0 con 0x7fda10102640 2024-09-21T06:30:15.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.250+0000 7fda0e59c640 1 --2- 172.21.15.100:0/3646906813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda10102640 0x7fda10198da0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fda0000d950 tx=0x7fda0000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:15.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.251+0000 7fda077fe640 1 -- 172.21.15.100:0/3646906813 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00017070 con 0x7fda10102640 2024-09-21T06:30:15.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.251+0000 7fda14f64640 1 -- 172.21.15.100:0/3646906813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda1019aac0 con 0x7fda10102640 2024-09-21T06:30:15.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.251+0000 7fda077fe640 1 -- 172.21.15.100:0/3646906813 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda00009c30 con 0x7fda10102640 2024-09-21T06:30:15.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.251+0000 7fda077fe640 1 -- 172.21.15.100:0/3646906813 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0000f620 con 0x7fda10102640 2024-09-21T06:30:15.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.251+0000 7fda14f64640 1 -- 172.21.15.100:0/3646906813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda101a11c0 con 0x7fda10102640 2024-09-21T06:30:15.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.252+0000 7fda14f64640 1 -- 172.21.15.100:0/3646906813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda10102ab0 con 0x7fda10102640 2024-09-21T06:30:15.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.254+0000 7fda077fe640 1 -- 172.21.15.100:0/3646906813 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fda00010460 con 0x7fda10102640 2024-09-21T06:30:15.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.255+0000 7fda077fe640 1 --2- 172.21.15.100:0/3646906813 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd9fc0b6510 0x7fd9fc0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:15.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.255+0000 7fda077fe640 1 -- 172.21.15.100:0/3646906813 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 3389+0+0 (secure 0 0 0) 0x7fda000e5b30 con 0x7fda10102640 2024-09-21T06:30:15.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.255+0000 7fda0dd9b640 1 --2- 172.21.15.100:0/3646906813 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd9fc0b6510 0x7fd9fc0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:15.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.255+0000 7fda0dd9b640 1 --2- 172.21.15.100:0/3646906813 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd9fc0b6510 0x7fd9fc0b8a20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80057c0 tx=0x7fd9f8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:15.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.258+0000 7fda077fe640 1 -- 172.21.15.100:0/3646906813 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fda00009de0 con 0x7fda10102640 2024-09-21T06:30:15.292 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 21 06:30:14 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-3[52794]: 2024-09-21T06:30:14.941+0000 7efd7b99c640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T06:30:15.292 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 21 06:30:14 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-3[52794]: 2024-09-21T06:30:14.948+0000 7efd73792640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:30:15.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:15.453+0000 7fda14f64640 1 -- 172.21.15.100:0/3646906813 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fda1002ea20 con 0x7fd9fc0b6510 2024-09-21T06:30:15.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:15 smithi067 ceph-mon[31232]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:30:15.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:15 smithi067 ceph-mon[31232]: from='osd.3 [v2:172.21.15.67:6810/4007517951,v1:172.21.15.67:6811/4007517951]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-21T06:30:15.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:15 smithi067 ceph-mon[31232]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T06:30:15.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:15 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:30:15.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:15 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:30:15.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:15 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:15.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:15 smithi067 ceph-mon[31232]: from='osd.3 [v2:172.21.15.67:6810/4007517951,v1:172.21.15.67:6811/4007517951]' entity='osd.3' 2024-09-21T06:30:15.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:15 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:15.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:15 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:30:15.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:15 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:30:15.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:15 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:15.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:15 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:30:15.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:15 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:30:15.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:15 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:15.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:15 smithi067 ceph-mon[31232]: osd.3 [v2:172.21.15.67:6810/4007517951,v1:172.21.15.67:6811/4007517951] boot 2024-09-21T06:30:15.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:15 smithi067 ceph-mon[31232]: osdmap e24: 4 total, 4 up, 4 in 2024-09-21T06:30:15.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:15 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:30:15.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:15 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:30:15.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:15 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:30:15.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:15 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:30:15.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:15 smithi100 ceph-mon[33401]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:30:15.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:15 smithi100 ceph-mon[33401]: from='osd.3 [v2:172.21.15.67:6810/4007517951,v1:172.21.15.67:6811/4007517951]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-21T06:30:15.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:15 smithi100 ceph-mon[33401]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T06:30:15.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:15 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:30:15.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:15 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:30:15.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:15 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:15.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:15 smithi100 ceph-mon[33401]: from='osd.3 [v2:172.21.15.67:6810/4007517951,v1:172.21.15.67:6811/4007517951]' entity='osd.3' 2024-09-21T06:30:15.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:15 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:15.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:15 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:30:15.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:15 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:30:15.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:15 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:15.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:15 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:30:15.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:15 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:30:15.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:15 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:15.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:15 smithi100 ceph-mon[33401]: osd.3 [v2:172.21.15.67:6810/4007517951,v1:172.21.15.67:6811/4007517951] boot 2024-09-21T06:30:15.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:15 smithi100 ceph-mon[33401]: osdmap e24: 4 total, 4 up, 4 in 2024-09-21T06:30:15.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:15 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:30:15.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:15 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:30:15.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:15 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:30:15.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:15 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:30:15.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:15 smithi203 ceph-mon[33735]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:30:15.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:15 smithi203 ceph-mon[33735]: from='osd.3 [v2:172.21.15.67:6810/4007517951,v1:172.21.15.67:6811/4007517951]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-21T06:30:15.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:15 smithi203 ceph-mon[33735]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T06:30:15.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:15 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:30:15.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:15 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:30:15.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:15 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:15.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:15 smithi203 ceph-mon[33735]: from='osd.3 [v2:172.21.15.67:6810/4007517951,v1:172.21.15.67:6811/4007517951]' entity='osd.3' 2024-09-21T06:30:15.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:15 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:15.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:15 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:30:15.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:15 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:30:15.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:15 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:15.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:15 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:30:15.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:15 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:30:15.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:15 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:15.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:15 smithi203 ceph-mon[33735]: osd.3 [v2:172.21.15.67:6810/4007517951,v1:172.21.15.67:6811/4007517951] boot 2024-09-21T06:30:15.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:15 smithi203 ceph-mon[33735]: osdmap e24: 4 total, 4 up, 4 in 2024-09-21T06:30:15.984 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:15 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:30:15.984 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:15 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:30:15.984 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:15 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:30:15.984 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:15 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:30:16.778 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:16 smithi100 ceph-mon[33401]: purged_snaps scrub starts 2024-09-21T06:30:16.778 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:16 smithi100 ceph-mon[33401]: purged_snaps scrub ok 2024-09-21T06:30:16.778 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:16 smithi100 ceph-mon[33401]: Detected new or changed devices on smithi067 2024-09-21T06:30:16.778 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:16 smithi100 ceph-mon[33401]: Adjusting osd_memory_target on smithi067 to 8544M 2024-09-21T06:30:16.778 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:16 smithi100 ceph-mon[33401]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:30:16.779 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:16 smithi100 ceph-mon[33401]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T06:30:16.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:16 smithi067 ceph-mon[31232]: purged_snaps scrub starts 2024-09-21T06:30:16.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:16 smithi067 ceph-mon[31232]: purged_snaps scrub ok 2024-09-21T06:30:16.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:16 smithi067 ceph-mon[31232]: Detected new or changed devices on smithi067 2024-09-21T06:30:16.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:16 smithi067 ceph-mon[31232]: Adjusting osd_memory_target on smithi067 to 8544M 2024-09-21T06:30:16.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:16 smithi067 ceph-mon[31232]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:30:16.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:16 smithi067 ceph-mon[31232]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T06:30:16.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:16 smithi203 ceph-mon[33735]: purged_snaps scrub starts 2024-09-21T06:30:16.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:16 smithi203 ceph-mon[33735]: purged_snaps scrub ok 2024-09-21T06:30:16.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:16 smithi203 ceph-mon[33735]: Detected new or changed devices on smithi067 2024-09-21T06:30:16.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:16 smithi203 ceph-mon[33735]: Adjusting osd_memory_target on smithi067 to 8544M 2024-09-21T06:30:16.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:16 smithi203 ceph-mon[33735]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:30:16.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:16 smithi203 ceph-mon[33735]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T06:30:17.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:17 smithi067 ceph-mon[31232]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:30:17.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:17 smithi067 ceph-mon[31232]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T06:30:17.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:17 smithi100 ceph-mon[33401]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:30:17.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:17 smithi100 ceph-mon[33401]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T06:30:17.883 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:17 smithi203 ceph-mon[33735]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:30:17.883 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:17 smithi203 ceph-mon[33735]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T06:30:18.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:18 smithi067 ceph-mon[31232]: from='client.? 172.21.15.100:0/3735769054' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "94f3a075-6d8a-4907-8418-a8513b35779a"} : dispatch 2024-09-21T06:30:18.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:18 smithi067 ceph-mon[31232]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "94f3a075-6d8a-4907-8418-a8513b35779a"} : dispatch 2024-09-21T06:30:18.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:18 smithi067 ceph-mon[31232]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94f3a075-6d8a-4907-8418-a8513b35779a"}]': finished 2024-09-21T06:30:18.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:18 smithi067 ceph-mon[31232]: osdmap e27: 5 total, 4 up, 5 in 2024-09-21T06:30:18.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:18 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:30:18.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:18 smithi067 ceph-mon[31232]: from='client.? 172.21.15.100:0/62036296' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:30:18.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:18 smithi100 ceph-mon[33401]: from='client.? 172.21.15.100:0/3735769054' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "94f3a075-6d8a-4907-8418-a8513b35779a"} : dispatch 2024-09-21T06:30:18.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:18 smithi100 ceph-mon[33401]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "94f3a075-6d8a-4907-8418-a8513b35779a"} : dispatch 2024-09-21T06:30:18.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:18 smithi100 ceph-mon[33401]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94f3a075-6d8a-4907-8418-a8513b35779a"}]': finished 2024-09-21T06:30:18.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:18 smithi100 ceph-mon[33401]: osdmap e27: 5 total, 4 up, 5 in 2024-09-21T06:30:18.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:18 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:30:18.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:18 smithi100 ceph-mon[33401]: from='client.? 172.21.15.100:0/62036296' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:30:18.895 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:18 smithi203 ceph-mon[33735]: from='client.? 172.21.15.100:0/3735769054' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "94f3a075-6d8a-4907-8418-a8513b35779a"} : dispatch 2024-09-21T06:30:18.895 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:18 smithi203 ceph-mon[33735]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "94f3a075-6d8a-4907-8418-a8513b35779a"} : dispatch 2024-09-21T06:30:18.895 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:18 smithi203 ceph-mon[33735]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94f3a075-6d8a-4907-8418-a8513b35779a"}]': finished 2024-09-21T06:30:18.895 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:18 smithi203 ceph-mon[33735]: osdmap e27: 5 total, 4 up, 5 in 2024-09-21T06:30:18.895 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:18 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:30:18.895 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:18 smithi203 ceph-mon[33735]: from='client.? 172.21.15.100:0/62036296' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:30:19.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:19 smithi067 ceph-mon[31232]: pgmap v114: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:30:19.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:19 smithi100 ceph-mon[33401]: pgmap v114: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:30:19.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:19 smithi203 ceph-mon[33735]: pgmap v114: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:30:21.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:21 smithi067 ceph-mon[31232]: pgmap v115: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:30:21.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:21 smithi100 ceph-mon[33401]: pgmap v115: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:30:21.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:21 smithi203 ceph-mon[33735]: pgmap v115: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:30:23.760 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:23 smithi100 ceph-mon[33401]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:30:23.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:23 smithi067 ceph-mon[31232]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:30:23.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:23 smithi203 ceph-mon[33735]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:30:25.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:25 smithi067 ceph-mon[31232]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-21T06:30:25.797 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:25 smithi100 ceph-mon[33401]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-21T06:30:25.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:25 smithi203 ceph-mon[33735]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-21T06:30:26.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:26 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T06:30:26.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:26 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:30:26.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:26 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T06:30:26.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:26 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:30:26.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:26 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T06:30:26.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:26 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:30:27.636 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:27 smithi100 ceph-mon[33401]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-21T06:30:27.637 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:27 smithi100 ceph-mon[33401]: Deploying daemon osd.4 on smithi100 2024-09-21T06:30:27.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:27 smithi067 ceph-mon[31232]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-21T06:30:27.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:27 smithi067 ceph-mon[31232]: Deploying daemon osd.4 on smithi100 2024-09-21T06:30:27.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:27 smithi203 ceph-mon[33735]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-21T06:30:27.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:27 smithi203 ceph-mon[33735]: Deploying daemon osd.4 on smithi100 2024-09-21T06:30:29.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:29 smithi067 ceph-mon[31232]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T06:30:29.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:29 smithi100 ceph-mon[33401]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T06:30:29.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:29 smithi203 ceph-mon[33735]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T06:30:31.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:31 smithi067 ceph-mon[31232]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:30:31.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:31 smithi100 ceph-mon[33401]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:30:31.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:31 smithi203 ceph-mon[33735]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:30:32.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:32 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:30:32.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:32 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:32.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:32 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:32.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:32 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:30:32.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:32 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:32.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:32 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:32.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:32 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:30:32.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:32 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:32.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:32 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:33.770 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:33 smithi100 ceph-mon[33401]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:30:33.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:33 smithi067 ceph-mon[31232]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:30:33.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:33 smithi203 ceph-mon[33735]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:30:35.770 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:35 smithi100 ceph-mon[33401]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:30:35.770 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:35 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:35.771 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:35 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:35.771 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:35 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:30:35.771 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:35 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:30:35.771 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:35 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:35.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:35 smithi067 ceph-mon[31232]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:30:35.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:35.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:35.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:30:35.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:30:35.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:35.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:35 smithi203 ceph-mon[33735]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:30:35.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:35 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:35.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:35 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:35.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:35 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:30:35.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:35 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:30:35.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:35 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:36.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:36.127+0000 7fda077fe640 1 -- 172.21.15.100:0/3646906813 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fda1002ea20 con 0x7fd9fc0b6510 2024-09-21T06:30:36.130 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 4 on host 'smithi100' 2024-09-21T06:30:36.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:36.130+0000 7fda14f64640 1 -- 172.21.15.100:0/3646906813 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd9fc0b6510 msgr2=0x7fd9fc0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:36.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:36.130+0000 7fda14f64640 1 --2- 172.21.15.100:0/3646906813 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd9fc0b6510 0x7fd9fc0b8a20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80057c0 tx=0x7fd9f8000bc0 comp rx=0 tx=0).stop 2024-09-21T06:30:36.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:36.130+0000 7fda14f64640 1 -- 172.21.15.100:0/3646906813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda10102640 msgr2=0x7fda10198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:36.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:36.130+0000 7fda14f64640 1 --2- 172.21.15.100:0/3646906813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda10102640 0x7fda10198da0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fda0000d950 tx=0x7fda0000de10 comp rx=0 tx=0).stop 2024-09-21T06:30:36.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:36.130+0000 7fda14f64640 1 -- 172.21.15.100:0/3646906813 shutdown_connections 2024-09-21T06:30:36.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:36.130+0000 7fda14f64640 1 --2- 172.21.15.100:0/3646906813 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd9fc0b6510 0x7fd9fc0b8a20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:36.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:36.130+0000 7fda14f64640 1 --2- 172.21.15.100:0/3646906813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda10104070 0x7fda10199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:36.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:36.130+0000 7fda14f64640 1 --2- 172.21.15.100:0/3646906813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda101036d0 0x7fda10199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:36.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:36.130+0000 7fda14f64640 1 --2- 172.21.15.100:0/3646906813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda10102640 0x7fda10198da0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:36.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:36.130+0000 7fda14f64640 1 -- 172.21.15.100:0/3646906813 >> 172.21.15.100:0/3646906813 conn(0x7fda100fdfb0 msgr2=0x7fda101069d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:36.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:36.130+0000 7fda14f64640 1 -- 172.21.15.100:0/3646906813 shutdown_connections 2024-09-21T06:30:36.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:30:36.130+0000 7fda14f64640 1 -- 172.21.15.100:0/3646906813 wait complete. 2024-09-21T06:30:36.792 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:36 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:30:36.792 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:36 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:36.792 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:36 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:36.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:36 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:30:36.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:36 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:36.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:36 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:36.826 DEBUG:teuthology.orchestra.run.smithi100:osd.4> sudo journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.4.service 2024-09-21T06:30:36.828 INFO:tasks.cephadm:Deploying osd.5 on smithi203 with /dev/vg_nvme/lv_3... 2024-09-21T06:30:36.828 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T06:30:36.857 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:36 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:30:36.857 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:36 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:36.857 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:36 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:37.087 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:30:37.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:37 smithi067 ceph-mon[31232]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:30:37.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:37 smithi100 ceph-mon[33401]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:30:37.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:37 smithi203 ceph-mon[33735]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:30:38.097 INFO:journalctl@ceph.osd.4.smithi100.stdout:Sep 21 06:30:37 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-4[43550]: 2024-09-21T06:30:37.975+0000 7f224b8db640 -1 osd.4 0 log_to_monitors true 2024-09-21T06:30:38.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:38 smithi203 ceph-mon[33735]: from='osd.4 [v2:172.21.15.100:6808/3773646501,v1:172.21.15.100:6809/3773646501]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T06:30:38.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:38 smithi203 ceph-mon[33735]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T06:30:38.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:38 smithi100 ceph-mon[33401]: from='osd.4 [v2:172.21.15.100:6808/3773646501,v1:172.21.15.100:6809/3773646501]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T06:30:38.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:38 smithi100 ceph-mon[33401]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T06:30:38.982 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:38 smithi067 ceph-mon[31232]: from='osd.4 [v2:172.21.15.100:6808/3773646501,v1:172.21.15.100:6809/3773646501]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T06:30:38.982 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:38 smithi067 ceph-mon[31232]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T06:30:39.528 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T06:30:39.557 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi203:vg_nvme/lv_3 2024-09-21T06:30:39.803 INFO:journalctl@ceph.osd.4.smithi100.stdout:Sep 21 06:30:39 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-4[43550]: 2024-09-21T06:30:39.550+0000 7f224785d640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T06:30:39.803 INFO:journalctl@ceph.osd.4.smithi100.stdout:Sep 21 06:30:39 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-4[43550]: 2024-09-21T06:30:39.559+0000 7f224365b640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:30:39.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:39 smithi100 ceph-mon[33401]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:30:39.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:39 smithi100 ceph-mon[33401]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T06:30:39.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:39 smithi100 ceph-mon[33401]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T06:30:39.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:39 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:30:39.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:39 smithi100 ceph-mon[33401]: from='osd.4 [v2:172.21.15.100:6808/3773646501,v1:172.21.15.100:6809/3773646501]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]} : dispatch 2024-09-21T06:30:39.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:39 smithi100 ceph-mon[33401]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]} : dispatch 2024-09-21T06:30:39.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:39 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:39.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:39 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:39.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:39 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:30:39.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:39 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:30:39.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:39 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:39.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:39 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:30:39.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:39 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:30:39.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:39 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:39.836 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:39 smithi203 ceph-mon[33735]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:30:39.836 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:39 smithi203 ceph-mon[33735]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T06:30:39.836 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:39 smithi203 ceph-mon[33735]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T06:30:39.836 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:39 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:30:39.836 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:39 smithi203 ceph-mon[33735]: from='osd.4 [v2:172.21.15.100:6808/3773646501,v1:172.21.15.100:6809/3773646501]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]} : dispatch 2024-09-21T06:30:39.836 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:39 smithi203 ceph-mon[33735]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]} : dispatch 2024-09-21T06:30:39.836 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:39 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:39.836 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:39 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:39.836 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:39 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:30:39.837 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:39 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:30:39.837 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:39 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:39.837 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:39 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:30:39.837 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:39 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:30:39.837 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:39 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:39.851 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:30:39.987 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:39 smithi067 ceph-mon[31232]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:30:39.987 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:39 smithi067 ceph-mon[31232]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T06:30:39.988 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:39 smithi067 ceph-mon[31232]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T06:30:39.988 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:39 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:30:39.988 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:39 smithi067 ceph-mon[31232]: from='osd.4 [v2:172.21.15.100:6808/3773646501,v1:172.21.15.100:6809/3773646501]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]} : dispatch 2024-09-21T06:30:39.988 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:39 smithi067 ceph-mon[31232]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]} : dispatch 2024-09-21T06:30:39.988 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:39 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:39.988 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:39 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:39.988 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:39 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:30:39.988 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:39 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:30:39.988 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:39 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:39.988 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:39 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:30:39.988 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:39 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:30:39.988 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:39 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:40.337 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.332+0000 7f4c450f1640 1 -- 172.21.15.203:0/154577115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c401036d0 msgr2=0x7f4c40103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:40.337 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.332+0000 7f4c450f1640 1 --2- 172.21.15.203:0/154577115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c401036d0 0x7f4c40103ae0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4c34009900 tx=0x7f4c3402a480 comp rx=0 tx=0).stop 2024-09-21T06:30:40.337 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.332+0000 7f4c450f1640 1 -- 172.21.15.203:0/154577115 shutdown_connections 2024-09-21T06:30:40.338 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.332+0000 7f4c450f1640 1 --2- 172.21.15.203:0/154577115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c40104070 0x7f4c4010aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:40.338 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.332+0000 7f4c450f1640 1 --2- 172.21.15.203:0/154577115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c401036d0 0x7f4c40103ae0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:40.338 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.332+0000 7f4c450f1640 1 --2- 172.21.15.203:0/154577115 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c40102640 0x7f4c40102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:40.338 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.332+0000 7f4c450f1640 1 -- 172.21.15.203:0/154577115 >> 172.21.15.203:0/154577115 conn(0x7f4c400fdfb0 msgr2=0x7f4c40100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:30:40.338 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.333+0000 7f4c450f1640 1 -- 172.21.15.203:0/154577115 shutdown_connections 2024-09-21T06:30:40.338 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.333+0000 7f4c450f1640 1 -- 172.21.15.203:0/154577115 wait complete. 2024-09-21T06:30:40.338 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.333+0000 7f4c450f1640 1 Processor -- start 2024-09-21T06:30:40.338 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.333+0000 7f4c450f1640 1 -- start start 2024-09-21T06:30:40.338 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.334+0000 7f4c450f1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c40102640 0x7f4c40106230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:40.338 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.334+0000 7f4c450f1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c401036d0 0x7f4c40104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:40.338 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.334+0000 7f4c450f1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c40104070 0x7f4c40105040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:40.338 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.334+0000 7f4c450f1640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4010e120 con 0x7f4c401036d0 2024-09-21T06:30:40.338 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.334+0000 7f4c450f1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4010dfa0 con 0x7f4c40104070 2024-09-21T06:30:40.339 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.334+0000 7f4c450f1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4010e2a0 con 0x7f4c40102640 2024-09-21T06:30:40.339 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.334+0000 7f4c3ed9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c40102640 0x7f4c40106230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:40.339 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.334+0000 7f4c3ed9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c40102640 0x7f4c40106230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:39366/0 (socket says 172.21.15.203:39366) 2024-09-21T06:30:40.339 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.334+0000 7f4c3ed9d640 1 -- 172.21.15.203:0/2636037227 learned_addr learned my addr 172.21.15.203:0/2636037227 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-09-21T06:30:40.339 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.334+0000 7f4c3f59e640 1 --2- 172.21.15.203:0/2636037227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c40104070 0x7f4c40105040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:40.339 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.334+0000 7f4c3e59c640 1 --2- 172.21.15.203:0/2636037227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c401036d0 0x7f4c40104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:40.339 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.334+0000 7f4c3ed9d640 1 -- 172.21.15.203:0/2636037227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c40104070 msgr2=0x7f4c40105040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:40.339 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.334+0000 7f4c3ed9d640 1 --2- 172.21.15.203:0/2636037227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c40104070 0x7f4c40105040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:40.339 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.334+0000 7f4c3ed9d640 1 -- 172.21.15.203:0/2636037227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c401036d0 msgr2=0x7f4c40104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:30:40.339 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.334+0000 7f4c3ed9d640 1 --2- 172.21.15.203:0/2636037227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c401036d0 0x7f4c40104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:30:40.339 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.334+0000 7f4c3ed9d640 1 -- 172.21.15.203:0/2636037227 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c40105740 con 0x7f4c40102640 2024-09-21T06:30:40.339 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.334+0000 7f4c3e59c640 1 --2- 172.21.15.203:0/2636037227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c401036d0 0x7f4c40104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:30:40.339 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.334+0000 7f4c3ed9d640 1 --2- 172.21.15.203:0/2636037227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c40102640 0x7f4c40106230 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000d950 tx=0x7f4c3000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:40.340 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.335+0000 7f4c27fff640 1 -- 172.21.15.203:0/2636037227 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c30017070 con 0x7f4c40102640 2024-09-21T06:30:40.340 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.335+0000 7f4c450f1640 1 -- 172.21.15.203:0/2636037227 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c401a3f30 con 0x7f4c40102640 2024-09-21T06:30:40.340 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.335+0000 7f4c450f1640 1 -- 172.21.15.203:0/2636037227 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c401a4320 con 0x7f4c40102640 2024-09-21T06:30:40.340 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.335+0000 7f4c27fff640 1 -- 172.21.15.203:0/2636037227 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c30009c30 con 0x7f4c40102640 2024-09-21T06:30:40.340 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.335+0000 7f4c27fff640 1 -- 172.21.15.203:0/2636037227 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c30020060 con 0x7f4c40102640 2024-09-21T06:30:40.341 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.336+0000 7f4c450f1640 1 -- 172.21.15.203:0/2636037227 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c2c0052c0 con 0x7f4c40102640 2024-09-21T06:30:40.341 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.337+0000 7f4c27fff640 1 -- 172.21.15.203:0/2636037227 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f4c30010460 con 0x7f4c40102640 2024-09-21T06:30:40.341 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.338+0000 7f4c27fff640 1 --2- 172.21.15.203:0/2636037227 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f4c280b6510 0x7f4c280b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:30:40.341 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.338+0000 7f4c27fff640 1 -- 172.21.15.203:0/2636037227 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3821+0+0 (secure 0 0 0) 0x7f4c300e5d10 con 0x7f4c40102640 2024-09-21T06:30:40.341 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.338+0000 7f4c3e59c640 1 --2- 172.21.15.203:0/2636037227 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f4c280b6510 0x7f4c280b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:30:40.341 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.339+0000 7f4c3e59c640 1 --2- 172.21.15.203:0/2636037227 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f4c280b6510 0x7f4c280b8a20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4c34000c00 tx=0x7f4c34031420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:30:40.344 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.343+0000 7f4c27fff640 1 -- 172.21.15.203:0/2636037227 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4c300ad730 con 0x7f4c40102640 2024-09-21T06:30:40.538 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:30:40.536+0000 7f4c450f1640 1 -- 172.21.15.203:0/2636037227 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4c2c000bf0 con 0x7f4c280b6510 2024-09-21T06:30:40.848 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:40 smithi203 ceph-mon[33735]: Detected new or changed devices on smithi100 2024-09-21T06:30:40.848 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:40 smithi203 ceph-mon[33735]: Adjusting osd_memory_target on smithi100 to 8543M 2024-09-21T06:30:40.849 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:40 smithi203 ceph-mon[33735]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-21T06:30:40.849 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:40 smithi203 ceph-mon[33735]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T06:30:40.849 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:30:40.849 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:30:40.849 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:30:40.849 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:30:40.849 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:30:41.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:40 smithi067 ceph-mon[31232]: Detected new or changed devices on smithi100 2024-09-21T06:30:41.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:40 smithi067 ceph-mon[31232]: Adjusting osd_memory_target on smithi100 to 8543M 2024-09-21T06:30:41.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:40 smithi067 ceph-mon[31232]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-21T06:30:41.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:40 smithi067 ceph-mon[31232]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T06:30:41.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:40 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:30:41.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:40 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:30:41.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:40 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:30:41.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:40 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:30:41.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:40 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:30:41.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:40 smithi100 ceph-mon[33401]: Detected new or changed devices on smithi100 2024-09-21T06:30:41.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:40 smithi100 ceph-mon[33401]: Adjusting osd_memory_target on smithi100 to 8543M 2024-09-21T06:30:41.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:40 smithi100 ceph-mon[33401]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-21T06:30:41.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:40 smithi100 ceph-mon[33401]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T06:30:41.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:40 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:30:41.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:40 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:30:41.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:40 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:30:41.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:40 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:30:41.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:40 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:30:41.879 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:41 smithi203 ceph-mon[33735]: purged_snaps scrub starts 2024-09-21T06:30:41.879 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:41 smithi203 ceph-mon[33735]: purged_snaps scrub ok 2024-09-21T06:30:41.879 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:41 smithi203 ceph-mon[33735]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:30:41.879 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:41 smithi203 ceph-mon[33735]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:30:41.879 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:41 smithi203 ceph-mon[33735]: osd.4 [v2:172.21.15.100:6808/3773646501,v1:172.21.15.100:6809/3773646501] boot 2024-09-21T06:30:41.879 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:41 smithi203 ceph-mon[33735]: osdmap e30: 5 total, 5 up, 5 in 2024-09-21T06:30:41.879 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:41 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:30:42.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:41 smithi067 ceph-mon[31232]: purged_snaps scrub starts 2024-09-21T06:30:42.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:41 smithi067 ceph-mon[31232]: purged_snaps scrub ok 2024-09-21T06:30:42.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:41 smithi067 ceph-mon[31232]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:30:42.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:41 smithi067 ceph-mon[31232]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:30:42.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:41 smithi067 ceph-mon[31232]: osd.4 [v2:172.21.15.100:6808/3773646501,v1:172.21.15.100:6809/3773646501] boot 2024-09-21T06:30:42.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:41 smithi067 ceph-mon[31232]: osdmap e30: 5 total, 5 up, 5 in 2024-09-21T06:30:42.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:41 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:30:42.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:41 smithi100 ceph-mon[33401]: purged_snaps scrub starts 2024-09-21T06:30:42.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:41 smithi100 ceph-mon[33401]: purged_snaps scrub ok 2024-09-21T06:30:42.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:41 smithi100 ceph-mon[33401]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:30:42.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:41 smithi100 ceph-mon[33401]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:30:42.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:41 smithi100 ceph-mon[33401]: osd.4 [v2:172.21.15.100:6808/3773646501,v1:172.21.15.100:6809/3773646501] boot 2024-09-21T06:30:42.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:41 smithi100 ceph-mon[33401]: osdmap e30: 5 total, 5 up, 5 in 2024-09-21T06:30:42.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:41 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:30:42.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:42 smithi203 ceph-mon[33735]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T06:30:43.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:42 smithi067 ceph-mon[31232]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T06:30:43.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:42 smithi100 ceph-mon[33401]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T06:30:43.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:43 smithi203 ceph-mon[33735]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:30:43.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:43 smithi203 ceph-mon[33735]: from='client.? 172.21.15.203:0/1257319053' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5d842edf-06ad-49ae-ac71-863e5657270c"} : dispatch 2024-09-21T06:30:43.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:43 smithi203 ceph-mon[33735]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5d842edf-06ad-49ae-ac71-863e5657270c"} : dispatch 2024-09-21T06:30:43.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:43 smithi203 ceph-mon[33735]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d842edf-06ad-49ae-ac71-863e5657270c"}]': finished 2024-09-21T06:30:43.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:43 smithi203 ceph-mon[33735]: osdmap e32: 6 total, 5 up, 6 in 2024-09-21T06:30:43.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:43 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:30:44.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:43 smithi067 ceph-mon[31232]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:30:44.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:43 smithi067 ceph-mon[31232]: from='client.? 172.21.15.203:0/1257319053' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5d842edf-06ad-49ae-ac71-863e5657270c"} : dispatch 2024-09-21T06:30:44.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:43 smithi067 ceph-mon[31232]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5d842edf-06ad-49ae-ac71-863e5657270c"} : dispatch 2024-09-21T06:30:44.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:43 smithi067 ceph-mon[31232]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d842edf-06ad-49ae-ac71-863e5657270c"}]': finished 2024-09-21T06:30:44.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:43 smithi067 ceph-mon[31232]: osdmap e32: 6 total, 5 up, 6 in 2024-09-21T06:30:44.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:43 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:30:44.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:43 smithi100 ceph-mon[33401]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:30:44.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:43 smithi100 ceph-mon[33401]: from='client.? 172.21.15.203:0/1257319053' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5d842edf-06ad-49ae-ac71-863e5657270c"} : dispatch 2024-09-21T06:30:44.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:43 smithi100 ceph-mon[33401]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5d842edf-06ad-49ae-ac71-863e5657270c"} : dispatch 2024-09-21T06:30:44.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:43 smithi100 ceph-mon[33401]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d842edf-06ad-49ae-ac71-863e5657270c"}]': finished 2024-09-21T06:30:44.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:43 smithi100 ceph-mon[33401]: osdmap e32: 6 total, 5 up, 6 in 2024-09-21T06:30:44.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:43 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:30:44.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:44 smithi203 ceph-mon[33735]: from='client.? 172.21.15.203:0/1578875582' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:30:45.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:44 smithi067 ceph-mon[31232]: from='client.? 172.21.15.203:0/1578875582' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:30:45.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:44 smithi100 ceph-mon[33401]: from='client.? 172.21.15.203:0/1578875582' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:30:45.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:45 smithi203 ceph-mon[33735]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:30:46.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:45 smithi067 ceph-mon[31232]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:30:46.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:45 smithi100 ceph-mon[33401]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:30:47.856 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:47 smithi100 ceph-mon[33401]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:30:47.969 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:47 smithi203 ceph-mon[33735]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:30:48.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:47 smithi067 ceph-mon[31232]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:30:48.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:48 smithi203 ceph-mon[33735]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:30:49.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:48 smithi067 ceph-mon[31232]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:30:49.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:48 smithi100 ceph-mon[33401]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:30:50.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:50 smithi203 ceph-mon[33735]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:30:51.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:50 smithi067 ceph-mon[31232]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:30:51.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:50 smithi100 ceph-mon[33401]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:30:51.939 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:51 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T06:30:51.939 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:51 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:30:51.940 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:51 smithi203 ceph-mon[33735]: Deploying daemon osd.5 on smithi203 2024-09-21T06:30:52.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:51 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T06:30:52.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:51 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:30:52.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:51 smithi067 ceph-mon[31232]: Deploying daemon osd.5 on smithi203 2024-09-21T06:30:52.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:51 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T06:30:52.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:51 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:30:52.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:51 smithi100 ceph-mon[33401]: Deploying daemon osd.5 on smithi203 2024-09-21T06:30:52.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:52 smithi203 ceph-mon[33735]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:30:53.291 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:52 smithi067 ceph-mon[31232]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:30:53.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:52 smithi100 ceph-mon[33401]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:30:55.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:54 smithi203 ceph-mon[33735]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:30:55.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:54 smithi067 ceph-mon[31232]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:30:55.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:54 smithi100 ceph-mon[33401]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:30:56.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:56 smithi203 ceph-mon[33735]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:30:57.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:56 smithi067 ceph-mon[31232]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:30:57.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:56 smithi100 ceph-mon[33401]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:30:58.091 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:30:58.091 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:58.091 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:58.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:30:58.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:58.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:58.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:30:58.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:58.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:30:59.139 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:30:58 smithi203 ceph-mon[33735]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:30:59.151 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:30:58 smithi100 ceph-mon[33401]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:30:59.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:30:58 smithi067 ceph-mon[31232]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:31:01.364 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:01.361+0000 7f4c27fff640 1 -- 172.21.15.203:0/2636037227 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4c2c000bf0 con 0x7f4c280b6510 2024-09-21T06:31:01.364 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 5 on host 'smithi203' 2024-09-21T06:31:01.366 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:01.364+0000 7f4c450f1640 1 -- 172.21.15.203:0/2636037227 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f4c280b6510 msgr2=0x7f4c280b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:01.366 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:01.364+0000 7f4c450f1640 1 --2- 172.21.15.203:0/2636037227 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f4c280b6510 0x7f4c280b8a20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4c34000c00 tx=0x7f4c34031420 comp rx=0 tx=0).stop 2024-09-21T06:31:01.366 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:01.364+0000 7f4c450f1640 1 -- 172.21.15.203:0/2636037227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c40102640 msgr2=0x7f4c40106230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:01.366 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:01.364+0000 7f4c450f1640 1 --2- 172.21.15.203:0/2636037227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c40102640 0x7f4c40106230 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000d950 tx=0x7f4c3000de10 comp rx=0 tx=0).stop 2024-09-21T06:31:01.366 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:01.364+0000 7f4c450f1640 1 -- 172.21.15.203:0/2636037227 shutdown_connections 2024-09-21T06:31:01.366 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:01.364+0000 7f4c450f1640 1 --2- 172.21.15.203:0/2636037227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c40104070 0x7f4c40105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.366 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:01.364+0000 7f4c450f1640 1 --2- 172.21.15.203:0/2636037227 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f4c280b6510 0x7f4c280b8a20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.366 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:01.364+0000 7f4c450f1640 1 --2- 172.21.15.203:0/2636037227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c401036d0 0x7f4c40104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.366 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:01.364+0000 7f4c450f1640 1 --2- 172.21.15.203:0/2636037227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c40102640 0x7f4c40106230 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:01.367 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:01.364+0000 7f4c450f1640 1 -- 172.21.15.203:0/2636037227 >> 172.21.15.203:0/2636037227 conn(0x7f4c400fdfb0 msgr2=0x7f4c400ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:01.367 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:01.364+0000 7f4c450f1640 1 -- 172.21.15.203:0/2636037227 shutdown_connections 2024-09-21T06:31:01.367 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:01.364+0000 7f4c450f1640 1 -- 172.21.15.203:0/2636037227 wait complete. 2024-09-21T06:31:01.714 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:01 smithi203 ceph-mon[33735]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:31:01.714 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:01 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:01.714 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:01 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:01.714 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:01 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:01.714 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:01 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:31:01.714 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:01 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:01.714 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:01 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:31:01.715 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:01 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:01.715 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:01 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:02.026 DEBUG:teuthology.orchestra.run.smithi203:osd.5> sudo journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.5.service 2024-09-21T06:31:02.029 INFO:tasks.cephadm:Deploying osd.6 on smithi067 with /dev/vg_nvme/lv_2... 2024-09-21T06:31:02.029 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T06:31:02.041 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:01 smithi067 ceph-mon[31232]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:31:02.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:01 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:02.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:01 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:02.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:01 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:02.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:01 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:31:02.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:01 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:02.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:01 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:31:02.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:01 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:02.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:01 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:02.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:01 smithi100 ceph-mon[33401]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:31:02.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:01 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:02.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:01 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:02.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:01 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:02.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:01 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:31:02.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:01 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:02.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:01 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:31:02.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:01 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:02.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:01 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:02.294 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:31:03.748 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:03 smithi067 ceph-mon[31232]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:31:03.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:03 smithi203 ceph-mon[33735]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:31:04.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:03 smithi100 ceph-mon[33401]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:31:04.232 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:31:03 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:31:03.940+0000 7f6f2c237640 -1 osd.5 0 log_to_monitors true 2024-09-21T06:31:04.687 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:04 smithi067 ceph-mon[31232]: from='osd.5 [v2:172.21.15.203:6808/2096171166,v1:172.21.15.203:6809/2096171166]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T06:31:04.687 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:04 smithi067 ceph-mon[31232]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T06:31:04.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:04 smithi203 ceph-mon[33735]: from='osd.5 [v2:172.21.15.203:6808/2096171166,v1:172.21.15.203:6809/2096171166]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T06:31:04.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:04 smithi203 ceph-mon[33735]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T06:31:05.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:04 smithi100 ceph-mon[33401]: from='osd.5 [v2:172.21.15.203:6808/2096171166,v1:172.21.15.203:6809/2096171166]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T06:31:05.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:04 smithi100 ceph-mon[33401]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T06:31:05.328 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:31:05.354 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi067:vg_nvme/lv_2 2024-09-21T06:31:05.658 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:31:05.910 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:05 smithi067 ceph-mon[31232]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:31:05.910 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:05 smithi067 ceph-mon[31232]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T06:31:05.910 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:05 smithi067 ceph-mon[31232]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T06:31:05.910 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:05 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:31:05.910 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:05 smithi067 ceph-mon[31232]: from='osd.5 [v2:172.21.15.203:6808/2096171166,v1:172.21.15.203:6809/2096171166]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T06:31:05.910 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:05 smithi067 ceph-mon[31232]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T06:31:05.910 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:05 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:05.911 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:05 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:05.911 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:05 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:31:05.911 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:05 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:31:05.911 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:05 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:05.911 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:05 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:05.911 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:05 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:31:05.911 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:05 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:05.982 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:31:05 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:31:05.577+0000 7f6f281b9640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T06:31:05.982 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:31:05 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:31:05.583+0000 7f6f23fb7640 -1 osd.5 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:31:05.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:05 smithi203 ceph-mon[33735]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:31:05.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:05 smithi203 ceph-mon[33735]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T06:31:05.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:05 smithi203 ceph-mon[33735]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T06:31:05.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:05 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:31:05.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:05 smithi203 ceph-mon[33735]: from='osd.5 [v2:172.21.15.203:6808/2096171166,v1:172.21.15.203:6809/2096171166]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T06:31:05.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:05 smithi203 ceph-mon[33735]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T06:31:05.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:05 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:05.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:05 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:05.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:05 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:31:05.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:05 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:31:05.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:05 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:05.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:05 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:05.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:05 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:31:05.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:05 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:06.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:05 smithi100 ceph-mon[33401]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:31:06.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:05 smithi100 ceph-mon[33401]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T06:31:06.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:05 smithi100 ceph-mon[33401]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T06:31:06.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:05 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:31:06.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:05 smithi100 ceph-mon[33401]: from='osd.5 [v2:172.21.15.203:6808/2096171166,v1:172.21.15.203:6809/2096171166]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T06:31:06.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:05 smithi100 ceph-mon[33401]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T06:31:06.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:05 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:06.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:05 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:06.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:05 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:31:06.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:05 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:31:06.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:05 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:06.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:05 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:06.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:05 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:31:06.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:05 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:06.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.139+0000 7fb8d7bae640 1 -- 172.21.15.67:0/3011918100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8d006b980 msgr2=0x7fb8d0106390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:06.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.139+0000 7fb8d7bae640 1 --2- 172.21.15.67:0/3011918100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8d006b980 0x7fb8d0106390 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8009900 tx=0x7fb8c802a410 comp rx=0 tx=0).stop 2024-09-21T06:31:06.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.139+0000 7fb8d7bae640 1 -- 172.21.15.67:0/3011918100 shutdown_connections 2024-09-21T06:31:06.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.139+0000 7fb8d7bae640 1 --2- 172.21.15.67:0/3011918100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8d01068d0 0x7fb8d0108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.139+0000 7fb8d7bae640 1 --2- 172.21.15.67:0/3011918100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8d0109470 0x7fb8d010b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.139+0000 7fb8d7bae640 1 --2- 172.21.15.67:0/3011918100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8d006b980 0x7fb8d0106390 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.139+0000 7fb8d7bae640 1 -- 172.21.15.67:0/3011918100 >> 172.21.15.67:0/3011918100 conn(0x7fb8d00fdfb0 msgr2=0x7fb8d0100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:06.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.139+0000 7fb8d7bae640 1 -- 172.21.15.67:0/3011918100 shutdown_connections 2024-09-21T06:31:06.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.139+0000 7fb8d7bae640 1 -- 172.21.15.67:0/3011918100 wait complete. 2024-09-21T06:31:06.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.140+0000 7fb8d7bae640 1 Processor -- start 2024-09-21T06:31:06.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.140+0000 7fb8d7bae640 1 -- start start 2024-09-21T06:31:06.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.140+0000 7fb8d7bae640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8d006b980 0x7fb8d0199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:06.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.140+0000 7fb8d7bae640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8d01068d0 0x7fb8d019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:06.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.140+0000 7fb8d7bae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8d0109470 0x7fb8d019b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:06.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.140+0000 7fb8d7bae640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d010f210 con 0x7fb8d006b980 2024-09-21T06:31:06.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.140+0000 7fb8d7bae640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d010f090 con 0x7fb8d0109470 2024-09-21T06:31:06.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.140+0000 7fb8d7bae640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d010f390 con 0x7fb8d01068d0 2024-09-21T06:31:06.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.140+0000 7fb8d594a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8d006b980 0x7fb8d0199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:06.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.140+0000 7fb8d594a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8d006b980 0x7fb8d0199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33066/0 (socket says 172.21.15.67:33066) 2024-09-21T06:31:06.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.140+0000 7fb8d594a640 1 -- 172.21.15.67:0/3955493893 learned_addr learned my addr 172.21.15.67:0/3955493893 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:31:06.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.140+0000 7fb8d5149640 1 --2- 172.21.15.67:0/3955493893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8d01068d0 0x7fb8d019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:06.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.140+0000 7fb8d614b640 1 --2- 172.21.15.67:0/3955493893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8d0109470 0x7fb8d019b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:06.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.140+0000 7fb8d594a640 1 -- 172.21.15.67:0/3955493893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8d01068d0 msgr2=0x7fb8d019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:06.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.140+0000 7fb8d594a640 1 --2- 172.21.15.67:0/3955493893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8d01068d0 0x7fb8d019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.140+0000 7fb8d594a640 1 -- 172.21.15.67:0/3955493893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8d0109470 msgr2=0x7fb8d019b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:06.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.141+0000 7fb8d594a640 1 --2- 172.21.15.67:0/3955493893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8d0109470 0x7fb8d019b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:06.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.141+0000 7fb8d594a640 1 -- 172.21.15.67:0/3955493893 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d019b960 con 0x7fb8d006b980 2024-09-21T06:31:06.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.141+0000 7fb8d594a640 1 --2- 172.21.15.67:0/3955493893 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8d006b980 0x7fb8d0199f30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8009230 tx=0x7fb8c80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:06.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.141+0000 7fb8ceffd640 1 -- 172.21.15.67:0/3955493893 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c803f030 con 0x7fb8d006b980 2024-09-21T06:31:06.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.141+0000 7fb8ceffd640 1 -- 172.21.15.67:0/3955493893 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8c8003c90 con 0x7fb8d006b980 2024-09-21T06:31:06.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.141+0000 7fb8d7bae640 1 -- 172.21.15.67:0/3955493893 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d019bbf0 con 0x7fb8d006b980 2024-09-21T06:31:06.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.141+0000 7fb8ceffd640 1 -- 172.21.15.67:0/3955493893 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8035070 con 0x7fb8d006b980 2024-09-21T06:31:06.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.141+0000 7fb8d7bae640 1 -- 172.21.15.67:0/3955493893 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d01a2350 con 0x7fb8d006b980 2024-09-21T06:31:06.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.142+0000 7fb8d7bae640 1 -- 172.21.15.67:0/3955493893 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8d00ffad0 con 0x7fb8d006b980 2024-09-21T06:31:06.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.144+0000 7fb8ceffd640 1 -- 172.21.15.67:0/3955493893 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fb8c8042030 con 0x7fb8d006b980 2024-09-21T06:31:06.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.145+0000 7fb8ceffd640 1 --2- 172.21.15.67:0/3955493893 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb8b80b6590 0x7fb8b80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:06.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.145+0000 7fb8d5149640 1 --2- 172.21.15.67:0/3955493893 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb8b80b6590 0x7fb8b80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:06.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.145+0000 7fb8ceffd640 1 -- 172.21.15.67:0/3955493893 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4253+0+0 (secure 0 0 0) 0x7fb8c8106ce0 con 0x7fb8d006b980 2024-09-21T06:31:06.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.146+0000 7fb8d5149640 1 --2- 172.21.15.67:0/3955493893 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb8b80b6590 0x7fb8b80b8aa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4005d00 tx=0x7fb8c400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:06.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.149+0000 7fb8ceffd640 1 -- 172.21.15.67:0/3955493893 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb8c80d2890 con 0x7fb8d006b980 2024-09-21T06:31:06.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:06.347+0000 7fb8d7bae640 1 -- 172.21.15.67:0/3955493893 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb8d0063800 con 0x7fb8b80b6590 2024-09-21T06:31:06.736 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:06 smithi067 ceph-mon[31232]: Detected new or changed devices on smithi203 2024-09-21T06:31:06.736 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:06 smithi067 ceph-mon[31232]: Adjusting osd_memory_target on smithi203 to 8544M 2024-09-21T06:31:06.737 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:06 smithi067 ceph-mon[31232]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-21T06:31:06.737 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:06 smithi067 ceph-mon[31232]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T06:31:06.737 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:06 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:31:06.737 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:06 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:31:06.737 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:06 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:31:06.737 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:06 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:31:06.737 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:06 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:06.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:06 smithi203 ceph-mon[33735]: Detected new or changed devices on smithi203 2024-09-21T06:31:06.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:06 smithi203 ceph-mon[33735]: Adjusting osd_memory_target on smithi203 to 8544M 2024-09-21T06:31:06.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:06 smithi203 ceph-mon[33735]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-21T06:31:06.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:06 smithi203 ceph-mon[33735]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T06:31:06.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:06 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:31:06.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:06 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:31:06.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:06 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:31:06.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:06 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:31:06.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:06 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:07.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:06 smithi100 ceph-mon[33401]: Detected new or changed devices on smithi203 2024-09-21T06:31:07.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:06 smithi100 ceph-mon[33401]: Adjusting osd_memory_target on smithi203 to 8544M 2024-09-21T06:31:07.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:06 smithi100 ceph-mon[33401]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-21T06:31:07.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:06 smithi100 ceph-mon[33401]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T06:31:07.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:06 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:31:07.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:06 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:31:07.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:06 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:31:07.055 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:06 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:31:07.055 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:06 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:07.903 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:07 smithi067 ceph-mon[31232]: purged_snaps scrub starts 2024-09-21T06:31:07.904 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:07 smithi067 ceph-mon[31232]: purged_snaps scrub ok 2024-09-21T06:31:07.904 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:07 smithi067 ceph-mon[31232]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:31:07.904 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:07 smithi067 ceph-mon[31232]: from='client.14547 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:31:07.904 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:07 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:31:07.904 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:07 smithi067 ceph-mon[31232]: osd.5 [v2:172.21.15.203:6808/2096171166,v1:172.21.15.203:6809/2096171166] boot 2024-09-21T06:31:07.904 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:07 smithi067 ceph-mon[31232]: osdmap e35: 6 total, 6 up, 6 in 2024-09-21T06:31:07.904 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:07 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:31:07.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:07 smithi203 ceph-mon[33735]: purged_snaps scrub starts 2024-09-21T06:31:07.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:07 smithi203 ceph-mon[33735]: purged_snaps scrub ok 2024-09-21T06:31:07.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:07 smithi203 ceph-mon[33735]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:31:07.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:07 smithi203 ceph-mon[33735]: from='client.14547 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:31:07.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:07 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:31:07.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:07 smithi203 ceph-mon[33735]: osd.5 [v2:172.21.15.203:6808/2096171166,v1:172.21.15.203:6809/2096171166] boot 2024-09-21T06:31:07.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:07 smithi203 ceph-mon[33735]: osdmap e35: 6 total, 6 up, 6 in 2024-09-21T06:31:07.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:07 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:31:08.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:07 smithi100 ceph-mon[33401]: purged_snaps scrub starts 2024-09-21T06:31:08.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:07 smithi100 ceph-mon[33401]: purged_snaps scrub ok 2024-09-21T06:31:08.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:07 smithi100 ceph-mon[33401]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:31:08.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:07 smithi100 ceph-mon[33401]: from='client.14547 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:31:08.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:07 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:31:08.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:07 smithi100 ceph-mon[33401]: osd.5 [v2:172.21.15.203:6808/2096171166,v1:172.21.15.203:6809/2096171166] boot 2024-09-21T06:31:08.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:07 smithi100 ceph-mon[33401]: osdmap e35: 6 total, 6 up, 6 in 2024-09-21T06:31:08.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:07 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:31:08.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:08 smithi067 ceph-mon[31232]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T06:31:08.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:08 smithi067 ceph-mon[31232]: pgmap v148: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:31:08.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:08 smithi203 ceph-mon[33735]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T06:31:08.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:08 smithi203 ceph-mon[33735]: pgmap v148: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:31:09.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:08 smithi100 ceph-mon[33401]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T06:31:09.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:08 smithi100 ceph-mon[33401]: pgmap v148: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:31:09.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:09 smithi067 ceph-mon[31232]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T06:31:09.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:09 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2724283584' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "85b9beed-c93d-48ba-bc7a-c4cd816fbdff"} : dispatch 2024-09-21T06:31:09.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:09 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2724283584' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85b9beed-c93d-48ba-bc7a-c4cd816fbdff"}]': finished 2024-09-21T06:31:09.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:09 smithi067 ceph-mon[31232]: osdmap e38: 7 total, 6 up, 7 in 2024-09-21T06:31:09.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:09 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:31:09.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:09 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1951997081' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:31:09.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:09 smithi203 ceph-mon[33735]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T06:31:09.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:09 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2724283584' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "85b9beed-c93d-48ba-bc7a-c4cd816fbdff"} : dispatch 2024-09-21T06:31:09.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:09 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2724283584' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85b9beed-c93d-48ba-bc7a-c4cd816fbdff"}]': finished 2024-09-21T06:31:09.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:09 smithi203 ceph-mon[33735]: osdmap e38: 7 total, 6 up, 7 in 2024-09-21T06:31:09.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:09 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:31:09.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:09 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1951997081' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:31:10.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:09 smithi100 ceph-mon[33401]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T06:31:10.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:09 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2724283584' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "85b9beed-c93d-48ba-bc7a-c4cd816fbdff"} : dispatch 2024-09-21T06:31:10.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:09 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2724283584' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85b9beed-c93d-48ba-bc7a-c4cd816fbdff"}]': finished 2024-09-21T06:31:10.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:09 smithi100 ceph-mon[33401]: osdmap e38: 7 total, 6 up, 7 in 2024-09-21T06:31:10.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:09 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:31:10.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:09 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1951997081' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:31:10.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:10 smithi203 ceph-mon[33735]: pgmap v151: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:31:11.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:10 smithi067 ceph-mon[31232]: pgmap v151: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:31:11.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:10 smithi100 ceph-mon[33401]: pgmap v151: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:31:13.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:12 smithi203 ceph-mon[33735]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-21T06:31:13.294 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:12 smithi067 ceph-mon[31232]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-21T06:31:13.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:12 smithi100 ceph-mon[33401]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-21T06:31:15.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:14 smithi203 ceph-mon[33735]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-21T06:31:15.263 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:14 smithi067 ceph-mon[31232]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-21T06:31:15.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:14 smithi100 ceph-mon[33401]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-21T06:31:17.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:16 smithi067 ceph-mon[31232]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:31:17.105 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:16 smithi100 ceph-mon[33401]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:31:17.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:16 smithi203 ceph-mon[33735]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:31:18.041 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:17 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T06:31:18.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:17 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:18.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:17 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T06:31:18.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:17 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:18.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:17 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T06:31:18.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:17 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:19.137 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:18 smithi067 ceph-mon[31232]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-21T06:31:19.137 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:18 smithi067 ceph-mon[31232]: Deploying daemon osd.6 on smithi067 2024-09-21T06:31:19.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:18 smithi203 ceph-mon[33735]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-21T06:31:19.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:18 smithi203 ceph-mon[33735]: Deploying daemon osd.6 on smithi067 2024-09-21T06:31:19.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:18 smithi100 ceph-mon[33401]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-21T06:31:19.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:18 smithi100 ceph-mon[33401]: Deploying daemon osd.6 on smithi067 2024-09-21T06:31:21.015 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:20 smithi067 ceph-mon[31232]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-21T06:31:21.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:20 smithi203 ceph-mon[33735]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-21T06:31:21.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:20 smithi100 ceph-mon[33401]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-21T06:31:22.949 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:22 smithi067 ceph-mon[31232]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:31:23.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:22 smithi203 ceph-mon[33735]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:31:23.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:22 smithi100 ceph-mon[33401]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:31:25.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:24 smithi067 ceph-mon[31232]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:31:25.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:24 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:31:25.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:24 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:25.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:24 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:25.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:24 smithi203 ceph-mon[33735]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:31:25.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:24 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:31:25.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:24 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:25.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:24 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:25.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:24 smithi100 ceph-mon[33401]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:31:25.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:24 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:31:25.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:24 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:25.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:24 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:27.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:26 smithi067 ceph-mon[31232]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:31:27.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:26 smithi203 ceph-mon[33735]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:31:27.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:26 smithi100 ceph-mon[33401]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:31:28.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:28.074+0000 7fb8ceffd640 1 -- 172.21.15.67:0/3955493893 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb8d0063800 con 0x7fb8b80b6590 2024-09-21T06:31:28.077 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 6 on host 'smithi067' 2024-09-21T06:31:28.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:28.076+0000 7fb8d7bae640 1 -- 172.21.15.67:0/3955493893 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb8b80b6590 msgr2=0x7fb8b80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:28.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:28.076+0000 7fb8d7bae640 1 --2- 172.21.15.67:0/3955493893 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb8b80b6590 0x7fb8b80b8aa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4005d00 tx=0x7fb8c400b410 comp rx=0 tx=0).stop 2024-09-21T06:31:28.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:28.076+0000 7fb8d7bae640 1 -- 172.21.15.67:0/3955493893 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8d006b980 msgr2=0x7fb8d0199f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:28.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:28.076+0000 7fb8d7bae640 1 --2- 172.21.15.67:0/3955493893 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8d006b980 0x7fb8d0199f30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8009230 tx=0x7fb8c80042f0 comp rx=0 tx=0).stop 2024-09-21T06:31:28.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:28.076+0000 7fb8d7bae640 1 -- 172.21.15.67:0/3955493893 shutdown_connections 2024-09-21T06:31:28.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:28.076+0000 7fb8d7bae640 1 --2- 172.21.15.67:0/3955493893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8d0109470 0x7fb8d019b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:28.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:28.076+0000 7fb8d7bae640 1 --2- 172.21.15.67:0/3955493893 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb8b80b6590 0x7fb8b80b8aa0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:28.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:28.076+0000 7fb8d7bae640 1 --2- 172.21.15.67:0/3955493893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8d01068d0 0x7fb8d019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:28.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:28.076+0000 7fb8d7bae640 1 --2- 172.21.15.67:0/3955493893 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8d006b980 0x7fb8d0199f30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:28.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:28.076+0000 7fb8d7bae640 1 -- 172.21.15.67:0/3955493893 >> 172.21.15.67:0/3955493893 conn(0x7fb8d00fdfb0 msgr2=0x7fb8d0104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:28.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:28.077+0000 7fb8d7bae640 1 -- 172.21.15.67:0/3955493893 shutdown_connections 2024-09-21T06:31:28.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:31:28.077+0000 7fb8d7bae640 1 -- 172.21.15.67:0/3955493893 wait complete. 2024-09-21T06:31:28.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:28 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:28.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:28 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:28.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:28 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:28.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:28 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:31:28.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:28 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:28.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:28 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:31:28.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:28 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:28.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:28 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:28.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:28 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:28.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:28 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:28.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:28 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:31:28.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:28 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:28.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:28 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:31:28.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:28 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:28.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:28 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:28.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:28 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:28.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:28 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:28.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:28 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:31:28.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:28 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:28.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:28 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:31:28.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:28 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:28.719 DEBUG:teuthology.orchestra.run.smithi067:osd.6> sudo journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.6.service 2024-09-21T06:31:28.721 INFO:tasks.cephadm:Deploying osd.7 on smithi100 with /dev/vg_nvme/lv_2... 2024-09-21T06:31:28.722 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T06:31:28.971 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:31:29.269 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:29 smithi067 ceph-mon[31232]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:31:29.269 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:29 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:29.328 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:29 smithi203 ceph-mon[33735]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:31:29.328 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:29 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:29.329 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:29 smithi100 ceph-mon[33401]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:31:29.329 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:29 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:30.214 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:30 smithi067 ceph-mon[31232]: from='osd.6 [v2:172.21.15.67:6818/3229936280,v1:172.21.15.67:6819/3229936280]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T06:31:30.214 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:31:29 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:31:29.862+0000 7f85c88ef640 -1 osd.6 0 log_to_monitors true 2024-09-21T06:31:30.376 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:30 smithi100 ceph-mon[33401]: from='osd.6 [v2:172.21.15.67:6818/3229936280,v1:172.21.15.67:6819/3229936280]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T06:31:30.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:30 smithi203 ceph-mon[33735]: from='osd.6 [v2:172.21.15.67:6818/3229936280,v1:172.21.15.67:6819/3229936280]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T06:31:31.473 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:31 smithi067 ceph-mon[31232]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:31:31.473 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:31 smithi067 ceph-mon[31232]: from='osd.6 [v2:172.21.15.67:6818/3229936280,v1:172.21.15.67:6819/3229936280]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T06:31:31.473 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:31 smithi067 ceph-mon[31232]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T06:31:31.473 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:31 smithi067 ceph-mon[31232]: from='osd.6 [v2:172.21.15.67:6818/3229936280,v1:172.21.15.67:6819/3229936280]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]} : dispatch 2024-09-21T06:31:31.473 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:31 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:31:31.474 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:31:31 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:31:31.090+0000 7f85c4871640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T06:31:31.474 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:31:31 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:31:31.103+0000 7f85c066f640 -1 osd.6 40 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:31:31.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:31 smithi203 ceph-mon[33735]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:31:31.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:31 smithi203 ceph-mon[33735]: from='osd.6 [v2:172.21.15.67:6818/3229936280,v1:172.21.15.67:6819/3229936280]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T06:31:31.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:31 smithi203 ceph-mon[33735]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T06:31:31.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:31 smithi203 ceph-mon[33735]: from='osd.6 [v2:172.21.15.67:6818/3229936280,v1:172.21.15.67:6819/3229936280]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]} : dispatch 2024-09-21T06:31:31.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:31 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:31:31.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:31 smithi100 ceph-mon[33401]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:31:31.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:31 smithi100 ceph-mon[33401]: from='osd.6 [v2:172.21.15.67:6818/3229936280,v1:172.21.15.67:6819/3229936280]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T06:31:31.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:31 smithi100 ceph-mon[33401]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T06:31:31.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:31 smithi100 ceph-mon[33401]: from='osd.6 [v2:172.21.15.67:6818/3229936280,v1:172.21.15.67:6819/3229936280]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]} : dispatch 2024-09-21T06:31:31.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:31 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:31:31.556 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-21T06:31:31.586 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi100:vg_nvme/lv_2 2024-09-21T06:31:31.841 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:31:32.122 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:32 smithi100 ceph-mon[33401]: from='osd.6 [v2:172.21.15.67:6818/3229936280,v1:172.21.15.67:6819/3229936280]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-21T06:31:32.122 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:32 smithi100 ceph-mon[33401]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T06:31:32.122 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:32 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:31:32.122 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:32 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:31:32.122 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:32 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:32.122 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:32 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:32.122 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:32 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:31:32.122 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:32 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:31:32.122 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:32 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:31:32.122 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:32 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:32.122 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:32 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:32.123 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:32 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:31:32.123 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:32 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:32.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:32 smithi067 ceph-mon[31232]: from='osd.6 [v2:172.21.15.67:6818/3229936280,v1:172.21.15.67:6819/3229936280]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-21T06:31:32.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:32 smithi067 ceph-mon[31232]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T06:31:32.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:32 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:31:32.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:32 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:31:32.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:32 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:32.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:32 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:32.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:32 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:31:32.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:32 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:31:32.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:32 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:31:32.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:32 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:32.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:32 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:32.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:32 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:31:32.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:32 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:32.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.307+0000 7fdc1c6a6640 1 -- 172.21.15.100:0/2068516016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc14108370 msgr2=0x7fdc1410a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:32.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.307+0000 7fdc1c6a6640 1 --2- 172.21.15.100:0/2068516016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc14108370 0x7fdc1410a880 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009ca0 tx=0x7fdc0c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:31:32.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.307+0000 7fdc1c6a6640 1 -- 172.21.15.100:0/2068516016 shutdown_connections 2024-09-21T06:31:32.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.307+0000 7fdc1c6a6640 1 --2- 172.21.15.100:0/2068516016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc14108370 0x7fdc1410a880 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:32.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.307+0000 7fdc1c6a6640 1 --2- 172.21.15.100:0/2068516016 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc14105750 0x7fdc14107c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:32.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.307+0000 7fdc1c6a6640 1 --2- 172.21.15.100:0/2068516016 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc14102d10 0x7fdc14105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:32.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.307+0000 7fdc1c6a6640 1 -- 172.21.15.100:0/2068516016 >> 172.21.15.100:0/2068516016 conn(0x7fdc140fd770 msgr2=0x7fdc140ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:32.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.307+0000 7fdc1c6a6640 1 -- 172.21.15.100:0/2068516016 shutdown_connections 2024-09-21T06:31:32.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.308+0000 7fdc1c6a6640 1 -- 172.21.15.100:0/2068516016 wait complete. 2024-09-21T06:31:32.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.308+0000 7fdc1c6a6640 1 Processor -- start 2024-09-21T06:31:32.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.308+0000 7fdc1c6a6640 1 -- start start 2024-09-21T06:31:32.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.308+0000 7fdc1c6a6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc14102d10 0x7fdc14198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:32.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.308+0000 7fdc1c6a6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc14105750 0x7fdc141993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:32.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.308+0000 7fdc1c6a6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc14108370 0x7fdc14199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:32.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.308+0000 7fdc1c6a6640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1410e180 con 0x7fdc14102d10 2024-09-21T06:31:32.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.308+0000 7fdc1c6a6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1410e000 con 0x7fdc14108370 2024-09-21T06:31:32.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.308+0000 7fdc1c6a6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1410e300 con 0x7fdc14105750 2024-09-21T06:31:32.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.309+0000 7fdc1ac43640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc14108370 0x7fdc14199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:32.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.309+0000 7fdc1ac43640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc14108370 0x7fdc14199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56564/0 (socket says 172.21.15.100:56564) 2024-09-21T06:31:32.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.309+0000 7fdc1ac43640 1 -- 172.21.15.100:0/384857603 learned_addr learned my addr 172.21.15.100:0/384857603 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-21T06:31:32.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.309+0000 7fdc1a442640 1 --2- 172.21.15.100:0/384857603 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc14102d10 0x7fdc14198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:32.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.309+0000 7fdc19c41640 1 --2- 172.21.15.100:0/384857603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc14105750 0x7fdc141993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:32.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.309+0000 7fdc1ac43640 1 -- 172.21.15.100:0/384857603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc14105750 msgr2=0x7fdc141993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:32.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.309+0000 7fdc1ac43640 1 --2- 172.21.15.100:0/384857603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc14105750 0x7fdc141993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:32.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.309+0000 7fdc1ac43640 1 -- 172.21.15.100:0/384857603 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc14102d10 msgr2=0x7fdc14198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:32.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.309+0000 7fdc1ac43640 1 --2- 172.21.15.100:0/384857603 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc14102d10 0x7fdc14198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:32.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.309+0000 7fdc1ac43640 1 -- 172.21.15.100:0/384857603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1419a780 con 0x7fdc14108370 2024-09-21T06:31:32.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.309+0000 7fdc1a442640 1 --2- 172.21.15.100:0/384857603 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc14102d10 0x7fdc14198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:31:32.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.309+0000 7fdc1ac43640 1 --2- 172.21.15.100:0/384857603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc14108370 0x7fdc14199f60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c005e30 tx=0x7fdc0c003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:32.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.309+0000 7fdc077fe640 1 -- 172.21.15.100:0/384857603 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c032e90 con 0x7fdc14108370 2024-09-21T06:31:32.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.310+0000 7fdc1c6a6640 1 -- 172.21.15.100:0/384857603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc1419aa10 con 0x7fdc14108370 2024-09-21T06:31:32.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.310+0000 7fdc077fe640 1 -- 172.21.15.100:0/384857603 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdc0c0044d0 con 0x7fdc14108370 2024-09-21T06:31:32.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.310+0000 7fdc1c6a6640 1 -- 172.21.15.100:0/384857603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc14101b40 con 0x7fdc14108370 2024-09-21T06:31:32.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.310+0000 7fdc077fe640 1 -- 172.21.15.100:0/384857603 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c030ce0 con 0x7fdc14108370 2024-09-21T06:31:32.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.311+0000 7fdc1c6a6640 1 -- 172.21.15.100:0/384857603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbf80052c0 con 0x7fdc14108370 2024-09-21T06:31:32.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.312+0000 7fdc077fe640 1 -- 172.21.15.100:0/384857603 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fdc0c004680 con 0x7fdc14108370 2024-09-21T06:31:32.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.313+0000 7fdc077fe640 1 --2- 172.21.15.100:0/384857603 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fdc000b6510 0x7fdc000b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:32.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.313+0000 7fdc077fe640 1 -- 172.21.15.100:0/384857603 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4965+0+0 (secure 0 0 0) 0x7fdc0c107940 con 0x7fdc14108370 2024-09-21T06:31:32.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.313+0000 7fdc1a442640 1 --2- 172.21.15.100:0/384857603 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fdc000b6510 0x7fdc000b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:32.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.314+0000 7fdc1a442640 1 --2- 172.21.15.100:0/384857603 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fdc000b6510 0x7fdc000b8a20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdc08005fd0 tx=0x7fdc0800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:32.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.316+0000 7fdc077fe640 1 -- 172.21.15.100:0/384857603 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdc0c0cef70 con 0x7fdc14108370 2024-09-21T06:31:32.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:32 smithi203 ceph-mon[33735]: from='osd.6 [v2:172.21.15.67:6818/3229936280,v1:172.21.15.67:6819/3229936280]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-21T06:31:32.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:32 smithi203 ceph-mon[33735]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T06:31:32.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:32 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:31:32.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:32 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:31:32.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:32 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:32.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:32 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:32.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:32 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:31:32.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:32 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:31:32.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:32 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:31:32.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:32 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:32.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:32 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:32.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:32 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:31:32.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:32 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:32.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:32.510+0000 7fdc1c6a6640 1 -- 172.21.15.100:0/384857603 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fdbf8000bf0 con 0x7fdc000b6510 2024-09-21T06:31:33.210 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:33 smithi100 ceph-mon[33401]: purged_snaps scrub starts 2024-09-21T06:31:33.210 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:33 smithi100 ceph-mon[33401]: purged_snaps scrub ok 2024-09-21T06:31:33.210 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:33 smithi100 ceph-mon[33401]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:31:33.210 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:33 smithi100 ceph-mon[33401]: Detected new or changed devices on smithi067 2024-09-21T06:31:33.210 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:33 smithi100 ceph-mon[33401]: Adjusting osd_memory_target on smithi067 to 5696M 2024-09-21T06:31:33.211 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:33 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:31:33.211 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:33 smithi100 ceph-mon[33401]: osd.6 [v2:172.21.15.67:6818/3229936280,v1:172.21.15.67:6819/3229936280] boot 2024-09-21T06:31:33.211 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:33 smithi100 ceph-mon[33401]: osdmap e41: 7 total, 7 up, 7 in 2024-09-21T06:31:33.211 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:33 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:31:33.211 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:33 smithi100 ceph-mon[33401]: from='client.24443 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:31:33.211 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:33 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:31:33.211 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:33 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:31:33.211 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:33 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:33.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:33 smithi203 ceph-mon[33735]: purged_snaps scrub starts 2024-09-21T06:31:33.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:33 smithi203 ceph-mon[33735]: purged_snaps scrub ok 2024-09-21T06:31:33.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:33 smithi203 ceph-mon[33735]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:31:33.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:33 smithi203 ceph-mon[33735]: Detected new or changed devices on smithi067 2024-09-21T06:31:33.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:33 smithi203 ceph-mon[33735]: Adjusting osd_memory_target on smithi067 to 5696M 2024-09-21T06:31:33.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:33 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:31:33.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:33 smithi203 ceph-mon[33735]: osd.6 [v2:172.21.15.67:6818/3229936280,v1:172.21.15.67:6819/3229936280] boot 2024-09-21T06:31:33.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:33 smithi203 ceph-mon[33735]: osdmap e41: 7 total, 7 up, 7 in 2024-09-21T06:31:33.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:33 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:31:33.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:33 smithi203 ceph-mon[33735]: from='client.24443 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:31:33.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:33 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:31:33.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:33 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:31:33.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:33 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:33.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:33 smithi067 ceph-mon[31232]: purged_snaps scrub starts 2024-09-21T06:31:33.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:33 smithi067 ceph-mon[31232]: purged_snaps scrub ok 2024-09-21T06:31:33.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:33 smithi067 ceph-mon[31232]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:31:33.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:33 smithi067 ceph-mon[31232]: Detected new or changed devices on smithi067 2024-09-21T06:31:33.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:33 smithi067 ceph-mon[31232]: Adjusting osd_memory_target on smithi067 to 5696M 2024-09-21T06:31:33.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:33 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:31:33.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:33 smithi067 ceph-mon[31232]: osd.6 [v2:172.21.15.67:6818/3229936280,v1:172.21.15.67:6819/3229936280] boot 2024-09-21T06:31:33.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:33 smithi067 ceph-mon[31232]: osdmap e41: 7 total, 7 up, 7 in 2024-09-21T06:31:33.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:33 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:31:33.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:33 smithi067 ceph-mon[31232]: from='client.24443 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:31:33.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:33 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:31:33.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:33 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:31:33.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:33 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:34.409 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:34 smithi100 ceph-mon[33401]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T06:31:34.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:34 smithi203 ceph-mon[33735]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T06:31:34.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:34 smithi067 ceph-mon[31232]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T06:31:35.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:35 smithi203 ceph-mon[33735]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:35.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:35 smithi203 ceph-mon[33735]: from='client.? 172.21.15.100:0/3508232859' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "86de30c2-b8ee-4c9d-9c17-8a56517934ee"} : dispatch 2024-09-21T06:31:35.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:35 smithi203 ceph-mon[33735]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "86de30c2-b8ee-4c9d-9c17-8a56517934ee"} : dispatch 2024-09-21T06:31:35.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:35 smithi203 ceph-mon[33735]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86de30c2-b8ee-4c9d-9c17-8a56517934ee"}]': finished 2024-09-21T06:31:35.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:35 smithi203 ceph-mon[33735]: osdmap e43: 8 total, 7 up, 8 in 2024-09-21T06:31:35.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:35 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:31:35.519 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:35 smithi100 ceph-mon[33401]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:35.519 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:35 smithi100 ceph-mon[33401]: from='client.? 172.21.15.100:0/3508232859' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "86de30c2-b8ee-4c9d-9c17-8a56517934ee"} : dispatch 2024-09-21T06:31:35.519 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:35 smithi100 ceph-mon[33401]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "86de30c2-b8ee-4c9d-9c17-8a56517934ee"} : dispatch 2024-09-21T06:31:35.519 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:35 smithi100 ceph-mon[33401]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86de30c2-b8ee-4c9d-9c17-8a56517934ee"}]': finished 2024-09-21T06:31:35.519 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:35 smithi100 ceph-mon[33401]: osdmap e43: 8 total, 7 up, 8 in 2024-09-21T06:31:35.519 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:35 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:31:35.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:35 smithi067 ceph-mon[31232]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:35.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:35 smithi067 ceph-mon[31232]: from='client.? 172.21.15.100:0/3508232859' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "86de30c2-b8ee-4c9d-9c17-8a56517934ee"} : dispatch 2024-09-21T06:31:35.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:35 smithi067 ceph-mon[31232]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "86de30c2-b8ee-4c9d-9c17-8a56517934ee"} : dispatch 2024-09-21T06:31:35.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:35 smithi067 ceph-mon[31232]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86de30c2-b8ee-4c9d-9c17-8a56517934ee"}]': finished 2024-09-21T06:31:35.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:35 smithi067 ceph-mon[31232]: osdmap e43: 8 total, 7 up, 8 in 2024-09-21T06:31:35.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:31:36.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:36 smithi203 ceph-mon[33735]: from='client.? 172.21.15.100:0/2996949158' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:31:36.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:36 smithi067 ceph-mon[31232]: from='client.? 172.21.15.100:0/2996949158' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:31:36.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:36 smithi100 ceph-mon[33401]: from='client.? 172.21.15.100:0/2996949158' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:31:37.443 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:37 smithi067 ceph-mon[31232]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:37.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:37 smithi203 ceph-mon[33735]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:37.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:37 smithi100 ceph-mon[33401]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:39.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:39 smithi203 ceph-mon[33735]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:39.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:39 smithi067 ceph-mon[31232]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:39.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:39 smithi100 ceph-mon[33401]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:41.469 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:41 smithi100 ceph-mon[33401]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:41.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:41 smithi203 ceph-mon[33735]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:41.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:41 smithi067 ceph-mon[31232]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:43.482 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:43 smithi100 ceph-mon[33401]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:43.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:43 smithi203 ceph-mon[33735]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:43.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:43 smithi067 ceph-mon[31232]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:44.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:44 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T06:31:44.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:44 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:44.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:44 smithi203 ceph-mon[33735]: Deploying daemon osd.7 on smithi100 2024-09-21T06:31:44.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:44 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T06:31:44.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:44 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:44.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:44 smithi067 ceph-mon[31232]: Deploying daemon osd.7 on smithi100 2024-09-21T06:31:44.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:44 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T06:31:44.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:44 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:44.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:44 smithi100 ceph-mon[33401]: Deploying daemon osd.7 on smithi100 2024-09-21T06:31:45.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:45 smithi203 ceph-mon[33735]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:45.492 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:45 smithi100 ceph-mon[33401]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:45.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:45 smithi067 ceph-mon[31232]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:47.251 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:47 smithi100 ceph-mon[33401]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:47.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:47 smithi203 ceph-mon[33735]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:47.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:47 smithi067 ceph-mon[31232]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:49.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:49 smithi100 ceph-mon[33401]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:49.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:49 smithi203 ceph-mon[33735]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:49.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:49 smithi067 ceph-mon[31232]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:50.286 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:50 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:31:50.286 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:50 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:50.286 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:50 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:50.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:50 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:31:50.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:50 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:50.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:50 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:50.541 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:50 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:31:50.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:50 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:50.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:50 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:51.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:51 smithi203 ceph-mon[33735]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:51.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:51 smithi067 ceph-mon[31232]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:51.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:51 smithi100 ceph-mon[33401]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:53.413 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:53 smithi100 ceph-mon[33401]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:53.413 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:53 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:53.413 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:53 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:53.413 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:53 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:53.413 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:53 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:31:53.413 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:53 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:53.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:53.453+0000 7fdc077fe640 1 -- 172.21.15.100:0/384857603 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdbf8000bf0 con 0x7fdc000b6510 2024-09-21T06:31:53.456 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 7 on host 'smithi100' 2024-09-21T06:31:53.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:53.456+0000 7fdc1c6a6640 1 -- 172.21.15.100:0/384857603 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fdc000b6510 msgr2=0x7fdc000b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:53.456+0000 7fdc1c6a6640 1 --2- 172.21.15.100:0/384857603 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fdc000b6510 0x7fdc000b8a20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdc08005fd0 tx=0x7fdc0800b390 comp rx=0 tx=0).stop 2024-09-21T06:31:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:53.456+0000 7fdc1c6a6640 1 -- 172.21.15.100:0/384857603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc14108370 msgr2=0x7fdc14199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:53.456+0000 7fdc1c6a6640 1 --2- 172.21.15.100:0/384857603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc14108370 0x7fdc14199f60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c005e30 tx=0x7fdc0c003ba0 comp rx=0 tx=0).stop 2024-09-21T06:31:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:53.456+0000 7fdc1c6a6640 1 -- 172.21.15.100:0/384857603 shutdown_connections 2024-09-21T06:31:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:53.456+0000 7fdc1c6a6640 1 --2- 172.21.15.100:0/384857603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc14108370 0x7fdc14199f60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:53.456+0000 7fdc1c6a6640 1 --2- 172.21.15.100:0/384857603 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fdc000b6510 0x7fdc000b8a20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:53.456+0000 7fdc1c6a6640 1 --2- 172.21.15.100:0/384857603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc14105750 0x7fdc141993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:53.456+0000 7fdc1c6a6640 1 --2- 172.21.15.100:0/384857603 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc14102d10 0x7fdc14198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:53.456+0000 7fdc1c6a6640 1 -- 172.21.15.100:0/384857603 >> 172.21.15.100:0/384857603 conn(0x7fdc140fd770 msgr2=0x7fdc14103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:53.456+0000 7fdc1c6a6640 1 -- 172.21.15.100:0/384857603 shutdown_connections 2024-09-21T06:31:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:31:53.456+0000 7fdc1c6a6640 1 -- 172.21.15.100:0/384857603 wait complete. 2024-09-21T06:31:53.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:53 smithi203 ceph-mon[33735]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:53.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:53 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:53.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:53 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:53.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:53 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:53.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:53 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:31:53.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:53 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:53.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:53 smithi067 ceph-mon[31232]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:53.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:53 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:53.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:53 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:53.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:53 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:53.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:53 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:31:53.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:53 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:54.079 DEBUG:teuthology.orchestra.run.smithi100:osd.7> sudo journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.7.service 2024-09-21T06:31:54.082 INFO:tasks.cephadm:Deploying osd.8 on smithi203 with /dev/vg_nvme/lv_2... 2024-09-21T06:31:54.082 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T06:31:54.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:54 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:31:54.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:54 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:54.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:54 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:54.339 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:31:54.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:54 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:31:54.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:54 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:54.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:54 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:54.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:54 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:31:54.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:54 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:54.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:54 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:55.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:55 smithi100 ceph-mon[33401]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:55.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:55 smithi100 ceph-mon[33401]: from='osd.7 [v2:172.21.15.100:6816/1622917909,v1:172.21.15.100:6817/1622917909]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T06:31:55.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:55 smithi100 ceph-mon[33401]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T06:31:55.554 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:31:55 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:31:55.419+0000 7fb251c93640 -1 osd.7 0 log_to_monitors true 2024-09-21T06:31:55.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:55 smithi203 ceph-mon[33735]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:55.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:55 smithi203 ceph-mon[33735]: from='osd.7 [v2:172.21.15.100:6816/1622917909,v1:172.21.15.100:6817/1622917909]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T06:31:55.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:55 smithi203 ceph-mon[33735]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T06:31:55.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:55 smithi067 ceph-mon[31232]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:55.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:55 smithi067 ceph-mon[31232]: from='osd.7 [v2:172.21.15.100:6816/1622917909,v1:172.21.15.100:6817/1622917909]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T06:31:55.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:55 smithi067 ceph-mon[31232]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T06:31:56.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:56 smithi100 ceph-mon[33401]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T06:31:56.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:56 smithi100 ceph-mon[33401]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T06:31:56.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:56 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:31:56.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:56 smithi100 ceph-mon[33401]: from='osd.7 [v2:172.21.15.100:6816/1622917909,v1:172.21.15.100:6817/1622917909]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]} : dispatch 2024-09-21T06:31:56.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:56 smithi100 ceph-mon[33401]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]} : dispatch 2024-09-21T06:31:56.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:56 smithi203 ceph-mon[33735]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T06:31:56.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:56 smithi203 ceph-mon[33735]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T06:31:56.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:56 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:31:56.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:56 smithi203 ceph-mon[33735]: from='osd.7 [v2:172.21.15.100:6816/1622917909,v1:172.21.15.100:6817/1622917909]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]} : dispatch 2024-09-21T06:31:56.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:56 smithi203 ceph-mon[33735]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]} : dispatch 2024-09-21T06:31:56.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:56 smithi067 ceph-mon[31232]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T06:31:56.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:56 smithi067 ceph-mon[31232]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T06:31:56.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:56 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:31:56.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:56 smithi067 ceph-mon[31232]: from='osd.7 [v2:172.21.15.100:6816/1622917909,v1:172.21.15.100:6817/1622917909]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]} : dispatch 2024-09-21T06:31:56.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:56 smithi067 ceph-mon[31232]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]} : dispatch 2024-09-21T06:31:56.900 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T06:31:56.930 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi203:vg_nvme/lv_2 2024-09-21T06:31:57.185 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:31:57.303 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:31:56 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:31:56.939+0000 7fb24dc15640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T06:31:57.304 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:31:56 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:31:56.952+0000 7fb245a0b640 -1 osd.7 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:31:57.601 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:57 smithi203 ceph-mon[33735]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:57.602 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:57 smithi203 ceph-mon[33735]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-21T06:31:57.602 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:57 smithi203 ceph-mon[33735]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T06:31:57.602 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:31:57.602 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:31:57.602 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:57.602 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:57.602 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:31:57.602 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:31:57.602 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:31:57.602 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:57.602 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:57.602 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:31:57.603 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:57 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:57.603 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:57 smithi203 ceph-mon[33735]: from='osd.7 ' entity='osd.7' 2024-09-21T06:31:57.701 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.692+0000 7f4c9aff8640 1 -- 172.21.15.203:0/3143152129 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c94105790 msgr2=0x7f4c94107c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:57.701 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.692+0000 7f4c9aff8640 1 --2- 172.21.15.203:0/3143152129 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c94105790 0x7f4c94107c70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4c80009900 tx=0x7f4c8002a410 comp rx=0 tx=0).stop 2024-09-21T06:31:57.701 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.692+0000 7f4c9aff8640 1 -- 172.21.15.203:0/3143152129 shutdown_connections 2024-09-21T06:31:57.702 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.692+0000 7f4c9aff8640 1 --2- 172.21.15.203:0/3143152129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c941083b0 0x7f4c9410a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:57.702 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.692+0000 7f4c9aff8640 1 --2- 172.21.15.203:0/3143152129 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c94105790 0x7f4c94107c70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:57.702 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.692+0000 7f4c9aff8640 1 --2- 172.21.15.203:0/3143152129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c94102d30 0x7f4c94105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:57.702 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.692+0000 7f4c9aff8640 1 -- 172.21.15.203:0/3143152129 >> 172.21.15.203:0/3143152129 conn(0x7f4c940fd770 msgr2=0x7f4c940ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:31:57.702 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.692+0000 7f4c9aff8640 1 -- 172.21.15.203:0/3143152129 shutdown_connections 2024-09-21T06:31:57.702 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.692+0000 7f4c9aff8640 1 -- 172.21.15.203:0/3143152129 wait complete. 2024-09-21T06:31:57.702 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.693+0000 7f4c9aff8640 1 Processor -- start 2024-09-21T06:31:57.702 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.693+0000 7f4c9aff8640 1 -- start start 2024-09-21T06:31:57.702 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.693+0000 7f4c9aff8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c94102d30 0x7f4c94102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:57.702 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.693+0000 7f4c9aff8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c94105790 0x7f4c9419ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:57.702 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.693+0000 7f4c9aff8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c941083b0 0x7f4c941a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:57.702 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.693+0000 7f4c9aff8640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9410e170 con 0x7f4c94105790 2024-09-21T06:31:57.702 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.693+0000 7f4c9aff8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9410dff0 con 0x7f4c941083b0 2024-09-21T06:31:57.703 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.693+0000 7f4c9aff8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9410e2f0 con 0x7f4c94102d30 2024-09-21T06:31:57.703 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.693+0000 7f4c98d94640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c94102d30 0x7f4c94102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:57.703 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.693+0000 7f4c98d94640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c94102d30 0x7f4c94102230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:39048/0 (socket says 172.21.15.203:39048) 2024-09-21T06:31:57.703 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.693+0000 7f4c98d94640 1 -- 172.21.15.203:0/254752448 learned_addr learned my addr 172.21.15.203:0/254752448 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-09-21T06:31:57.703 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.693+0000 7f4c93fff640 1 --2- 172.21.15.203:0/254752448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c94105790 0x7f4c9419ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:57.703 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.693+0000 7f4c99595640 1 --2- 172.21.15.203:0/254752448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c941083b0 0x7f4c941a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:57.703 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.694+0000 7f4c98d94640 1 -- 172.21.15.203:0/254752448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c941083b0 msgr2=0x7f4c941a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:57.703 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.694+0000 7f4c98d94640 1 --2- 172.21.15.203:0/254752448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c941083b0 0x7f4c941a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:57.703 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.694+0000 7f4c98d94640 1 -- 172.21.15.203:0/254752448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c94105790 msgr2=0x7f4c9419ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:31:57.703 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.694+0000 7f4c98d94640 1 --2- 172.21.15.203:0/254752448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c94105790 0x7f4c9419ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:31:57.703 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.694+0000 7f4c98d94640 1 -- 172.21.15.203:0/254752448 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c941a19c0 con 0x7f4c94102d30 2024-09-21T06:31:57.703 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.694+0000 7f4c99595640 1 --2- 172.21.15.203:0/254752448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c941083b0 0x7f4c941a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:31:57.703 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.694+0000 7f4c98d94640 1 --2- 172.21.15.203:0/254752448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c94102d30 0x7f4c94102230 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800d950 tx=0x7f4c8800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:57.703 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.694+0000 7f4c91ffb640 1 -- 172.21.15.203:0/254752448 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c88017070 con 0x7f4c94102d30 2024-09-21T06:31:57.704 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.694+0000 7f4c91ffb640 1 -- 172.21.15.203:0/254752448 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c88009c30 con 0x7f4c94102d30 2024-09-21T06:31:57.704 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.694+0000 7f4c9aff8640 1 -- 172.21.15.203:0/254752448 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c941a1cb0 con 0x7f4c94102d30 2024-09-21T06:31:57.704 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.694+0000 7f4c91ffb640 1 -- 172.21.15.203:0/254752448 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8800f620 con 0x7f4c94102d30 2024-09-21T06:31:57.704 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.694+0000 7f4c9aff8640 1 -- 172.21.15.203:0/254752448 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c941a21e0 con 0x7f4c94102d30 2024-09-21T06:31:57.704 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.695+0000 7f4c9aff8640 1 -- 172.21.15.203:0/254752448 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c940ff290 con 0x7f4c94102d30 2024-09-21T06:31:57.704 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.697+0000 7f4c91ffb640 1 -- 172.21.15.203:0/254752448 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f4c88010460 con 0x7f4c94102d30 2024-09-21T06:31:57.704 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.698+0000 7f4c91ffb640 1 --2- 172.21.15.203:0/254752448 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f4c840b6590 0x7f4c840b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:31:57.704 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.698+0000 7f4c91ffb640 1 -- 172.21.15.203:0/254752448 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5430+0+0 (secure 0 0 0) 0x7f4c880e6340 con 0x7f4c94102d30 2024-09-21T06:31:57.704 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.698+0000 7f4c93fff640 1 --2- 172.21.15.203:0/254752448 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f4c840b6590 0x7f4c840b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:31:57.704 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.699+0000 7f4c93fff640 1 --2- 172.21.15.203:0/254752448 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f4c840b6590 0x7f4c840b8aa0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4c800057c0 tx=0x7f4c80000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:31:57.704 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.702+0000 7f4c91ffb640 1 -- 172.21.15.203:0/254752448 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4c880ad710 con 0x7f4c94102d30 2024-09-21T06:31:57.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:57 smithi067 ceph-mon[31232]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:57.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:57 smithi067 ceph-mon[31232]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-21T06:31:57.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:57 smithi067 ceph-mon[31232]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T06:31:57.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:31:57.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:31:57.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:57.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:57.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:31:57.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:31:57.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:31:57.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:57.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:57.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:31:57.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:57 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:57.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:57 smithi067 ceph-mon[31232]: from='osd.7 ' entity='osd.7' 2024-09-21T06:31:57.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:57 smithi100 ceph-mon[33401]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:31:57.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:57 smithi100 ceph-mon[33401]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-21T06:31:57.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:57 smithi100 ceph-mon[33401]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T06:31:57.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:31:57.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:31:57.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:57.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:57.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:31:57.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:31:57.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:31:57.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:57.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:57.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:31:57.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:57 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:31:57.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:57 smithi100 ceph-mon[33401]: from='osd.7 ' entity='osd.7' 2024-09-21T06:31:57.891 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:31:57.888+0000 7f4c9aff8640 1 -- 172.21.15.203:0/254752448 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f4c94063800 con 0x7f4c840b6590 2024-09-21T06:31:58.572 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:58 smithi203 ceph-mon[33735]: purged_snaps scrub starts 2024-09-21T06:31:58.572 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:58 smithi203 ceph-mon[33735]: purged_snaps scrub ok 2024-09-21T06:31:58.572 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:58 smithi203 ceph-mon[33735]: Detected new or changed devices on smithi100 2024-09-21T06:31:58.572 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:58 smithi203 ceph-mon[33735]: Adjusting osd_memory_target on smithi100 to 5695M 2024-09-21T06:31:58.572 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:58 smithi203 ceph-mon[33735]: osd.7 [v2:172.21.15.100:6816/1622917909,v1:172.21.15.100:6817/1622917909] boot 2024-09-21T06:31:58.572 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:58 smithi203 ceph-mon[33735]: osdmap e46: 8 total, 8 up, 8 in 2024-09-21T06:31:58.572 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:58 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:31:58.572 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:58 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:31:58.573 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:58 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:31:58.573 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:58 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:58.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:58 smithi067 ceph-mon[31232]: purged_snaps scrub starts 2024-09-21T06:31:58.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:58 smithi067 ceph-mon[31232]: purged_snaps scrub ok 2024-09-21T06:31:58.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:58 smithi067 ceph-mon[31232]: Detected new or changed devices on smithi100 2024-09-21T06:31:58.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:58 smithi067 ceph-mon[31232]: Adjusting osd_memory_target on smithi100 to 5695M 2024-09-21T06:31:58.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:58 smithi067 ceph-mon[31232]: osd.7 [v2:172.21.15.100:6816/1622917909,v1:172.21.15.100:6817/1622917909] boot 2024-09-21T06:31:58.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:58 smithi067 ceph-mon[31232]: osdmap e46: 8 total, 8 up, 8 in 2024-09-21T06:31:58.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:58 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:31:58.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:58 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:31:58.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:58 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:31:58.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:58 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:58.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:58 smithi100 ceph-mon[33401]: purged_snaps scrub starts 2024-09-21T06:31:58.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:58 smithi100 ceph-mon[33401]: purged_snaps scrub ok 2024-09-21T06:31:58.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:58 smithi100 ceph-mon[33401]: Detected new or changed devices on smithi100 2024-09-21T06:31:58.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:58 smithi100 ceph-mon[33401]: Adjusting osd_memory_target on smithi100 to 5695M 2024-09-21T06:31:58.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:58 smithi100 ceph-mon[33401]: osd.7 [v2:172.21.15.100:6816/1622917909,v1:172.21.15.100:6817/1622917909] boot 2024-09-21T06:31:58.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:58 smithi100 ceph-mon[33401]: osdmap e46: 8 total, 8 up, 8 in 2024-09-21T06:31:58.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:58 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:31:58.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:58 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:31:58.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:58 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:31:58.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:58 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:31:59.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:59 smithi067 ceph-mon[31232]: pgmap v183: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:31:59.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:59 smithi067 ceph-mon[31232]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:31:59.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:31:59 smithi067 ceph-mon[31232]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T06:31:59.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:59 smithi100 ceph-mon[33401]: pgmap v183: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:31:59.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:59 smithi100 ceph-mon[33401]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:31:59.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:31:59 smithi100 ceph-mon[33401]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T06:31:59.886 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:59 smithi203 ceph-mon[33735]: pgmap v183: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:31:59.887 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:59 smithi203 ceph-mon[33735]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:31:59.887 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:31:59 smithi203 ceph-mon[33735]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T06:32:00.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:00 smithi067 ceph-mon[31232]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T06:32:00.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:00 smithi067 ceph-mon[31232]: from='client.? 172.21.15.203:0/686365252' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0e96cdad-a135-4115-9e2d-9a28306700e9"} : dispatch 2024-09-21T06:32:00.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:00 smithi067 ceph-mon[31232]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0e96cdad-a135-4115-9e2d-9a28306700e9"} : dispatch 2024-09-21T06:32:00.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:00 smithi067 ceph-mon[31232]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e96cdad-a135-4115-9e2d-9a28306700e9"}]': finished 2024-09-21T06:32:00.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:00 smithi067 ceph-mon[31232]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T06:32:00.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:00 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:32:00.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:00 smithi100 ceph-mon[33401]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T06:32:00.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:00 smithi100 ceph-mon[33401]: from='client.? 172.21.15.203:0/686365252' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0e96cdad-a135-4115-9e2d-9a28306700e9"} : dispatch 2024-09-21T06:32:00.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:00 smithi100 ceph-mon[33401]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0e96cdad-a135-4115-9e2d-9a28306700e9"} : dispatch 2024-09-21T06:32:00.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:00 smithi100 ceph-mon[33401]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e96cdad-a135-4115-9e2d-9a28306700e9"}]': finished 2024-09-21T06:32:00.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:00 smithi100 ceph-mon[33401]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T06:32:00.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:00 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:32:00.867 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:00 smithi203 ceph-mon[33735]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T06:32:00.867 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:00 smithi203 ceph-mon[33735]: from='client.? 172.21.15.203:0/686365252' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0e96cdad-a135-4115-9e2d-9a28306700e9"} : dispatch 2024-09-21T06:32:00.867 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:00 smithi203 ceph-mon[33735]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0e96cdad-a135-4115-9e2d-9a28306700e9"} : dispatch 2024-09-21T06:32:00.867 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:00 smithi203 ceph-mon[33735]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e96cdad-a135-4115-9e2d-9a28306700e9"}]': finished 2024-09-21T06:32:00.867 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:00 smithi203 ceph-mon[33735]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T06:32:00.867 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:00 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:32:01.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:01 smithi067 ceph-mon[31232]: pgmap v186: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:32:01.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:01 smithi067 ceph-mon[31232]: from='client.? 172.21.15.203:0/1705626458' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:32:01.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:01 smithi100 ceph-mon[33401]: pgmap v186: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:32:01.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:01 smithi100 ceph-mon[33401]: from='client.? 172.21.15.203:0/1705626458' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:32:01.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:01 smithi203 ceph-mon[33735]: pgmap v186: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:32:01.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:01 smithi203 ceph-mon[33735]: from='client.? 172.21.15.203:0/1705626458' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:32:03.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:03 smithi067 ceph-mon[31232]: pgmap v188: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:32:03.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:03 smithi100 ceph-mon[33401]: pgmap v188: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:32:03.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:03 smithi203 ceph-mon[33735]: pgmap v188: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:32:05.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:05 smithi203 ceph-mon[33735]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 192 KiB/s, 0 objects/s recovering 2024-09-21T06:32:05.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:05 smithi067 ceph-mon[31232]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 192 KiB/s, 0 objects/s recovering 2024-09-21T06:32:05.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:05 smithi100 ceph-mon[33401]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 192 KiB/s, 0 objects/s recovering 2024-09-21T06:32:07.557 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:07 smithi203 ceph-mon[33735]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 157 KiB/s, 0 objects/s recovering 2024-09-21T06:32:07.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:07 smithi067 ceph-mon[31232]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 157 KiB/s, 0 objects/s recovering 2024-09-21T06:32:07.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:07 smithi100 ceph-mon[33401]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 157 KiB/s, 0 objects/s recovering 2024-09-21T06:32:09.658 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:09 smithi203 ceph-mon[33735]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-09-21T06:32:09.658 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:09 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T06:32:09.659 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:09 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:32:09.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:09 smithi067 ceph-mon[31232]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-09-21T06:32:09.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:09 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T06:32:09.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:09 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:32:09.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:09 smithi100 ceph-mon[33401]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-09-21T06:32:09.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:09 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T06:32:09.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:09 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:32:10.673 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:10 smithi203 ceph-mon[33735]: Deploying daemon osd.8 on smithi203 2024-09-21T06:32:10.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:10 smithi067 ceph-mon[31232]: Deploying daemon osd.8 on smithi203 2024-09-21T06:32:10.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:10 smithi100 ceph-mon[33401]: Deploying daemon osd.8 on smithi203 2024-09-21T06:32:11.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:11 smithi203 ceph-mon[33735]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-21T06:32:11.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:11 smithi067 ceph-mon[31232]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-21T06:32:11.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:11 smithi100 ceph-mon[33401]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-21T06:32:13.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:13 smithi067 ceph-mon[31232]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-21T06:32:13.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:13 smithi100 ceph-mon[33401]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-21T06:32:13.909 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:13 smithi203 ceph-mon[33735]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-21T06:32:15.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:15 smithi067 ceph-mon[31232]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:32:15.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:15 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:32:15.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:15 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:15.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:15 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:15.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:15 smithi100 ceph-mon[33401]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:32:15.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:15 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:32:15.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:15 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:15.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:15 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:15.831 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:15 smithi203 ceph-mon[33735]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:32:15.831 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:15 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:32:15.831 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:15 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:15.831 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:15 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:17.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:17 smithi067 ceph-mon[31232]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:32:17.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:17 smithi100 ceph-mon[33401]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:32:17.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:17 smithi203 ceph-mon[33735]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:32:18.960 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:32:18.958+0000 7f4c91ffb640 1 -- 172.21.15.203:0/254752448 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4c94063800 con 0x7f4c840b6590 2024-09-21T06:32:18.961 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 8 on host 'smithi203' 2024-09-21T06:32:18.963 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:32:18.961+0000 7f4c9aff8640 1 -- 172.21.15.203:0/254752448 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f4c840b6590 msgr2=0x7f4c840b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:18.963 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:32:18.961+0000 7f4c9aff8640 1 --2- 172.21.15.203:0/254752448 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f4c840b6590 0x7f4c840b8aa0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4c800057c0 tx=0x7f4c80000bc0 comp rx=0 tx=0).stop 2024-09-21T06:32:18.963 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:32:18.961+0000 7f4c9aff8640 1 -- 172.21.15.203:0/254752448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c94102d30 msgr2=0x7f4c94102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:18.963 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:32:18.961+0000 7f4c9aff8640 1 --2- 172.21.15.203:0/254752448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c94102d30 0x7f4c94102230 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800d950 tx=0x7f4c8800de10 comp rx=0 tx=0).stop 2024-09-21T06:32:18.963 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:32:18.961+0000 7f4c9aff8640 1 -- 172.21.15.203:0/254752448 shutdown_connections 2024-09-21T06:32:18.963 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:32:18.961+0000 7f4c9aff8640 1 --2- 172.21.15.203:0/254752448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c941083b0 0x7f4c941a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:18.963 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:32:18.961+0000 7f4c9aff8640 1 --2- 172.21.15.203:0/254752448 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f4c840b6590 0x7f4c840b8aa0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:18.964 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:32:18.961+0000 7f4c9aff8640 1 --2- 172.21.15.203:0/254752448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c94105790 0x7f4c9419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:18.964 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:32:18.961+0000 7f4c9aff8640 1 --2- 172.21.15.203:0/254752448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c94102d30 0x7f4c94102230 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:18.964 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:32:18.961+0000 7f4c9aff8640 1 -- 172.21.15.203:0/254752448 >> 172.21.15.203:0/254752448 conn(0x7f4c940fd770 msgr2=0x7f4c94103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:18.964 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:32:18.961+0000 7f4c9aff8640 1 -- 172.21.15.203:0/254752448 shutdown_connections 2024-09-21T06:32:18.964 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:32:18.961+0000 7f4c9aff8640 1 -- 172.21.15.203:0/254752448 wait complete. 2024-09-21T06:32:19.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:18 smithi203 ceph-mon[33735]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:32:19.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:18 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:19.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:18 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:19.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:18 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:32:19.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:18 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:32:19.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:18 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:19.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:18 smithi067 ceph-mon[31232]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:32:19.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:18 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:19.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:18 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:19.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:18 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:32:19.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:18 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:32:19.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:18 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:19.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:18 smithi100 ceph-mon[33401]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:32:19.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:18 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:19.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:18 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:19.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:18 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:32:19.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:18 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:32:19.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:18 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:19.657 DEBUG:teuthology.orchestra.run.smithi203:osd.8> sudo journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.8.service 2024-09-21T06:32:19.659 INFO:tasks.cephadm:Deploying osd.9 on smithi067 with /dev/vg_nvme/lv_1... 2024-09-21T06:32:19.659 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T06:32:19.928 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:32:20.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:19 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:32:20.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:19 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:20.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:19 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:20.166 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:19 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:32:20.166 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:19 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:20.166 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:19 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:20.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:19 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:32:20.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:19 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:20.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:19 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:21.139 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:20 smithi203 ceph-mon[33735]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:32:21.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:20 smithi067 ceph-mon[31232]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:32:21.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:20 smithi100 ceph-mon[33401]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:32:21.408 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:32:21 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:32:21.220+0000 7f4c1d49e640 -1 osd.8 0 log_to_monitors true 2024-09-21T06:32:22.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:21 smithi203 ceph-mon[33735]: from='osd.8 [v2:172.21.15.203:6816/3452445943,v1:172.21.15.203:6817/3452445943]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T06:32:22.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:21 smithi203 ceph-mon[33735]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T06:32:22.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:21 smithi067 ceph-mon[31232]: from='osd.8 [v2:172.21.15.203:6816/3452445943,v1:172.21.15.203:6817/3452445943]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T06:32:22.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:21 smithi067 ceph-mon[31232]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T06:32:22.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:21 smithi100 ceph-mon[33401]: from='osd.8 [v2:172.21.15.203:6816/3452445943,v1:172.21.15.203:6817/3452445943]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T06:32:22.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:21 smithi100 ceph-mon[33401]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T06:32:22.787 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:32:22.815 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi067:vg_nvme/lv_1 2024-09-21T06:32:23.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:22 smithi067 ceph-mon[31232]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:32:23.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:22 smithi067 ceph-mon[31232]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T06:32:23.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:22 smithi067 ceph-mon[31232]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T06:32:23.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:22 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:32:23.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:22 smithi067 ceph-mon[31232]: from='osd.8 [v2:172.21.15.203:6816/3452445943,v1:172.21.15.203:6817/3452445943]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T06:32:23.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:22 smithi067 ceph-mon[31232]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T06:32:23.043 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:22 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:23.043 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:22 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:23.043 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:22 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:32:23.043 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:22 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:32:23.043 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:22 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:32:23.043 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:22 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:23.043 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:22 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:32:23.043 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:22 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:32:23.043 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:22 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:23.074 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:32:23.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:22 smithi203 ceph-mon[33735]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:32:23.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:22 smithi203 ceph-mon[33735]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T06:32:23.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:22 smithi203 ceph-mon[33735]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T06:32:23.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:22 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:32:23.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:22 smithi203 ceph-mon[33735]: from='osd.8 [v2:172.21.15.203:6816/3452445943,v1:172.21.15.203:6817/3452445943]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T06:32:23.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:22 smithi203 ceph-mon[33735]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T06:32:23.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:22 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:23.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:22 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:23.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:22 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:32:23.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:22 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:32:23.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:22 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:32:23.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:22 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:23.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:22 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:32:23.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:22 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:32:23.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:22 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:23.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:22 smithi100 ceph-mon[33401]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:32:23.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:22 smithi100 ceph-mon[33401]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T06:32:23.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:22 smithi100 ceph-mon[33401]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T06:32:23.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:22 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:32:23.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:22 smithi100 ceph-mon[33401]: from='osd.8 [v2:172.21.15.203:6816/3452445943,v1:172.21.15.203:6817/3452445943]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T06:32:23.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:22 smithi100 ceph-mon[33401]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T06:32:23.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:22 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:23.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:22 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:23.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:22 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:32:23.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:22 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:32:23.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:22 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:32:23.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:22 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:23.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:22 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:32:23.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:22 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:32:23.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:22 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:23.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.582+0000 7ff389bd3640 1 -- 172.21.15.67:0/4171933807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff384102670 msgr2=0x7ff384102ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:23.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.582+0000 7ff389bd3640 1 --2- 172.21.15.67:0/4171933807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff384102670 0x7ff384102ae0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff374009ca0 tx=0x7ff37402a950 comp rx=0 tx=0).stop 2024-09-21T06:32:23.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.582+0000 7ff389bd3640 1 -- 172.21.15.67:0/4171933807 shutdown_connections 2024-09-21T06:32:23.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.582+0000 7ff389bd3640 1 --2- 172.21.15.67:0/4171933807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff384103700 0x7ff384103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:23.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.582+0000 7ff389bd3640 1 --2- 172.21.15.67:0/4171933807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3841040a0 0x7ff38410aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:23.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.582+0000 7ff389bd3640 1 --2- 172.21.15.67:0/4171933807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff384102670 0x7ff384102ae0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:23.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.582+0000 7ff389bd3640 1 -- 172.21.15.67:0/4171933807 >> 172.21.15.67:0/4171933807 conn(0x7ff3840fdfe0 msgr2=0x7ff384100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:23.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.582+0000 7ff389bd3640 1 -- 172.21.15.67:0/4171933807 shutdown_connections 2024-09-21T06:32:23.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.582+0000 7ff389bd3640 1 -- 172.21.15.67:0/4171933807 wait complete. 2024-09-21T06:32:23.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.583+0000 7ff389bd3640 1 Processor -- start 2024-09-21T06:32:23.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.583+0000 7ff389bd3640 1 -- start start 2024-09-21T06:32:23.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.583+0000 7ff389bd3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff384102670 0x7ff384198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:23.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.583+0000 7ff389bd3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff384103700 0x7ff384199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:23.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.583+0000 7ff389bd3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3841040a0 0x7ff38419a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:23.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.583+0000 7ff389bd3640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff38410e140 con 0x7ff384102670 2024-09-21T06:32:23.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.583+0000 7ff389bd3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff38410dfc0 con 0x7ff384103700 2024-09-21T06:32:23.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.583+0000 7ff389bd3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff38410e2c0 con 0x7ff3841040a0 2024-09-21T06:32:23.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.583+0000 7ff3837fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff384102670 0x7ff384198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:23.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.583+0000 7ff3837fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff384102670 0x7ff384198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56926/0 (socket says 172.21.15.67:56926) 2024-09-21T06:32:23.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.583+0000 7ff3837fe640 1 -- 172.21.15.67:0/909507114 learned_addr learned my addr 172.21.15.67:0/909507114 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:32:23.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.584+0000 7ff383fff640 1 --2- 172.21.15.67:0/909507114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3841040a0 0x7ff38419a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:23.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.584+0000 7ff382ffd640 1 --2- 172.21.15.67:0/909507114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff384103700 0x7ff384199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:23.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.584+0000 7ff3837fe640 1 -- 172.21.15.67:0/909507114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3841040a0 msgr2=0x7ff38419a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:23.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.584+0000 7ff3837fe640 1 --2- 172.21.15.67:0/909507114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3841040a0 0x7ff38419a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:23.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.584+0000 7ff3837fe640 1 -- 172.21.15.67:0/909507114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff384103700 msgr2=0x7ff384199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:23.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.584+0000 7ff3837fe640 1 --2- 172.21.15.67:0/909507114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff384103700 0x7ff384199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:23.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.584+0000 7ff3837fe640 1 -- 172.21.15.67:0/909507114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38419a790 con 0x7ff384102670 2024-09-21T06:32:23.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.584+0000 7ff382ffd640 1 --2- 172.21.15.67:0/909507114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff384103700 0x7ff384199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:32:23.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.584+0000 7ff3837fe640 1 --2- 172.21.15.67:0/909507114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff384102670 0x7ff384198df0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff374003f10 tx=0x7ff374003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:23.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.584+0000 7ff380ff9640 1 -- 172.21.15.67:0/909507114 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374034040 con 0x7ff384102670 2024-09-21T06:32:23.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.584+0000 7ff389bd3640 1 -- 172.21.15.67:0/909507114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff38419aa20 con 0x7ff384102670 2024-09-21T06:32:23.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.584+0000 7ff380ff9640 1 -- 172.21.15.67:0/909507114 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff374030910 con 0x7ff384102670 2024-09-21T06:32:23.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.584+0000 7ff380ff9640 1 -- 172.21.15.67:0/909507114 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374004840 con 0x7ff384102670 2024-09-21T06:32:23.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.584+0000 7ff389bd3640 1 -- 172.21.15.67:0/909507114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3841a1040 con 0x7ff384102670 2024-09-21T06:32:23.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.586+0000 7ff389bd3640 1 -- 172.21.15.67:0/909507114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff384109850 con 0x7ff384102670 2024-09-21T06:32:23.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.587+0000 7ff380ff9640 1 -- 172.21.15.67:0/909507114 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7ff374037020 con 0x7ff384102670 2024-09-21T06:32:23.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.588+0000 7ff380ff9640 1 --2- 172.21.15.67:0/909507114 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7ff36c0b6510 0x7ff36c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:23.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.588+0000 7ff382ffd640 1 --2- 172.21.15.67:0/909507114 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7ff36c0b6510 0x7ff36c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:23.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.588+0000 7ff380ff9640 1 -- 172.21.15.67:0/909507114 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5549+0+0 (secure 0 0 0) 0x7ff3741077c0 con 0x7ff384102670 2024-09-21T06:32:23.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.589+0000 7ff382ffd640 1 --2- 172.21.15.67:0/909507114 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7ff36c0b6510 0x7ff36c0b8a20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff3780058c0 tx=0x7ff378005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:23.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.591+0000 7ff380ff9640 1 -- 172.21.15.67:0/909507114 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff3740098b0 con 0x7ff384102670 2024-09-21T06:32:23.732 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:32:23 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:32:23.380+0000 7f4c19420640 -1 osd.8 0 waiting for initial osdmap 2024-09-21T06:32:23.732 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:32:23 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:32:23.389+0000 7f4c11216640 -1 osd.8 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:32:23.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:23.792+0000 7ff389bd3640 1 -- 172.21.15.67:0/909507114 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff384063800 con 0x7ff36c0b6510 2024-09-21T06:32:24.187 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:23 smithi067 ceph-mon[31232]: Detected new or changed devices on smithi203 2024-09-21T06:32:24.187 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:23 smithi067 ceph-mon[31232]: Adjusting osd_memory_target on smithi203 to 5696M 2024-09-21T06:32:24.187 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:23 smithi067 ceph-mon[31232]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-21T06:32:24.187 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:23 smithi067 ceph-mon[31232]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T06:32:24.187 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:23 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:32:24.187 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:23 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:32:24.187 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:23 smithi067 ceph-mon[31232]: from='osd.8 ' entity='osd.8' 2024-09-21T06:32:24.187 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:23 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:32:24.187 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:23 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:32:24.187 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:23 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:32:24.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:23 smithi203 ceph-mon[33735]: Detected new or changed devices on smithi203 2024-09-21T06:32:24.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:23 smithi203 ceph-mon[33735]: Adjusting osd_memory_target on smithi203 to 5696M 2024-09-21T06:32:24.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:23 smithi203 ceph-mon[33735]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-21T06:32:24.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:23 smithi203 ceph-mon[33735]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T06:32:24.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:23 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:32:24.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:23 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:32:24.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:23 smithi203 ceph-mon[33735]: from='osd.8 ' entity='osd.8' 2024-09-21T06:32:24.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:23 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:32:24.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:23 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:32:24.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:23 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:32:24.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:23 smithi100 ceph-mon[33401]: Detected new or changed devices on smithi203 2024-09-21T06:32:24.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:23 smithi100 ceph-mon[33401]: Adjusting osd_memory_target on smithi203 to 5696M 2024-09-21T06:32:24.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:23 smithi100 ceph-mon[33401]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-21T06:32:24.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:23 smithi100 ceph-mon[33401]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T06:32:24.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:23 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:32:24.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:23 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:32:24.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:23 smithi100 ceph-mon[33401]: from='osd.8 ' entity='osd.8' 2024-09-21T06:32:24.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:23 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:32:24.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:23 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:32:24.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:23 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:32:25.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:24 smithi203 ceph-mon[33735]: purged_snaps scrub starts 2024-09-21T06:32:25.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:24 smithi203 ceph-mon[33735]: purged_snaps scrub ok 2024-09-21T06:32:25.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:24 smithi203 ceph-mon[33735]: from='client.14625 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:32:25.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:24 smithi203 ceph-mon[33735]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:32:25.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:24 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:32:25.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:24 smithi203 ceph-mon[33735]: osd.8 [v2:172.21.15.203:6816/3452445943,v1:172.21.15.203:6817/3452445943] boot 2024-09-21T06:32:25.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:24 smithi203 ceph-mon[33735]: osdmap e52: 9 total, 9 up, 9 in 2024-09-21T06:32:25.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:24 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:32:25.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:24 smithi067 ceph-mon[31232]: purged_snaps scrub starts 2024-09-21T06:32:25.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:24 smithi067 ceph-mon[31232]: purged_snaps scrub ok 2024-09-21T06:32:25.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:24 smithi067 ceph-mon[31232]: from='client.14625 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:32:25.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:24 smithi067 ceph-mon[31232]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:32:25.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:24 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:32:25.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:24 smithi067 ceph-mon[31232]: osd.8 [v2:172.21.15.203:6816/3452445943,v1:172.21.15.203:6817/3452445943] boot 2024-09-21T06:32:25.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:24 smithi067 ceph-mon[31232]: osdmap e52: 9 total, 9 up, 9 in 2024-09-21T06:32:25.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:24 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:32:25.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:24 smithi100 ceph-mon[33401]: purged_snaps scrub starts 2024-09-21T06:32:25.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:24 smithi100 ceph-mon[33401]: purged_snaps scrub ok 2024-09-21T06:32:25.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:24 smithi100 ceph-mon[33401]: from='client.14625 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:32:25.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:24 smithi100 ceph-mon[33401]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:32:25.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:24 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:32:25.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:24 smithi100 ceph-mon[33401]: osd.8 [v2:172.21.15.203:6816/3452445943,v1:172.21.15.203:6817/3452445943] boot 2024-09-21T06:32:25.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:24 smithi100 ceph-mon[33401]: osdmap e52: 9 total, 9 up, 9 in 2024-09-21T06:32:25.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:24 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:32:26.172 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:25 smithi067 ceph-mon[31232]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T06:32:26.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:25 smithi100 ceph-mon[33401]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T06:32:26.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:25 smithi203 ceph-mon[33735]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T06:32:26.991 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:26 smithi067 ceph-mon[31232]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:32:26.991 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:26 smithi067 ceph-mon[31232]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T06:32:26.991 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:26 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/803738248' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "95704ed6-6286-4094-9c6d-23919c7005ca"} : dispatch 2024-09-21T06:32:26.992 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:26 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/803738248' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95704ed6-6286-4094-9c6d-23919c7005ca"}]': finished 2024-09-21T06:32:26.992 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:26 smithi067 ceph-mon[31232]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T06:32:26.992 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:26 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:32:27.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:26 smithi100 ceph-mon[33401]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:32:27.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:26 smithi100 ceph-mon[33401]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T06:32:27.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:26 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/803738248' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "95704ed6-6286-4094-9c6d-23919c7005ca"} : dispatch 2024-09-21T06:32:27.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:26 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/803738248' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95704ed6-6286-4094-9c6d-23919c7005ca"}]': finished 2024-09-21T06:32:27.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:26 smithi100 ceph-mon[33401]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T06:32:27.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:26 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:32:27.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:26 smithi203 ceph-mon[33735]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:32:27.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:26 smithi203 ceph-mon[33735]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T06:32:27.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:26 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/803738248' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "95704ed6-6286-4094-9c6d-23919c7005ca"} : dispatch 2024-09-21T06:32:27.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:26 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/803738248' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95704ed6-6286-4094-9c6d-23919c7005ca"}]': finished 2024-09-21T06:32:27.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:26 smithi203 ceph-mon[33735]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T06:32:27.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:26 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:32:28.291 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:27 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2483241051' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:32:28.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:27 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2483241051' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:32:28.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:27 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2483241051' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:32:29.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:28 smithi067 ceph-mon[31232]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-21T06:32:29.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:28 smithi100 ceph-mon[33401]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-21T06:32:29.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:28 smithi203 ceph-mon[33735]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-21T06:32:31.216 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:31 smithi067 ceph-mon[31232]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-21T06:32:31.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:31 smithi100 ceph-mon[33401]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-21T06:32:31.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:31 smithi203 ceph-mon[33735]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-21T06:32:33.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:33 smithi067 ceph-mon[31232]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-21T06:32:33.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:33 smithi100 ceph-mon[33401]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-21T06:32:33.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:33 smithi203 ceph-mon[33735]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-21T06:32:35.276 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:35 smithi067 ceph-mon[31232]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:32:35.277 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T06:32:35.277 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:32:35.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:35 smithi100 ceph-mon[33401]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:32:35.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:35 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T06:32:35.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:35 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:32:35.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:35 smithi203 ceph-mon[33735]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:32:35.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:35 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T06:32:35.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:35 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:32:36.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:36 smithi067 ceph-mon[31232]: Deploying daemon osd.9 on smithi067 2024-09-21T06:32:36.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:36 smithi100 ceph-mon[33401]: Deploying daemon osd.9 on smithi067 2024-09-21T06:32:36.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:36 smithi203 ceph-mon[33735]: Deploying daemon osd.9 on smithi067 2024-09-21T06:32:37.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:37 smithi067 ceph-mon[31232]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-21T06:32:37.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:37 smithi100 ceph-mon[33401]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-21T06:32:37.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:37 smithi203 ceph-mon[33735]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-21T06:32:39.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:39 smithi067 ceph-mon[31232]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T06:32:39.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:39 smithi100 ceph-mon[33401]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T06:32:39.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:39 smithi203 ceph-mon[33735]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T06:32:41.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:41 smithi067 ceph-mon[31232]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:32:41.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:41 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:32:41.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:41 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:41.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:41 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:41.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:41 smithi100 ceph-mon[33401]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:32:41.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:41 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:32:41.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:41 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:41.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:41 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:41.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:41 smithi203 ceph-mon[33735]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:32:41.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:41 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:32:41.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:41 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:41.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:41 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:43.245 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:43 smithi067 ceph-mon[31232]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:32:43.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:43 smithi100 ceph-mon[33401]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:32:43.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:43 smithi203 ceph-mon[33735]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:32:44.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:44.460+0000 7ff380ff9640 1 -- 172.21.15.67:0/909507114 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff384063800 con 0x7ff36c0b6510 2024-09-21T06:32:44.462 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 9 on host 'smithi067' 2024-09-21T06:32:44.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:44.462+0000 7ff389bd3640 1 -- 172.21.15.67:0/909507114 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7ff36c0b6510 msgr2=0x7ff36c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:44.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:44.462+0000 7ff389bd3640 1 --2- 172.21.15.67:0/909507114 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7ff36c0b6510 0x7ff36c0b8a20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff3780058c0 tx=0x7ff378005850 comp rx=0 tx=0).stop 2024-09-21T06:32:44.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:44.462+0000 7ff389bd3640 1 -- 172.21.15.67:0/909507114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff384102670 msgr2=0x7ff384198df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:44.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:44.462+0000 7ff389bd3640 1 --2- 172.21.15.67:0/909507114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff384102670 0x7ff384198df0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff374003f10 tx=0x7ff374003f40 comp rx=0 tx=0).stop 2024-09-21T06:32:44.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:44.462+0000 7ff389bd3640 1 -- 172.21.15.67:0/909507114 shutdown_connections 2024-09-21T06:32:44.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:44.462+0000 7ff389bd3640 1 --2- 172.21.15.67:0/909507114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff384103700 0x7ff384199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:44.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:44.462+0000 7ff389bd3640 1 --2- 172.21.15.67:0/909507114 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7ff36c0b6510 0x7ff36c0b8a20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:44.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:44.462+0000 7ff389bd3640 1 --2- 172.21.15.67:0/909507114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3841040a0 0x7ff38419a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:44.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:44.463+0000 7ff389bd3640 1 --2- 172.21.15.67:0/909507114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff384102670 0x7ff384198df0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:44.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:44.463+0000 7ff389bd3640 1 -- 172.21.15.67:0/909507114 >> 172.21.15.67:0/909507114 conn(0x7ff3840fdfe0 msgr2=0x7ff384106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:44.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:44.463+0000 7ff389bd3640 1 -- 172.21.15.67:0/909507114 shutdown_connections 2024-09-21T06:32:44.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:32:44.463+0000 7ff389bd3640 1 -- 172.21.15.67:0/909507114 wait complete. 2024-09-21T06:32:44.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:44 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:44.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:44 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:44.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:44 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:32:44.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:44 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:32:44.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:44 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:44.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:44 smithi067 ceph-mon[31232]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:32:44.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:44 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:32:44.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:44 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:44.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:44 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:44.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:44 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:44.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:44 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:44.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:44 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:32:44.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:44 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:32:44.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:44 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:44.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:44 smithi203 ceph-mon[33735]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:32:44.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:44 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:32:44.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:44 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:44.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:44 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:45.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:44 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:45.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:44 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:45.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:44 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:32:45.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:44 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:32:45.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:44 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:45.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:44 smithi100 ceph-mon[33401]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:32:45.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:44 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:32:45.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:44 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:45.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:44 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:45.180 DEBUG:teuthology.orchestra.run.smithi067:osd.9> sudo journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.9.service 2024-09-21T06:32:45.225 INFO:tasks.cephadm:Deploying osd.10 on smithi100 with /dev/vg_nvme/lv_1... 2024-09-21T06:32:45.226 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T06:32:45.481 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:32:46.906 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:46 smithi100 ceph-mon[33401]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:32:47.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:46 smithi203 ceph-mon[33735]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:32:47.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:46 smithi067 ceph-mon[31232]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:32:47.292 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:32:47 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:32:47.006+0000 7f3cb09d9640 -1 osd.9 0 log_to_monitors true 2024-09-21T06:32:48.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:47 smithi203 ceph-mon[33735]: from='osd.9 [v2:172.21.15.67:6826/2998307533,v1:172.21.15.67:6827/2998307533]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T06:32:48.258 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:47 smithi100 ceph-mon[33401]: from='osd.9 [v2:172.21.15.67:6826/2998307533,v1:172.21.15.67:6827/2998307533]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T06:32:48.261 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-21T06:32:48.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:47 smithi067 ceph-mon[31232]: from='osd.9 [v2:172.21.15.67:6826/2998307533,v1:172.21.15.67:6827/2998307533]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T06:32:48.294 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi100:vg_nvme/lv_1 2024-09-21T06:32:48.542 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:32:48.962 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:48 smithi100 ceph-mon[33401]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:32:48.962 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:48 smithi100 ceph-mon[33401]: from='osd.9 [v2:172.21.15.67:6826/2998307533,v1:172.21.15.67:6827/2998307533]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T06:32:48.962 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:48 smithi100 ceph-mon[33401]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T06:32:48.962 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:48 smithi100 ceph-mon[33401]: from='osd.9 [v2:172.21.15.67:6826/2998307533,v1:172.21.15.67:6827/2998307533]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]} : dispatch 2024-09-21T06:32:48.962 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:48 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:32:48.962 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:48 smithi100 ceph-mon[33401]: Detected new or changed devices on smithi067 2024-09-21T06:32:48.962 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:48 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:48.963 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:48 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:48.963 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:48 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:32:48.963 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:48 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:32:48.963 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:48 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:32:48.963 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:48 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T06:32:48.963 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:48 smithi100 ceph-mon[33401]: Adjusting osd_memory_target on smithi067 to 4272M 2024-09-21T06:32:48.963 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:48 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:48.963 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:48 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:32:48.963 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:48 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:32:48.963 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:48 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:49.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.052+0000 7f4287694640 1 -- 172.21.15.100:0/3873547126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4280101b40 msgr2=0x7f4280101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:49.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.052+0000 7f4287694640 1 --2- 172.21.15.100:0/3873547126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4280101b40 0x7f4280101f50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f427c009900 tx=0x7f427c02a410 comp rx=0 tx=0).stop 2024-09-21T06:32:49.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.053+0000 7f4287694640 1 -- 172.21.15.100:0/3873547126 shutdown_connections 2024-09-21T06:32:49.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.053+0000 7f4287694640 1 --2- 172.21.15.100:0/3873547126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4280109540 0x7f428010b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:49.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.053+0000 7f4287694640 1 --2- 172.21.15.100:0/3873547126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4280106be0 0x7f4280109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:49.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.053+0000 7f4287694640 1 --2- 172.21.15.100:0/3873547126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4280101b40 0x7f4280101f50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:49.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.053+0000 7f4287694640 1 -- 172.21.15.100:0/3873547126 >> 172.21.15.100:0/3873547126 conn(0x7f42800fd770 msgr2=0x7f42800ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:32:49.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.053+0000 7f4287694640 1 -- 172.21.15.100:0/3873547126 shutdown_connections 2024-09-21T06:32:49.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.053+0000 7f4287694640 1 -- 172.21.15.100:0/3873547126 wait complete. 2024-09-21T06:32:49.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.053+0000 7f4287694640 1 Processor -- start 2024-09-21T06:32:49.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.054+0000 7f4287694640 1 -- start start 2024-09-21T06:32:49.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.054+0000 7f4287694640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4280101b40 0x7f4280199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:49.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.054+0000 7f4287694640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4280106be0 0x7f428019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:49.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.054+0000 7f4287694640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4280109540 0x7f428019b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:49.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.054+0000 7f4287694640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f428010f270 con 0x7f4280101b40 2024-09-21T06:32:49.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.054+0000 7f4287694640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f428010f0f0 con 0x7f4280106be0 2024-09-21T06:32:49.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.054+0000 7f4287694640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f428010f3f0 con 0x7f4280109540 2024-09-21T06:32:49.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.054+0000 7f4284c2f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4280106be0 0x7f428019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:49.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.054+0000 7f4284c2f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4280106be0 0x7f428019a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38624/0 (socket says 172.21.15.100:38624) 2024-09-21T06:32:49.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.054+0000 7f4284c2f640 1 -- 172.21.15.100:0/1910482448 learned_addr learned my addr 172.21.15.100:0/1910482448 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-21T06:32:49.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.054+0000 7f4285c31640 1 --2- 172.21.15.100:0/1910482448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4280109540 0x7f428019b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:49.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.054+0000 7f4285430640 1 --2- 172.21.15.100:0/1910482448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4280101b40 0x7f4280199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:49.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.054+0000 7f4284c2f640 1 -- 172.21.15.100:0/1910482448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4280109540 msgr2=0x7f428019b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:49.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.054+0000 7f4284c2f640 1 --2- 172.21.15.100:0/1910482448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4280109540 0x7f428019b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:49.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.054+0000 7f4284c2f640 1 -- 172.21.15.100:0/1910482448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4280101b40 msgr2=0x7f4280199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:32:49.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.054+0000 7f4284c2f640 1 --2- 172.21.15.100:0/1910482448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4280101b40 0x7f4280199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:32:49.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.054+0000 7f4284c2f640 1 -- 172.21.15.100:0/1910482448 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f428019b8d0 con 0x7f4280106be0 2024-09-21T06:32:49.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.054+0000 7f4285430640 1 --2- 172.21.15.100:0/1910482448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4280101b40 0x7f4280199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:32:49.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.055+0000 7f4284c2f640 1 --2- 172.21.15.100:0/1910482448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4280106be0 0x7f428019a510 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f427000c890 tx=0x7f427000cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:49.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.055+0000 7f42767fc640 1 -- 172.21.15.100:0/1910482448 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4270004220 con 0x7f4280106be0 2024-09-21T06:32:49.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.055+0000 7f4287694640 1 -- 172.21.15.100:0/1910482448 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f428019bbc0 con 0x7f4280106be0 2024-09-21T06:32:49.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.055+0000 7f42767fc640 1 -- 172.21.15.100:0/1910482448 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4270007e40 con 0x7f4280106be0 2024-09-21T06:32:49.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.055+0000 7f42767fc640 1 -- 172.21.15.100:0/1910482448 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4270004a00 con 0x7f4280106be0 2024-09-21T06:32:49.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.055+0000 7f4287694640 1 -- 172.21.15.100:0/1910482448 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42801a22c0 con 0x7f4280106be0 2024-09-21T06:32:49.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.056+0000 7f4287694640 1 -- 172.21.15.100:0/1910482448 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42800ff290 con 0x7f4280106be0 2024-09-21T06:32:49.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.057+0000 7f42767fc640 1 -- 172.21.15.100:0/1910482448 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f42700079b0 con 0x7f4280106be0 2024-09-21T06:32:49.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.058+0000 7f42767fc640 1 --2- 172.21.15.100:0/1910482448 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f426c0b6510 0x7f426c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:32:49.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.058+0000 7f42767fc640 1 -- 172.21.15.100:0/1910482448 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5981+0+0 (secure 0 0 0) 0x7f42700e6720 con 0x7f4280106be0 2024-09-21T06:32:49.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.058+0000 7f4285430640 1 --2- 172.21.15.100:0/1910482448 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f426c0b6510 0x7f426c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:32:49.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.059+0000 7f4285430640 1 --2- 172.21.15.100:0/1910482448 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f426c0b6510 0x7f426c0b8a20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f427c000c00 tx=0x7f427c031420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:32:49.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.063+0000 7f42767fc640 1 -- 172.21.15.100:0/1910482448 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4270014030 con 0x7f4280106be0 2024-09-21T06:32:49.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:48 smithi203 ceph-mon[33735]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:32:49.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:48 smithi203 ceph-mon[33735]: from='osd.9 [v2:172.21.15.67:6826/2998307533,v1:172.21.15.67:6827/2998307533]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T06:32:49.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:48 smithi203 ceph-mon[33735]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T06:32:49.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:48 smithi203 ceph-mon[33735]: from='osd.9 [v2:172.21.15.67:6826/2998307533,v1:172.21.15.67:6827/2998307533]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]} : dispatch 2024-09-21T06:32:49.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:48 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:32:49.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:48 smithi203 ceph-mon[33735]: Detected new or changed devices on smithi067 2024-09-21T06:32:49.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:48 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:49.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:48 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:49.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:48 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:32:49.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:48 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:32:49.234 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:48 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:32:49.234 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:48 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T06:32:49.234 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:48 smithi203 ceph-mon[33735]: Adjusting osd_memory_target on smithi067 to 4272M 2024-09-21T06:32:49.234 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:48 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:49.234 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:48 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:32:49.234 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:48 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:32:49.234 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:48 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:49.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:32:49.250+0000 7f4287694640 1 -- 172.21.15.100:0/1910482448 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4280063800 con 0x7f426c0b6510 2024-09-21T06:32:49.292 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:32:48 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:32:48.890+0000 7f3cac95b640 -1 osd.9 0 waiting for initial osdmap 2024-09-21T06:32:49.292 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:32:48 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:32:48.903+0000 7f3ca8759640 -1 osd.9 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:32:49.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:48 smithi067 ceph-mon[31232]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:32:49.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:48 smithi067 ceph-mon[31232]: from='osd.9 [v2:172.21.15.67:6826/2998307533,v1:172.21.15.67:6827/2998307533]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T06:32:49.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:48 smithi067 ceph-mon[31232]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T06:32:49.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:48 smithi067 ceph-mon[31232]: from='osd.9 [v2:172.21.15.67:6826/2998307533,v1:172.21.15.67:6827/2998307533]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]} : dispatch 2024-09-21T06:32:49.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:48 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:32:49.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:48 smithi067 ceph-mon[31232]: Detected new or changed devices on smithi067 2024-09-21T06:32:49.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:48 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:49.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:48 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:49.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:48 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:32:49.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:48 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:32:49.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:48 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:32:49.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:48 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T06:32:49.294 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:48 smithi067 ceph-mon[31232]: Adjusting osd_memory_target on smithi067 to 4272M 2024-09-21T06:32:49.294 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:48 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:49.294 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:48 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:32:49.294 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:48 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:32:49.294 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:48 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:32:50.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:49 smithi203 ceph-mon[33735]: from='osd.9 [v2:172.21.15.67:6826/2998307533,v1:172.21.15.67:6827/2998307533]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-21T06:32:50.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:49 smithi203 ceph-mon[33735]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T06:32:50.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:49 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:32:50.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:49 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:32:50.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:49 smithi203 ceph-mon[33735]: from='client.24512 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:32:50.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:49 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:32:50.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:49 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:32:50.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:49 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:32:50.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:49 smithi067 ceph-mon[31232]: from='osd.9 [v2:172.21.15.67:6826/2998307533,v1:172.21.15.67:6827/2998307533]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-21T06:32:50.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:49 smithi067 ceph-mon[31232]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T06:32:50.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:49 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:32:50.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:49 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:32:50.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:49 smithi067 ceph-mon[31232]: from='client.24512 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:32:50.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:49 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:32:50.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:49 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:32:50.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:49 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:32:50.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:49 smithi100 ceph-mon[33401]: from='osd.9 [v2:172.21.15.67:6826/2998307533,v1:172.21.15.67:6827/2998307533]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-21T06:32:50.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:49 smithi100 ceph-mon[33401]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T06:32:50.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:49 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:32:50.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:49 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:32:50.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:49 smithi100 ceph-mon[33401]: from='client.24512 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:32:50.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:49 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:32:50.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:49 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:32:50.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:49 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:32:50.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:50 smithi100 ceph-mon[33401]: purged_snaps scrub starts 2024-09-21T06:32:50.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:50 smithi100 ceph-mon[33401]: purged_snaps scrub ok 2024-09-21T06:32:50.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:50 smithi100 ceph-mon[33401]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:32:50.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:50 smithi100 ceph-mon[33401]: osd.9 [v2:172.21.15.67:6826/2998307533,v1:172.21.15.67:6827/2998307533] boot 2024-09-21T06:32:50.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:50 smithi100 ceph-mon[33401]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T06:32:50.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:50 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:32:51.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:50 smithi203 ceph-mon[33735]: purged_snaps scrub starts 2024-09-21T06:32:51.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:50 smithi203 ceph-mon[33735]: purged_snaps scrub ok 2024-09-21T06:32:51.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:50 smithi203 ceph-mon[33735]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:32:51.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:50 smithi203 ceph-mon[33735]: osd.9 [v2:172.21.15.67:6826/2998307533,v1:172.21.15.67:6827/2998307533] boot 2024-09-21T06:32:51.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:50 smithi203 ceph-mon[33735]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T06:32:51.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:50 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:32:51.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:50 smithi067 ceph-mon[31232]: purged_snaps scrub starts 2024-09-21T06:32:51.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:50 smithi067 ceph-mon[31232]: purged_snaps scrub ok 2024-09-21T06:32:51.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:50 smithi067 ceph-mon[31232]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:32:51.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:50 smithi067 ceph-mon[31232]: osd.9 [v2:172.21.15.67:6826/2998307533,v1:172.21.15.67:6827/2998307533] boot 2024-09-21T06:32:51.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:50 smithi067 ceph-mon[31232]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T06:32:51.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:50 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:32:52.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:51 smithi203 ceph-mon[33735]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T06:32:52.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:51 smithi203 ceph-mon[33735]: from='client.? 172.21.15.100:0/2138376898' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bd2a7d33-b935-46c8-95c3-d76f28242ffe"} : dispatch 2024-09-21T06:32:52.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:51 smithi203 ceph-mon[33735]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bd2a7d33-b935-46c8-95c3-d76f28242ffe"} : dispatch 2024-09-21T06:32:52.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:51 smithi203 ceph-mon[33735]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd2a7d33-b935-46c8-95c3-d76f28242ffe"}]': finished 2024-09-21T06:32:52.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:51 smithi203 ceph-mon[33735]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T06:32:52.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:51 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:32:52.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:51 smithi067 ceph-mon[31232]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T06:32:52.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:51 smithi067 ceph-mon[31232]: from='client.? 172.21.15.100:0/2138376898' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bd2a7d33-b935-46c8-95c3-d76f28242ffe"} : dispatch 2024-09-21T06:32:52.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:51 smithi067 ceph-mon[31232]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bd2a7d33-b935-46c8-95c3-d76f28242ffe"} : dispatch 2024-09-21T06:32:52.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:51 smithi067 ceph-mon[31232]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd2a7d33-b935-46c8-95c3-d76f28242ffe"}]': finished 2024-09-21T06:32:52.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:51 smithi067 ceph-mon[31232]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T06:32:52.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:51 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:32:52.300 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:51 smithi100 ceph-mon[33401]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T06:32:52.300 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:51 smithi100 ceph-mon[33401]: from='client.? 172.21.15.100:0/2138376898' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bd2a7d33-b935-46c8-95c3-d76f28242ffe"} : dispatch 2024-09-21T06:32:52.300 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:51 smithi100 ceph-mon[33401]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bd2a7d33-b935-46c8-95c3-d76f28242ffe"} : dispatch 2024-09-21T06:32:52.301 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:51 smithi100 ceph-mon[33401]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd2a7d33-b935-46c8-95c3-d76f28242ffe"}]': finished 2024-09-21T06:32:52.301 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:51 smithi100 ceph-mon[33401]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T06:32:52.301 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:51 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:32:53.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:52 smithi203 ceph-mon[33735]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:32:53.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:52 smithi203 ceph-mon[33735]: from='client.? 172.21.15.100:0/2688076452' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:32:53.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:52 smithi067 ceph-mon[31232]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:32:53.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:52 smithi067 ceph-mon[31232]: from='client.? 172.21.15.100:0/2688076452' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:32:53.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:52 smithi100 ceph-mon[33401]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:32:53.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:52 smithi100 ceph-mon[33401]: from='client.? 172.21.15.100:0/2688076452' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:32:55.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:54 smithi203 ceph-mon[33735]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:32:55.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:54 smithi067 ceph-mon[31232]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:32:55.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:54 smithi100 ceph-mon[33401]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:32:57.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:56 smithi203 ceph-mon[33735]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:32:57.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:56 smithi067 ceph-mon[31232]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:32:57.298 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:56 smithi100 ceph-mon[33401]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:32:59.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:32:58 smithi100 ceph-mon[33401]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:32:59.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:32:58 smithi203 ceph-mon[33735]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:32:59.291 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:32:58 smithi067 ceph-mon[31232]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:33:01.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:00 smithi100 ceph-mon[33401]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:33:01.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:00 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T06:33:01.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:00 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:01.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:00 smithi100 ceph-mon[33401]: Deploying daemon osd.10 on smithi100 2024-09-21T06:33:01.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:00 smithi203 ceph-mon[33735]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:33:01.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:00 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T06:33:01.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:00 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:01.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:00 smithi203 ceph-mon[33735]: Deploying daemon osd.10 on smithi100 2024-09-21T06:33:01.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:00 smithi067 ceph-mon[31232]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:33:01.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:00 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T06:33:01.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:00 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:01.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:00 smithi067 ceph-mon[31232]: Deploying daemon osd.10 on smithi100 2024-09-21T06:33:03.116 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:02 smithi100 ceph-mon[33401]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:33:03.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:02 smithi203 ceph-mon[33735]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:33:03.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:02 smithi067 ceph-mon[31232]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:33:05.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:04 smithi100 ceph-mon[33401]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:33:05.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:04 smithi203 ceph-mon[33735]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:33:05.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:04 smithi067 ceph-mon[31232]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:33:07.217 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:06 smithi100 ceph-mon[33401]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:33:07.217 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:06 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:33:07.217 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:06 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:07.217 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:06 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:07.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:06 smithi203 ceph-mon[33735]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:33:07.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:06 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:33:07.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:06 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:07.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:06 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:07.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:06 smithi067 ceph-mon[31232]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:33:07.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:06 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:33:07.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:06 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:07.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:06 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:09.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:08 smithi203 ceph-mon[33735]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:33:09.291 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:08 smithi067 ceph-mon[31232]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:33:09.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:08 smithi100 ceph-mon[33401]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:33:10.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:33:10.101+0000 7f42767fc640 1 -- 172.21.15.100:0/1910482448 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f4280063800 con 0x7f426c0b6510 2024-09-21T06:33:10.103 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 10 on host 'smithi100' 2024-09-21T06:33:10.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:33:10.104+0000 7f4287694640 1 -- 172.21.15.100:0/1910482448 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f426c0b6510 msgr2=0x7f426c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:10.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:33:10.104+0000 7f4287694640 1 --2- 172.21.15.100:0/1910482448 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f426c0b6510 0x7f426c0b8a20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f427c000c00 tx=0x7f427c031420 comp rx=0 tx=0).stop 2024-09-21T06:33:10.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:33:10.104+0000 7f4287694640 1 -- 172.21.15.100:0/1910482448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4280106be0 msgr2=0x7f428019a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:10.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:33:10.104+0000 7f4287694640 1 --2- 172.21.15.100:0/1910482448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4280106be0 0x7f428019a510 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f427000c890 tx=0x7f427000cd50 comp rx=0 tx=0).stop 2024-09-21T06:33:10.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:33:10.104+0000 7f4287694640 1 -- 172.21.15.100:0/1910482448 shutdown_connections 2024-09-21T06:33:10.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:33:10.104+0000 7f4287694640 1 --2- 172.21.15.100:0/1910482448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4280106be0 0x7f428019a510 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:10.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:33:10.104+0000 7f4287694640 1 --2- 172.21.15.100:0/1910482448 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f426c0b6510 0x7f426c0b8a20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:10.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:33:10.104+0000 7f4287694640 1 --2- 172.21.15.100:0/1910482448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4280109540 0x7f428019b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:10.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:33:10.104+0000 7f4287694640 1 --2- 172.21.15.100:0/1910482448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4280101b40 0x7f4280199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:10.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:33:10.104+0000 7f4287694640 1 -- 172.21.15.100:0/1910482448 >> 172.21.15.100:0/1910482448 conn(0x7f42800fd770 msgr2=0x7f4280107ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:10.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:33:10.104+0000 7f4287694640 1 -- 172.21.15.100:0/1910482448 shutdown_connections 2024-09-21T06:33:10.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-21T06:33:10.104+0000 7f4287694640 1 -- 172.21.15.100:0/1910482448 wait complete. 2024-09-21T06:33:10.463 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:10 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:10.463 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:10 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:10.463 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:10 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:10.463 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:10 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:33:10.463 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:10 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:10.463 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:10 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:33:10.463 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:10 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:10.464 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:10 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:10.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:10 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:10.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:10 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:10.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:10 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:10.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:10 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:33:10.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:10 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:10.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:10 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:33:10.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:10 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:10.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:10 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:10.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:10 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:10.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:10 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:10.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:10 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:10.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:10 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:33:10.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:10 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:10.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:10 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:33:10.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:10 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:10.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:10 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:10.792 DEBUG:teuthology.orchestra.run.smithi100:osd.10> sudo journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.10.service 2024-09-21T06:33:10.798 INFO:tasks.cephadm:Deploying osd.11 on smithi203 with /dev/vg_nvme/lv_1... 2024-09-21T06:33:10.799 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T06:33:11.058 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:33:11.448 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:11 smithi203 ceph-mon[33735]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:33:11.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:11 smithi100 ceph-mon[33401]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:33:11.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:11 smithi067 ceph-mon[31232]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:33:13.304 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:33:12 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:33:12.850+0000 7f32e2ac0640 -1 osd.10 0 log_to_monitors true 2024-09-21T06:33:13.432 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:13 smithi203 ceph-mon[33735]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:33:13.432 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:13 smithi203 ceph-mon[33735]: from='osd.10 [v2:172.21.15.100:6824/2819057142,v1:172.21.15.100:6825/2819057142]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T06:33:13.432 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:13 smithi203 ceph-mon[33735]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T06:33:13.615 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:13 smithi100 ceph-mon[33401]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:33:13.615 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:13 smithi100 ceph-mon[33401]: from='osd.10 [v2:172.21.15.100:6824/2819057142,v1:172.21.15.100:6825/2819057142]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T06:33:13.616 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:13 smithi100 ceph-mon[33401]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T06:33:13.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:13 smithi067 ceph-mon[31232]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:33:13.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:13 smithi067 ceph-mon[31232]: from='osd.10 [v2:172.21.15.100:6824/2819057142,v1:172.21.15.100:6825/2819057142]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T06:33:13.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:13 smithi067 ceph-mon[31232]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T06:33:13.835 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T06:33:13.862 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi203:vg_nvme/lv_1 2024-09-21T06:33:14.111 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:33:14.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:14 smithi203 ceph-mon[33735]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T06:33:14.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:14 smithi203 ceph-mon[33735]: from='osd.10 [v2:172.21.15.100:6824/2819057142,v1:172.21.15.100:6825/2819057142]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]} : dispatch 2024-09-21T06:33:14.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:14 smithi203 ceph-mon[33735]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T06:33:14.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:14 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:33:14.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:14 smithi203 ceph-mon[33735]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]} : dispatch 2024-09-21T06:33:14.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:14 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:14.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:14 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:14.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:14 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:33:14.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:14 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:33:14.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:14 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:33:14.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:14 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T06:33:14.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:14 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:14.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:14 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:14.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:14 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:33:14.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:14 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:14.635 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.627+0000 7f0ab05d0640 1 -- 172.21.15.203:0/3833632482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aa8109470 msgr2=0x7f0aa810b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:14.636 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.627+0000 7f0ab05d0640 1 --2- 172.21.15.203:0/3833632482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aa8109470 0x7f0aa810b980 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0009900 tx=0x7f0aa002a410 comp rx=0 tx=0).stop 2024-09-21T06:33:14.636 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.627+0000 7f0ab05d0640 1 -- 172.21.15.203:0/3833632482 shutdown_connections 2024-09-21T06:33:14.636 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.627+0000 7f0ab05d0640 1 --2- 172.21.15.203:0/3833632482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aa8109470 0x7f0aa810b980 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:14.636 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.627+0000 7f0ab05d0640 1 --2- 172.21.15.203:0/3833632482 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0aa81068d0 0x7f0aa8108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:14.636 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.627+0000 7f0ab05d0640 1 --2- 172.21.15.203:0/3833632482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aa806c700 0x7f0aa8106390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:14.636 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.627+0000 7f0ab05d0640 1 -- 172.21.15.203:0/3833632482 >> 172.21.15.203:0/3833632482 conn(0x7f0aa80fdfb0 msgr2=0x7f0aa8100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:14.636 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.627+0000 7f0ab05d0640 1 -- 172.21.15.203:0/3833632482 shutdown_connections 2024-09-21T06:33:14.636 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.627+0000 7f0ab05d0640 1 -- 172.21.15.203:0/3833632482 wait complete. 2024-09-21T06:33:14.636 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.628+0000 7f0ab05d0640 1 Processor -- start 2024-09-21T06:33:14.636 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.628+0000 7f0ab05d0640 1 -- start start 2024-09-21T06:33:14.637 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.628+0000 7f0ab05d0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0aa806c700 0x7f0aa8199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:14.637 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.628+0000 7f0ab05d0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aa81068d0 0x7f0aa819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:14.637 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.628+0000 7f0ab05d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aa8109470 0x7f0aa819b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:14.637 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.628+0000 7f0ab05d0640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa810f220 con 0x7f0aa806c700 2024-09-21T06:33:14.637 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.628+0000 7f0ab05d0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa810f0a0 con 0x7f0aa8109470 2024-09-21T06:33:14.637 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.628+0000 7f0ab05d0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa810f3a0 con 0x7f0aa81068d0 2024-09-21T06:33:14.637 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.628+0000 7f0aadb6b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aa81068d0 0x7f0aa819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:14.637 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.628+0000 7f0aadb6b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aa81068d0 0x7f0aa819a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:37918/0 (socket says 172.21.15.203:37918) 2024-09-21T06:33:14.637 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.628+0000 7f0aadb6b640 1 -- 172.21.15.203:0/965761338 learned_addr learned my addr 172.21.15.203:0/965761338 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-09-21T06:33:14.637 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.628+0000 7f0aaeb6d640 1 --2- 172.21.15.203:0/965761338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aa8109470 0x7f0aa819b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:14.637 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.628+0000 7f0aae36c640 1 --2- 172.21.15.203:0/965761338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0aa806c700 0x7f0aa8199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:14.638 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.628+0000 7f0aadb6b640 1 -- 172.21.15.203:0/965761338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aa8109470 msgr2=0x7f0aa819b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:14.638 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.628+0000 7f0aadb6b640 1 --2- 172.21.15.203:0/965761338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aa8109470 0x7f0aa819b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:14.638 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.628+0000 7f0aadb6b640 1 -- 172.21.15.203:0/965761338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0aa806c700 msgr2=0x7f0aa8199e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:14.638 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.628+0000 7f0aadb6b640 1 --2- 172.21.15.203:0/965761338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0aa806c700 0x7f0aa8199e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:14.638 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.628+0000 7f0aadb6b640 1 -- 172.21.15.203:0/965761338 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa819b880 con 0x7f0aa81068d0 2024-09-21T06:33:14.638 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.628+0000 7f0aae36c640 1 --2- 172.21.15.203:0/965761338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0aa806c700 0x7f0aa8199e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:14.638 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.629+0000 7f0aadb6b640 1 --2- 172.21.15.203:0/965761338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aa81068d0 0x7f0aa819a4c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0a9800e9f0 tx=0x7f0a9800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:14.638 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.629+0000 7f0a9f7fe640 1 -- 172.21.15.203:0/965761338 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9800cd80 con 0x7f0aa81068d0 2024-09-21T06:33:14.638 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.629+0000 7f0a9f7fe640 1 -- 172.21.15.203:0/965761338 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0a980108c0 con 0x7f0aa81068d0 2024-09-21T06:33:14.638 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.629+0000 7f0ab05d0640 1 -- 172.21.15.203:0/965761338 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aa819bb70 con 0x7f0aa81068d0 2024-09-21T06:33:14.638 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.629+0000 7f0a9f7fe640 1 -- 172.21.15.203:0/965761338 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a980187c0 con 0x7f0aa81068d0 2024-09-21T06:33:14.638 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.629+0000 7f0ab05d0640 1 -- 172.21.15.203:0/965761338 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aa81a2270 con 0x7f0aa81068d0 2024-09-21T06:33:14.639 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.630+0000 7f0ab05d0640 1 -- 172.21.15.203:0/965761338 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aa80ffad0 con 0x7f0aa81068d0 2024-09-21T06:33:14.639 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.632+0000 7f0a9f7fe640 1 -- 172.21.15.203:0/965761338 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f0a98010430 con 0x7f0aa81068d0 2024-09-21T06:33:14.639 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.632+0000 7f0a9f7fe640 1 --2- 172.21.15.203:0/965761338 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f0a940b6510 0x7f0a940b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:14.639 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.633+0000 7f0a9f7fe640 1 -- 172.21.15.203:0/965761338 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6414+0+0 (secure 0 0 0) 0x7f0a980e7400 con 0x7f0aa81068d0 2024-09-21T06:33:14.639 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.633+0000 7f0aae36c640 1 --2- 172.21.15.203:0/965761338 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f0a940b6510 0x7f0a940b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:14.639 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.633+0000 7f0aae36c640 1 --2- 172.21.15.203:0/965761338 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f0a940b6510 0x7f0a940b8a20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4005fd0 tx=0x7f0aa4009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:14.639 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.635+0000 7f0a9f7fe640 1 -- 172.21.15.203:0/965761338 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0a980ae400 con 0x7f0aa81068d0 2024-09-21T06:33:14.749 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:14 smithi100 ceph-mon[33401]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T06:33:14.749 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:14 smithi100 ceph-mon[33401]: from='osd.10 [v2:172.21.15.100:6824/2819057142,v1:172.21.15.100:6825/2819057142]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]} : dispatch 2024-09-21T06:33:14.750 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:14 smithi100 ceph-mon[33401]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T06:33:14.750 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:14 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:33:14.750 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:14 smithi100 ceph-mon[33401]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]} : dispatch 2024-09-21T06:33:14.750 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:14 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:14.750 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:14 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:14.750 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:14 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:33:14.750 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:14 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:33:14.750 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:14 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:33:14.750 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:14 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T06:33:14.750 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:14 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:14.750 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:14 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:14.750 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:14 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:33:14.750 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:14 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:14.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:14 smithi067 ceph-mon[31232]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T06:33:14.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:14 smithi067 ceph-mon[31232]: from='osd.10 [v2:172.21.15.100:6824/2819057142,v1:172.21.15.100:6825/2819057142]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]} : dispatch 2024-09-21T06:33:14.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:14 smithi067 ceph-mon[31232]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T06:33:14.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:14 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:33:14.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:14 smithi067 ceph-mon[31232]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]} : dispatch 2024-09-21T06:33:14.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:14 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:14.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:14 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:14.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:14 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:33:14.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:14 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:33:14.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:14 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:33:14.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:14 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T06:33:14.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:14 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:14.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:14 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:14.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:14 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:33:14.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:14 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:14.830 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:14.828+0000 7f0ab05d0640 1 -- 172.21.15.203:0/965761338 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0aa8063800 con 0x7f0a940b6510 2024-09-21T06:33:15.053 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:33:14 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:33:14.747+0000 7f32df257640 -1 osd.10 0 waiting for initial osdmap 2024-09-21T06:33:15.053 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:33:14 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:33:14.762+0000 7f32d6838640 -1 osd.10 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:33:15.474 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:15 smithi203 ceph-mon[33735]: Detected new or changed devices on smithi100 2024-09-21T06:33:15.474 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:15 smithi203 ceph-mon[33735]: Adjusting osd_memory_target on smithi100 to 4271M 2024-09-21T06:33:15.474 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:15 smithi203 ceph-mon[33735]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:33:15.474 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:15 smithi203 ceph-mon[33735]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-21T06:33:15.474 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:15 smithi203 ceph-mon[33735]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T06:33:15.474 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:15 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:33:15.474 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:15 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:33:15.474 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:15 smithi203 ceph-mon[33735]: from='osd.10 ' entity='osd.10' 2024-09-21T06:33:15.474 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:15 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:33:15.474 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:15 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:33:15.474 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:15 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:15.475 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:15 smithi203 ceph-mon[33735]: osd.10 [v2:172.21.15.100:6824/2819057142,v1:172.21.15.100:6825/2819057142] boot 2024-09-21T06:33:15.475 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:15 smithi203 ceph-mon[33735]: osdmap e63: 11 total, 11 up, 11 in 2024-09-21T06:33:15.475 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:15 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:33:15.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:15 smithi067 ceph-mon[31232]: Detected new or changed devices on smithi100 2024-09-21T06:33:15.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:15 smithi067 ceph-mon[31232]: Adjusting osd_memory_target on smithi100 to 4271M 2024-09-21T06:33:15.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:15 smithi067 ceph-mon[31232]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:33:15.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:15 smithi067 ceph-mon[31232]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-21T06:33:15.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:15 smithi067 ceph-mon[31232]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T06:33:15.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:15 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:33:15.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:15 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:33:15.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:15 smithi067 ceph-mon[31232]: from='osd.10 ' entity='osd.10' 2024-09-21T06:33:15.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:15 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:33:15.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:15 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:33:15.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:15 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:15.795 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:15 smithi067 ceph-mon[31232]: osd.10 [v2:172.21.15.100:6824/2819057142,v1:172.21.15.100:6825/2819057142] boot 2024-09-21T06:33:15.795 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:15 smithi067 ceph-mon[31232]: osdmap e63: 11 total, 11 up, 11 in 2024-09-21T06:33:15.795 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:15 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:33:15.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:15 smithi100 ceph-mon[33401]: Detected new or changed devices on smithi100 2024-09-21T06:33:15.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:15 smithi100 ceph-mon[33401]: Adjusting osd_memory_target on smithi100 to 4271M 2024-09-21T06:33:15.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:15 smithi100 ceph-mon[33401]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:33:15.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:15 smithi100 ceph-mon[33401]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-21T06:33:15.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:15 smithi100 ceph-mon[33401]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T06:33:15.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:15 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:33:15.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:15 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:33:15.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:15 smithi100 ceph-mon[33401]: from='osd.10 ' entity='osd.10' 2024-09-21T06:33:15.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:15 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:33:15.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:15 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:33:15.806 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:15 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:15.806 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:15 smithi100 ceph-mon[33401]: osd.10 [v2:172.21.15.100:6824/2819057142,v1:172.21.15.100:6825/2819057142] boot 2024-09-21T06:33:15.806 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:15 smithi100 ceph-mon[33401]: osdmap e63: 11 total, 11 up, 11 in 2024-09-21T06:33:15.806 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:15 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:33:16.582 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:16 smithi203 ceph-mon[33735]: purged_snaps scrub starts 2024-09-21T06:33:16.582 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:16 smithi203 ceph-mon[33735]: purged_snaps scrub ok 2024-09-21T06:33:16.582 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:16 smithi203 ceph-mon[33735]: from='client.24520 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:33:16.582 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:16 smithi203 ceph-mon[33735]: osdmap e64: 11 total, 11 up, 11 in 2024-09-21T06:33:16.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:16 smithi067 ceph-mon[31232]: purged_snaps scrub starts 2024-09-21T06:33:16.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:16 smithi067 ceph-mon[31232]: purged_snaps scrub ok 2024-09-21T06:33:16.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:16 smithi067 ceph-mon[31232]: from='client.24520 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:33:16.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:16 smithi067 ceph-mon[31232]: osdmap e64: 11 total, 11 up, 11 in 2024-09-21T06:33:16.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:16 smithi100 ceph-mon[33401]: purged_snaps scrub starts 2024-09-21T06:33:16.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:16 smithi100 ceph-mon[33401]: purged_snaps scrub ok 2024-09-21T06:33:16.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:16 smithi100 ceph-mon[33401]: from='client.24520 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:33:16.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:16 smithi100 ceph-mon[33401]: osdmap e64: 11 total, 11 up, 11 in 2024-09-21T06:33:17.600 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:17 smithi203 ceph-mon[33735]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:33:17.600 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:17 smithi203 ceph-mon[33735]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T06:33:17.600 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:17 smithi203 ceph-mon[33735]: from='client.? 172.21.15.203:0/1293667186' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ea591507-10e2-4c08-8d29-d38ba9be96ef"} : dispatch 2024-09-21T06:33:17.600 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:17 smithi203 ceph-mon[33735]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ea591507-10e2-4c08-8d29-d38ba9be96ef"} : dispatch 2024-09-21T06:33:17.601 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:17 smithi203 ceph-mon[33735]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ea591507-10e2-4c08-8d29-d38ba9be96ef"}]': finished 2024-09-21T06:33:17.601 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:17 smithi203 ceph-mon[33735]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T06:33:17.601 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:17 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:33:17.705 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:17 smithi100 ceph-mon[33401]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:33:17.705 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:17 smithi100 ceph-mon[33401]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T06:33:17.705 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:17 smithi100 ceph-mon[33401]: from='client.? 172.21.15.203:0/1293667186' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ea591507-10e2-4c08-8d29-d38ba9be96ef"} : dispatch 2024-09-21T06:33:17.705 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:17 smithi100 ceph-mon[33401]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ea591507-10e2-4c08-8d29-d38ba9be96ef"} : dispatch 2024-09-21T06:33:17.705 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:17 smithi100 ceph-mon[33401]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ea591507-10e2-4c08-8d29-d38ba9be96ef"}]': finished 2024-09-21T06:33:17.705 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:17 smithi100 ceph-mon[33401]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T06:33:17.706 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:17 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:33:17.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:17 smithi067 ceph-mon[31232]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:33:17.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:17 smithi067 ceph-mon[31232]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T06:33:17.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:17 smithi067 ceph-mon[31232]: from='client.? 172.21.15.203:0/1293667186' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ea591507-10e2-4c08-8d29-d38ba9be96ef"} : dispatch 2024-09-21T06:33:17.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:17 smithi067 ceph-mon[31232]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ea591507-10e2-4c08-8d29-d38ba9be96ef"} : dispatch 2024-09-21T06:33:17.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:17 smithi067 ceph-mon[31232]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ea591507-10e2-4c08-8d29-d38ba9be96ef"}]': finished 2024-09-21T06:33:17.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:17 smithi067 ceph-mon[31232]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T06:33:17.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:17 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:33:18.725 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:18 smithi100 ceph-mon[33401]: from='client.? 172.21.15.203:0/2743006217' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:33:18.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:18 smithi203 ceph-mon[33735]: from='client.? 172.21.15.203:0/2743006217' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:33:18.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:18 smithi067 ceph-mon[31232]: from='client.? 172.21.15.203:0/2743006217' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:33:19.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:19 smithi203 ceph-mon[33735]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 165 KiB/s, 0 objects/s recovering 2024-09-21T06:33:19.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:19 smithi067 ceph-mon[31232]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 165 KiB/s, 0 objects/s recovering 2024-09-21T06:33:19.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:19 smithi100 ceph-mon[33401]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 165 KiB/s, 0 objects/s recovering 2024-09-21T06:33:21.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:21 smithi203 ceph-mon[33735]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-09-21T06:33:21.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:21 smithi067 ceph-mon[31232]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-09-21T06:33:21.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:21 smithi100 ceph-mon[33401]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-09-21T06:33:23.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:23 smithi203 ceph-mon[33735]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-21T06:33:23.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:23 smithi067 ceph-mon[31232]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-21T06:33:23.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:23 smithi100 ceph-mon[33401]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-21T06:33:25.476 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:25 smithi203 ceph-mon[33735]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:33:25.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:25 smithi067 ceph-mon[31232]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:33:25.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:25 smithi100 ceph-mon[33401]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:33:26.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:26 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T06:33:26.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:26 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:26.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:26 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T06:33:26.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:26 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:26.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:26 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T06:33:26.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:26 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:27.549 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:27 smithi203 ceph-mon[33735]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-21T06:33:27.549 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:27 smithi203 ceph-mon[33735]: Deploying daemon osd.11 on smithi203 2024-09-21T06:33:27.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:27 smithi067 ceph-mon[31232]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-21T06:33:27.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:27 smithi067 ceph-mon[31232]: Deploying daemon osd.11 on smithi203 2024-09-21T06:33:27.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:27 smithi100 ceph-mon[33401]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-21T06:33:27.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:27 smithi100 ceph-mon[33401]: Deploying daemon osd.11 on smithi203 2024-09-21T06:33:29.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:29 smithi203 ceph-mon[33735]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-21T06:33:29.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:29 smithi067 ceph-mon[31232]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-21T06:33:29.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:29 smithi100 ceph-mon[33401]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-21T06:33:31.545 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:31 smithi203 ceph-mon[33735]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:33:31.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:31 smithi067 ceph-mon[31232]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:33:31.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:31 smithi100 ceph-mon[33401]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:33:32.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:32 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:33:32.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:32 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:32.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:32 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:32.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:32 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:33:32.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:32 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:32.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:32 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:32.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:32 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:33:32.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:32 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:32.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:32 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:33.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:33 smithi203 ceph-mon[33735]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:33:33.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:33 smithi067 ceph-mon[31232]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:33:33.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:33 smithi100 ceph-mon[33401]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:33:35.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:35 smithi203 ceph-mon[33735]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:33:35.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:35 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:35.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:35 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:35.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:35 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:35.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:35 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:33:35.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:35 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:35.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:35 smithi067 ceph-mon[31232]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:33:35.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:35.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:35.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:35.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:33:35.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:35.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:35 smithi100 ceph-mon[33401]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:33:35.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:35 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:35.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:35 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:35.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:35 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:35.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:35 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:33:35.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:35 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:35.854 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:35.852+0000 7f0a9f7fe640 1 -- 172.21.15.203:0/965761338 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f0aa8063800 con 0x7f0a940b6510 2024-09-21T06:33:35.854 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 11 on host 'smithi203' 2024-09-21T06:33:35.857 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:35.855+0000 7f0ab05d0640 1 -- 172.21.15.203:0/965761338 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f0a940b6510 msgr2=0x7f0a940b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:35.857 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:35.855+0000 7f0ab05d0640 1 --2- 172.21.15.203:0/965761338 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f0a940b6510 0x7f0a940b8a20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4005fd0 tx=0x7f0aa4009600 comp rx=0 tx=0).stop 2024-09-21T06:33:35.857 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:35.855+0000 7f0ab05d0640 1 -- 172.21.15.203:0/965761338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aa81068d0 msgr2=0x7f0aa819a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:35.857 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:35.855+0000 7f0ab05d0640 1 --2- 172.21.15.203:0/965761338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aa81068d0 0x7f0aa819a4c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0a9800e9f0 tx=0x7f0a9800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:33:35.857 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:35.855+0000 7f0ab05d0640 1 -- 172.21.15.203:0/965761338 shutdown_connections 2024-09-21T06:33:35.857 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:35.855+0000 7f0ab05d0640 1 --2- 172.21.15.203:0/965761338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aa8109470 0x7f0aa819b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:35.857 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:35.855+0000 7f0ab05d0640 1 --2- 172.21.15.203:0/965761338 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f0a940b6510 0x7f0a940b8a20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:35.858 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:35.855+0000 7f0ab05d0640 1 --2- 172.21.15.203:0/965761338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aa81068d0 0x7f0aa819a4c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:35.858 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:35.855+0000 7f0ab05d0640 1 --2- 172.21.15.203:0/965761338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0aa806c700 0x7f0aa8199e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:35.858 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:35.855+0000 7f0ab05d0640 1 -- 172.21.15.203:0/965761338 >> 172.21.15.203:0/965761338 conn(0x7f0aa80fdfb0 msgr2=0x7f0aa8104e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:35.858 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:35.855+0000 7f0ab05d0640 1 -- 172.21.15.203:0/965761338 shutdown_connections 2024-09-21T06:33:35.858 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:35.855+0000 7f0ab05d0640 1 -- 172.21.15.203:0/965761338 wait complete. 2024-09-21T06:33:36.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:36 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:33:36.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:36 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:36.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:36 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:36.576 DEBUG:teuthology.orchestra.run.smithi203:osd.11> sudo journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.11.service 2024-09-21T06:33:36.578 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-21T06:33:36.578 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T06:33:36.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:36 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:33:36.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:36 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:36.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:36 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:36.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:36 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:33:36.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:36 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:36.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:36 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:36.849 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:33:37.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.442+0000 7f45c5ff0640 1 -- 172.21.15.67:0/711118629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45c006c340 msgr2=0x7f45c01063e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:37.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.442+0000 7f45c5ff0640 1 --2- 172.21.15.67:0/711118629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45c006c340 0x7f45c01063e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009900 tx=0x7f45b002a410 comp rx=0 tx=0).stop 2024-09-21T06:33:37.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.442+0000 7f45c5ff0640 1 -- 172.21.15.67:0/711118629 shutdown_connections 2024-09-21T06:33:37.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.442+0000 7f45c5ff0640 1 --2- 172.21.15.67:0/711118629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45c01094c0 0x7f45c010b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:37.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.442+0000 7f45c5ff0640 1 --2- 172.21.15.67:0/711118629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45c0106920 0x7f45c0108d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:37.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.442+0000 7f45c5ff0640 1 --2- 172.21.15.67:0/711118629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45c006c340 0x7f45c01063e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:37.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.442+0000 7f45c5ff0640 1 -- 172.21.15.67:0/711118629 >> 172.21.15.67:0/711118629 conn(0x7f45c00fe000 msgr2=0x7f45c01004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:37.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.443+0000 7f45c5ff0640 1 -- 172.21.15.67:0/711118629 shutdown_connections 2024-09-21T06:33:37.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.443+0000 7f45c5ff0640 1 -- 172.21.15.67:0/711118629 wait complete. 2024-09-21T06:33:37.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.443+0000 7f45c5ff0640 1 Processor -- start 2024-09-21T06:33:37.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.443+0000 7f45c5ff0640 1 -- start start 2024-09-21T06:33:37.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.444+0000 7f45c5ff0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45c006c340 0x7f45c0199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:37.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.444+0000 7f45c5ff0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45c0106920 0x7f45c019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:37.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.444+0000 7f45c5ff0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45c01094c0 0x7f45c019b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:37.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.444+0000 7f45c5ff0640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f45c010f260 con 0x7f45c01094c0 2024-09-21T06:33:37.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.444+0000 7f45c5ff0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f45c010f0e0 con 0x7f45c006c340 2024-09-21T06:33:37.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.444+0000 7f45c5ff0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f45c010f3e0 con 0x7f45c0106920 2024-09-21T06:33:37.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.444+0000 7f45bffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45c01094c0 0x7f45c019b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:37.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.444+0000 7f45bffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45c01094c0 0x7f45c019b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59850/0 (socket says 172.21.15.67:59850) 2024-09-21T06:33:37.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.444+0000 7f45bffff640 1 -- 172.21.15.67:0/2734229140 learned_addr learned my addr 172.21.15.67:0/2734229140 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:33:37.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.444+0000 7f45b6ffd640 1 --2- 172.21.15.67:0/2734229140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45c0106920 0x7f45c019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:37.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.444+0000 7f45bf7fe640 1 --2- 172.21.15.67:0/2734229140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45c006c340 0x7f45c0199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:37.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.444+0000 7f45bffff640 1 -- 172.21.15.67:0/2734229140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45c0106920 msgr2=0x7f45c019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:37.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.444+0000 7f45bffff640 1 --2- 172.21.15.67:0/2734229140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45c0106920 0x7f45c019a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:37.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.444+0000 7f45bffff640 1 -- 172.21.15.67:0/2734229140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45c006c340 msgr2=0x7f45c0199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:37.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.444+0000 7f45bffff640 1 --2- 172.21.15.67:0/2734229140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45c006c340 0x7f45c0199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:37.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.444+0000 7f45bffff640 1 -- 172.21.15.67:0/2734229140 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c019b890 con 0x7f45c01094c0 2024-09-21T06:33:37.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.444+0000 7f45bf7fe640 1 --2- 172.21.15.67:0/2734229140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45c006c340 0x7f45c0199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:33:37.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.444+0000 7f45bffff640 1 --2- 172.21.15.67:0/2734229140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45c01094c0 0x7f45c019b100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f45b800eaa0 tx=0x7f45b800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:37.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.445+0000 7f45bd7fa640 1 -- 172.21.15.67:0/2734229140 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8004d20 con 0x7f45c01094c0 2024-09-21T06:33:37.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.445+0000 7f45bd7fa640 1 -- 172.21.15.67:0/2734229140 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f45b8005020 con 0x7f45c01094c0 2024-09-21T06:33:37.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.445+0000 7f45c5ff0640 1 -- 172.21.15.67:0/2734229140 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c019bb80 con 0x7f45c01094c0 2024-09-21T06:33:37.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.445+0000 7f45bd7fa640 1 -- 172.21.15.67:0/2734229140 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b80116d0 con 0x7f45c01094c0 2024-09-21T06:33:37.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.445+0000 7f45c5ff0640 1 -- 172.21.15.67:0/2734229140 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c01a2140 con 0x7f45c01094c0 2024-09-21T06:33:37.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.446+0000 7f45c5ff0640 1 -- 172.21.15.67:0/2734229140 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45c00ffb20 con 0x7f45c01094c0 2024-09-21T06:33:37.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.447+0000 7f45bd7fa640 1 -- 172.21.15.67:0/2734229140 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f45b80051d0 con 0x7f45c01094c0 2024-09-21T06:33:37.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.448+0000 7f45bd7fa640 1 --2- 172.21.15.67:0/2734229140 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f45ac0b6510 0x7f45ac0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:37.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.449+0000 7f45bf7fe640 1 --2- 172.21.15.67:0/2734229140 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f45ac0b6510 0x7f45ac0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:37.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.449+0000 7f45bd7fa640 1 -- 172.21.15.67:0/2734229140 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6809+0+0 (secure 0 0 0) 0x7f45b80e6ee0 con 0x7f45c01094c0 2024-09-21T06:33:37.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.449+0000 7f45bf7fe640 1 --2- 172.21.15.67:0/2734229140 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f45ac0b6510 0x7f45ac0b8a20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f45b0005f70 tx=0x7f45b0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:37.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.452+0000 7f45bd7fa640 1 -- 172.21.15.67:0/2734229140 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f45b8016030 con 0x7f45c01094c0 2024-09-21T06:33:37.494 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:37 smithi067 ceph-mon[31232]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:33:37.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.643+0000 7f45c5ff0640 1 -- 172.21.15.67:0/2734229140 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f45c002e640 con 0x7f45c01094c0 2024-09-21T06:33:37.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.643+0000 7f45bd7fa640 1 -- 172.21.15.67:0/2734229140 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v66) ==== 74+0+133 (secure 0 0 0) 0x7f45b80add50 con 0x7f45c01094c0 2024-09-21T06:33:37.645 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:33:37.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.646+0000 7f45c5ff0640 1 -- 172.21.15.67:0/2734229140 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f45ac0b6510 msgr2=0x7f45ac0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:37.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.646+0000 7f45c5ff0640 1 --2- 172.21.15.67:0/2734229140 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f45ac0b6510 0x7f45ac0b8a20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f45b0005f70 tx=0x7f45b0000bc0 comp rx=0 tx=0).stop 2024-09-21T06:33:37.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.646+0000 7f45c5ff0640 1 -- 172.21.15.67:0/2734229140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45c01094c0 msgr2=0x7f45c019b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:37.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.646+0000 7f45c5ff0640 1 --2- 172.21.15.67:0/2734229140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45c01094c0 0x7f45c019b100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f45b800eaa0 tx=0x7f45b800eea0 comp rx=0 tx=0).stop 2024-09-21T06:33:37.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.646+0000 7f45c5ff0640 1 -- 172.21.15.67:0/2734229140 shutdown_connections 2024-09-21T06:33:37.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.646+0000 7f45c5ff0640 1 --2- 172.21.15.67:0/2734229140 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f45ac0b6510 0x7f45ac0b8a20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:37.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.646+0000 7f45c5ff0640 1 --2- 172.21.15.67:0/2734229140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45c01094c0 0x7f45c019b100 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:37.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.646+0000 7f45c5ff0640 1 --2- 172.21.15.67:0/2734229140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45c0106920 0x7f45c019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:37.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.646+0000 7f45c5ff0640 1 --2- 172.21.15.67:0/2734229140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45c006c340 0x7f45c0199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:37.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.646+0000 7f45c5ff0640 1 -- 172.21.15.67:0/2734229140 >> 172.21.15.67:0/2734229140 conn(0x7f45c00fe000 msgr2=0x7f45c0104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:37.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.646+0000 7f45c5ff0640 1 -- 172.21.15.67:0/2734229140 shutdown_connections 2024-09-21T06:33:37.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:37.646+0000 7f45c5ff0640 1 -- 172.21.15.67:0/2734229140 wait complete. 2024-09-21T06:33:37.728 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:37 smithi203 ceph-mon[33735]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:33:37.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:37 smithi100 ceph-mon[33401]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:33:38.374 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":66,"num_osds":12,"num_up_osds":11,"osd_up_since":1726900395,"num_in_osds":12,"osd_in_since":1726900397,"num_remapped_pgs":0} 2024-09-21T06:33:38.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:38 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2734229140' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:33:38.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:38 smithi203 ceph-mon[33735]: from='osd.11 [v2:172.21.15.203:6824/566502600,v1:172.21.15.203:6825/566502600]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T06:33:38.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:38 smithi203 ceph-mon[33735]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T06:33:38.710 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:33:38 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:33:38.262+0000 7f3105a0a640 -1 osd.11 0 log_to_monitors true 2024-09-21T06:33:38.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:38 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2734229140' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:33:38.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:38 smithi067 ceph-mon[31232]: from='osd.11 [v2:172.21.15.203:6824/566502600,v1:172.21.15.203:6825/566502600]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T06:33:38.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:38 smithi067 ceph-mon[31232]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T06:33:38.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:38 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2734229140' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:33:38.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:38 smithi100 ceph-mon[33401]: from='osd.11 [v2:172.21.15.203:6824/566502600,v1:172.21.15.203:6825/566502600]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T06:33:38.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:38 smithi100 ceph-mon[33401]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T06:33:39.376 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T06:33:39.652 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:33:39.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:39 smithi203 ceph-mon[33735]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:33:39.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:39 smithi203 ceph-mon[33735]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T06:33:39.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:39 smithi203 ceph-mon[33735]: from='osd.11 [v2:172.21.15.203:6824/566502600,v1:172.21.15.203:6825/566502600]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T06:33:39.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:39 smithi203 ceph-mon[33735]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T06:33:39.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:39 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:33:39.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:39 smithi203 ceph-mon[33735]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T06:33:39.773 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:39 smithi067 ceph-mon[31232]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:33:39.773 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:39 smithi067 ceph-mon[31232]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T06:33:39.773 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:39 smithi067 ceph-mon[31232]: from='osd.11 [v2:172.21.15.203:6824/566502600,v1:172.21.15.203:6825/566502600]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T06:33:39.773 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:39 smithi067 ceph-mon[31232]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T06:33:39.773 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:39 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:33:39.773 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:39 smithi067 ceph-mon[31232]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T06:33:39.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:39 smithi100 ceph-mon[33401]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:33:39.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:39 smithi100 ceph-mon[33401]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T06:33:39.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:39 smithi100 ceph-mon[33401]: from='osd.11 [v2:172.21.15.203:6824/566502600,v1:172.21.15.203:6825/566502600]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T06:33:39.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:39 smithi100 ceph-mon[33401]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T06:33:39.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:39 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:33:39.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:39 smithi100 ceph-mon[33401]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T06:33:40.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.141+0000 7f22768de640 1 -- 172.21.15.67:0/3666107970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2270104070 msgr2=0x7f227010aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:40.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.141+0000 7f22768de640 1 --2- 172.21.15.67:0/3666107970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2270104070 0x7f227010aa10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2268009ca0 tx=0x7f226802a950 comp rx=0 tx=0).stop 2024-09-21T06:33:40.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.141+0000 7f22768de640 1 -- 172.21.15.67:0/3666107970 shutdown_connections 2024-09-21T06:33:40.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.141+0000 7f22768de640 1 --2- 172.21.15.67:0/3666107970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22701036d0 0x7f2270103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:40.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.141+0000 7f22768de640 1 --2- 172.21.15.67:0/3666107970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2270104070 0x7f227010aa10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:40.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.141+0000 7f22768de640 1 --2- 172.21.15.67:0/3666107970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2270102640 0x7f2270102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:40.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.141+0000 7f22768de640 1 -- 172.21.15.67:0/3666107970 >> 172.21.15.67:0/3666107970 conn(0x7f22700fdfb0 msgr2=0x7f2270100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:40.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.141+0000 7f22768de640 1 -- 172.21.15.67:0/3666107970 shutdown_connections 2024-09-21T06:33:40.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.141+0000 7f22768de640 1 -- 172.21.15.67:0/3666107970 wait complete. 2024-09-21T06:33:40.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.142+0000 7f22768de640 1 Processor -- start 2024-09-21T06:33:40.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.142+0000 7f22768de640 1 -- start start 2024-09-21T06:33:40.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.142+0000 7f22768de640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2270102640 0x7f227006b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:40.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.142+0000 7f22768de640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22701036d0 0x7f227019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:40.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.142+0000 7f22768de640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2270104070 0x7f22701a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:40.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.142+0000 7f22768de640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f227010e140 con 0x7f2270104070 2024-09-21T06:33:40.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.142+0000 7f22768de640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f227010dfc0 con 0x7f2270102640 2024-09-21T06:33:40.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.142+0000 7f22768de640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f227010e2c0 con 0x7f22701036d0 2024-09-21T06:33:40.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.142+0000 7f2274e7b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2270104070 0x7f22701a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:40.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.142+0000 7f2274e7b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2270104070 0x7f22701a1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59874/0 (socket says 172.21.15.67:59874) 2024-09-21T06:33:40.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.142+0000 7f226ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2270102640 0x7f227006b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:40.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.142+0000 7f2274e7b640 1 -- 172.21.15.67:0/3450605176 learned_addr learned my addr 172.21.15.67:0/3450605176 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:33:40.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.142+0000 7f2267fff640 1 --2- 172.21.15.67:0/3450605176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22701036d0 0x7f227019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:40.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.143+0000 7f2274e7b640 1 -- 172.21.15.67:0/3450605176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22701036d0 msgr2=0x7f227019ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:40.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.143+0000 7f2274e7b640 1 --2- 172.21.15.67:0/3450605176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22701036d0 0x7f227019ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:40.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.143+0000 7f2274e7b640 1 -- 172.21.15.67:0/3450605176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2270102640 msgr2=0x7f227006b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:40.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.143+0000 7f2274e7b640 1 --2- 172.21.15.67:0/3450605176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2270102640 0x7f227006b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:40.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.143+0000 7f2274e7b640 1 -- 172.21.15.67:0/3450605176 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22701a1970 con 0x7f2270104070 2024-09-21T06:33:40.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.143+0000 7f2274e7b640 1 --2- 172.21.15.67:0/3450605176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2270104070 0x7f22701a1170 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f226802f800 tx=0x7f2268003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:40.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.143+0000 7f226dffb640 1 -- 172.21.15.67:0/3450605176 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2268034040 con 0x7f2270104070 2024-09-21T06:33:40.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.143+0000 7f22768de640 1 -- 172.21.15.67:0/3450605176 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22701a1c00 con 0x7f2270104070 2024-09-21T06:33:40.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.143+0000 7f226dffb640 1 -- 172.21.15.67:0/3450605176 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2268030850 con 0x7f2270104070 2024-09-21T06:33:40.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.143+0000 7f226dffb640 1 -- 172.21.15.67:0/3450605176 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2268039430 con 0x7f2270104070 2024-09-21T06:33:40.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.143+0000 7f22768de640 1 -- 172.21.15.67:0/3450605176 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22701a2130 con 0x7f2270104070 2024-09-21T06:33:40.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.144+0000 7f22768de640 1 -- 172.21.15.67:0/3450605176 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2270109820 con 0x7f2270104070 2024-09-21T06:33:40.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.146+0000 7f226dffb640 1 -- 172.21.15.67:0/3450605176 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f2268037020 con 0x7f2270104070 2024-09-21T06:33:40.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.146+0000 7f226dffb640 1 --2- 172.21.15.67:0/3450605176 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f225c0b6510 0x7f225c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:40.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.147+0000 7f226ffff640 1 --2- 172.21.15.67:0/3450605176 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f225c0b6510 0x7f225c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:40.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.147+0000 7f226dffb640 1 -- 172.21.15.67:0/3450605176 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6847+0+0 (secure 0 0 0) 0x7f2268107dd0 con 0x7f2270104070 2024-09-21T06:33:40.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.147+0000 7f226ffff640 1 --2- 172.21.15.67:0/3450605176 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f225c0b6510 0x7f225c0b8a20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2260005fd0 tx=0x7f226000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:40.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.150+0000 7f226dffb640 1 -- 172.21.15.67:0/3450605176 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f22680cecb0 con 0x7f2270104070 2024-09-21T06:33:40.232 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:33:39 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:33:39.881+0000 7f310198c640 -1 osd.11 0 waiting for initial osdmap 2024-09-21T06:33:40.232 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:33:39 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:33:39.895+0000 7f30f9782640 -1 osd.11 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:33:40.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.338+0000 7f22768de640 1 -- 172.21.15.67:0/3450605176 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f227002e890 con 0x7f2270104070 2024-09-21T06:33:40.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.339+0000 7f226dffb640 1 -- 172.21.15.67:0/3450605176 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v68) ==== 74+0+133 (secure 0 0 0) 0x7f22680d3090 con 0x7f2270104070 2024-09-21T06:33:40.343 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:33:40.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.341+0000 7f22768de640 1 -- 172.21.15.67:0/3450605176 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f225c0b6510 msgr2=0x7f225c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:40.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.341+0000 7f22768de640 1 --2- 172.21.15.67:0/3450605176 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f225c0b6510 0x7f225c0b8a20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2260005fd0 tx=0x7f226000b390 comp rx=0 tx=0).stop 2024-09-21T06:33:40.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.341+0000 7f22768de640 1 -- 172.21.15.67:0/3450605176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2270104070 msgr2=0x7f22701a1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:40.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.341+0000 7f22768de640 1 --2- 172.21.15.67:0/3450605176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2270104070 0x7f22701a1170 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f226802f800 tx=0x7f2268003f90 comp rx=0 tx=0).stop 2024-09-21T06:33:40.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.341+0000 7f22768de640 1 -- 172.21.15.67:0/3450605176 shutdown_connections 2024-09-21T06:33:40.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.341+0000 7f22768de640 1 --2- 172.21.15.67:0/3450605176 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f225c0b6510 0x7f225c0b8a20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:40.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.341+0000 7f22768de640 1 --2- 172.21.15.67:0/3450605176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2270104070 0x7f22701a1170 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:40.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.341+0000 7f22768de640 1 --2- 172.21.15.67:0/3450605176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22701036d0 0x7f227019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:40.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.341+0000 7f22768de640 1 --2- 172.21.15.67:0/3450605176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2270102640 0x7f227006b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:40.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.341+0000 7f22768de640 1 -- 172.21.15.67:0/3450605176 >> 172.21.15.67:0/3450605176 conn(0x7f22700fdfb0 msgr2=0x7f22701069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:40.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.341+0000 7f22768de640 1 -- 172.21.15.67:0/3450605176 shutdown_connections 2024-09-21T06:33:40.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:40.341+0000 7f22768de640 1 -- 172.21.15.67:0/3450605176 wait complete. 2024-09-21T06:33:40.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:40 smithi203 ceph-mon[33735]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-21T06:33:40.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:40 smithi203 ceph-mon[33735]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T06:33:40.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:33:40.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:33:40.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:40 smithi203 ceph-mon[33735]: Detected new or changed devices on smithi203 2024-09-21T06:33:40.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:40.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:40.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:33:40.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:33:40.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:33:40.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T06:33:40.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:40 smithi203 ceph-mon[33735]: Adjusting osd_memory_target on smithi203 to 4272M 2024-09-21T06:33:40.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:40.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:40.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:33:40.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:40.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:40 smithi203 ceph-mon[33735]: from='osd.11 ' entity='osd.11' 2024-09-21T06:33:40.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:40 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3450605176' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:33:40.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:40 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:33:40.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:40 smithi067 ceph-mon[31232]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-21T06:33:40.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:40 smithi067 ceph-mon[31232]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T06:33:40.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:40 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:33:40.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:40 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:33:40.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:40 smithi067 ceph-mon[31232]: Detected new or changed devices on smithi203 2024-09-21T06:33:40.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:40 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:40.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:40 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:40.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:40 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:33:40.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:40 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:33:40.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:40 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:33:40.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:40 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T06:33:40.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:40 smithi067 ceph-mon[31232]: Adjusting osd_memory_target on smithi203 to 4272M 2024-09-21T06:33:40.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:40 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:40.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:40 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:40.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:40 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:33:40.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:40 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:40.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:40 smithi067 ceph-mon[31232]: from='osd.11 ' entity='osd.11' 2024-09-21T06:33:40.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:40 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3450605176' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:33:40.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:40 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:33:40.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:40 smithi100 ceph-mon[33401]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-21T06:33:40.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:40 smithi100 ceph-mon[33401]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T06:33:40.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:40 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:33:40.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:40 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:33:40.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:40 smithi100 ceph-mon[33401]: Detected new or changed devices on smithi203 2024-09-21T06:33:40.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:40 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:40.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:40 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:40.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:40 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:33:40.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:40 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:33:40.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:40 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:33:40.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:40 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T06:33:40.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:40 smithi100 ceph-mon[33401]: Adjusting osd_memory_target on smithi203 to 4272M 2024-09-21T06:33:40.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:40 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:40.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:40 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:40.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:40 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:33:40.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:40 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:40.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:40 smithi100 ceph-mon[33401]: from='osd.11 ' entity='osd.11' 2024-09-21T06:33:40.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:40 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3450605176' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:33:40.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:40 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:33:40.975 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":68,"num_osds":12,"num_up_osds":11,"osd_up_since":1726900395,"num_in_osds":12,"osd_in_since":1726900397,"num_remapped_pgs":0} 2024-09-21T06:33:41.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:41 smithi067 ceph-mon[31232]: purged_snaps scrub starts 2024-09-21T06:33:41.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:41 smithi067 ceph-mon[31232]: purged_snaps scrub ok 2024-09-21T06:33:41.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:41 smithi067 ceph-mon[31232]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:33:41.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:41 smithi067 ceph-mon[31232]: osd.11 [v2:172.21.15.203:6824/566502600,v1:172.21.15.203:6825/566502600] boot 2024-09-21T06:33:41.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:41 smithi067 ceph-mon[31232]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T06:33:41.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:41 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:33:41.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:41 smithi100 ceph-mon[33401]: purged_snaps scrub starts 2024-09-21T06:33:41.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:41 smithi100 ceph-mon[33401]: purged_snaps scrub ok 2024-09-21T06:33:41.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:41 smithi100 ceph-mon[33401]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:33:41.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:41 smithi100 ceph-mon[33401]: osd.11 [v2:172.21.15.203:6824/566502600,v1:172.21.15.203:6825/566502600] boot 2024-09-21T06:33:41.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:41 smithi100 ceph-mon[33401]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T06:33:41.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:41 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:33:41.976 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T06:33:41.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:41 smithi203 ceph-mon[33735]: purged_snaps scrub starts 2024-09-21T06:33:41.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:41 smithi203 ceph-mon[33735]: purged_snaps scrub ok 2024-09-21T06:33:41.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:41 smithi203 ceph-mon[33735]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:33:41.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:41 smithi203 ceph-mon[33735]: osd.11 [v2:172.21.15.203:6824/566502600,v1:172.21.15.203:6825/566502600] boot 2024-09-21T06:33:41.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:41 smithi203 ceph-mon[33735]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T06:33:41.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:41 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:33:42.245 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:33:42.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.726+0000 7f7278c41640 1 -- 172.21.15.67:0/3683251477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72741036e0 msgr2=0x7f7274103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:42.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.726+0000 7f7278c41640 1 --2- 172.21.15.67:0/3683251477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72741036e0 0x7f7274103af0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7268009900 tx=0x7f726802a480 comp rx=0 tx=0).stop 2024-09-21T06:33:42.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.726+0000 7f7278c41640 1 -- 172.21.15.67:0/3683251477 shutdown_connections 2024-09-21T06:33:42.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.726+0000 7f7278c41640 1 --2- 172.21.15.67:0/3683251477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7274104080 0x7f727410aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:42.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.726+0000 7f7278c41640 1 --2- 172.21.15.67:0/3683251477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72741036e0 0x7f7274103af0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.726+0000 7f7278c41640 1 --2- 172.21.15.67:0/3683251477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7274102650 0x7f7274102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.726+0000 7f7278c41640 1 -- 172.21.15.67:0/3683251477 >> 172.21.15.67:0/3683251477 conn(0x7f72740fdfe0 msgr2=0x7f7274100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.726+0000 7f7278c41640 1 -- 172.21.15.67:0/3683251477 shutdown_connections 2024-09-21T06:33:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.726+0000 7f7278c41640 1 -- 172.21.15.67:0/3683251477 wait complete. 2024-09-21T06:33:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.727+0000 7f7278c41640 1 Processor -- start 2024-09-21T06:33:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.727+0000 7f7278c41640 1 -- start start 2024-09-21T06:33:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.727+0000 7f7278c41640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7274102650 0x7f727406c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.727+0000 7f7278c41640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72741036e0 0x7f727419ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.727+0000 7f7278c41640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7274104080 0x7f72741a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.727+0000 7f7278c41640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f727410e1a0 con 0x7f7274104080 2024-09-21T06:33:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.727+0000 7f7278c41640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f727410e020 con 0x7f72741036e0 2024-09-21T06:33:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.727+0000 7f7278c41640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f727410e320 con 0x7f7274102650 2024-09-21T06:33:42.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.727+0000 7f7272d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7274104080 0x7f72741a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:42.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.727+0000 7f7272d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7274104080 0x7f72741a1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59888/0 (socket says 172.21.15.67:59888) 2024-09-21T06:33:42.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.727+0000 7f7272d9d640 1 -- 172.21.15.67:0/2107746034 learned_addr learned my addr 172.21.15.67:0/2107746034 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:33:42.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.727+0000 7f7271d9b640 1 --2- 172.21.15.67:0/2107746034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72741036e0 0x7f727419ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:42.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.727+0000 7f727259c640 1 --2- 172.21.15.67:0/2107746034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7274102650 0x7f727406c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:42.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.728+0000 7f7272d9d640 1 -- 172.21.15.67:0/2107746034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7274102650 msgr2=0x7f727406c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:42.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.728+0000 7f7272d9d640 1 --2- 172.21.15.67:0/2107746034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7274102650 0x7f727406c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:42.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.728+0000 7f7272d9d640 1 -- 172.21.15.67:0/2107746034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72741036e0 msgr2=0x7f727419ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:42.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.728+0000 7f7272d9d640 1 --2- 172.21.15.67:0/2107746034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72741036e0 0x7f727419ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:42.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.728+0000 7f7272d9d640 1 -- 172.21.15.67:0/2107746034 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72741a19a0 con 0x7f7274104080 2024-09-21T06:33:42.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.728+0000 7f727259c640 1 --2- 172.21.15.67:0/2107746034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7274102650 0x7f727406c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:42.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.728+0000 7f7272d9d640 1 --2- 172.21.15.67:0/2107746034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7274104080 0x7f72741a1230 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f726c00cce0 tx=0x7f726c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:42.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.728+0000 7f72637fe640 1 -- 172.21.15.67:0/2107746034 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c00f450 con 0x7f7274104080 2024-09-21T06:33:42.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.728+0000 7f72637fe640 1 -- 172.21.15.67:0/2107746034 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f726c00fa80 con 0x7f7274104080 2024-09-21T06:33:42.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.728+0000 7f7278c41640 1 -- 172.21.15.67:0/2107746034 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72741a1c90 con 0x7f7274104080 2024-09-21T06:33:42.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.728+0000 7f72637fe640 1 -- 172.21.15.67:0/2107746034 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c018ad0 con 0x7f7274104080 2024-09-21T06:33:42.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.728+0000 7f7278c41640 1 -- 172.21.15.67:0/2107746034 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72741a2180 con 0x7f7274104080 2024-09-21T06:33:42.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.729+0000 7f7278c41640 1 -- 172.21.15.67:0/2107746034 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7274109830 con 0x7f7274104080 2024-09-21T06:33:42.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.732+0000 7f72637fe640 1 -- 172.21.15.67:0/2107746034 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f726c00f5f0 con 0x7f7274104080 2024-09-21T06:33:42.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.733+0000 7f72637fe640 1 --2- 172.21.15.67:0/2107746034 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f725c0b6590 0x7f725c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:42.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.733+0000 7f727259c640 1 --2- 172.21.15.67:0/2107746034 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f725c0b6590 0x7f725c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:42.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.733+0000 7f72637fe640 1 -- 172.21.15.67:0/2107746034 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f726c0e76f0 con 0x7f7274104080 2024-09-21T06:33:42.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.733+0000 7f727259c640 1 --2- 172.21.15.67:0/2107746034 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f725c0b6590 0x7f725c0b8aa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7264009930 tx=0x7f7264008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:42.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.736+0000 7f72637fe640 1 -- 172.21.15.67:0/2107746034 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f726c014030 con 0x7f7274104080 2024-09-21T06:33:42.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.923+0000 7f7278c41640 1 -- 172.21.15.67:0/2107746034 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f727402e890 con 0x7f7274104080 2024-09-21T06:33:42.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.923+0000 7f72637fe640 1 -- 172.21.15.67:0/2107746034 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7f726c0ae490 con 0x7f7274104080 2024-09-21T06:33:42.926 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:33:42.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.926+0000 7f7278c41640 1 -- 172.21.15.67:0/2107746034 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f725c0b6590 msgr2=0x7f725c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:42.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.926+0000 7f7278c41640 1 --2- 172.21.15.67:0/2107746034 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f725c0b6590 0x7f725c0b8aa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7264009930 tx=0x7f7264008040 comp rx=0 tx=0).stop 2024-09-21T06:33:42.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.926+0000 7f7278c41640 1 -- 172.21.15.67:0/2107746034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7274104080 msgr2=0x7f72741a1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:42.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.926+0000 7f7278c41640 1 --2- 172.21.15.67:0/2107746034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7274104080 0x7f72741a1230 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f726c00cce0 tx=0x7f726c00a620 comp rx=0 tx=0).stop 2024-09-21T06:33:42.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.926+0000 7f7278c41640 1 -- 172.21.15.67:0/2107746034 shutdown_connections 2024-09-21T06:33:42.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.926+0000 7f7278c41640 1 --2- 172.21.15.67:0/2107746034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72741036e0 0x7f727419ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:42.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.926+0000 7f7278c41640 1 --2- 172.21.15.67:0/2107746034 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f725c0b6590 0x7f725c0b8aa0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:42.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.926+0000 7f7278c41640 1 --2- 172.21.15.67:0/2107746034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7274104080 0x7f72741a1230 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:42.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.926+0000 7f7278c41640 1 --2- 172.21.15.67:0/2107746034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7274102650 0x7f727406c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:42.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.926+0000 7f7278c41640 1 -- 172.21.15.67:0/2107746034 >> 172.21.15.67:0/2107746034 conn(0x7f72740fdfe0 msgr2=0x7f7274106a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:42.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.926+0000 7f7278c41640 1 -- 172.21.15.67:0/2107746034 shutdown_connections 2024-09-21T06:33:42.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:42.926+0000 7f7278c41640 1 -- 172.21.15.67:0/2107746034 wait complete. 2024-09-21T06:33:43.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:42 smithi203 ceph-mon[33735]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:33:43.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:42 smithi203 ceph-mon[33735]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T06:33:43.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:42 smithi067 ceph-mon[31232]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:33:43.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:42 smithi067 ceph-mon[31232]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T06:33:43.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:42 smithi100 ceph-mon[33401]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:33:43.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:42 smithi100 ceph-mon[33401]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T06:33:43.675 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":70,"num_osds":12,"num_up_osds":12,"osd_up_since":1726900420,"num_in_osds":12,"osd_in_since":1726900397,"num_remapped_pgs":0} 2024-09-21T06:33:43.676 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:33:43.937 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:33:44.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:43 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2107746034' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:33:44.243 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:43 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2107746034' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:33:44.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:43 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2107746034' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:33:44.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.474+0000 7f084480a640 1 -- 172.21.15.67:0/4164442831 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0840108d70 msgr2=0x7f084010f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:44.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.474+0000 7f084480a640 1 --2- 172.21.15.67:0/4164442831 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0840108d70 0x7f084010f5a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0838009900 tx=0x7f083802a410 comp rx=0 tx=0).stop 2024-09-21T06:33:44.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.474+0000 7f084480a640 1 -- 172.21.15.67:0/4164442831 shutdown_connections 2024-09-21T06:33:44.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.474+0000 7f084480a640 1 --2- 172.21.15.67:0/4164442831 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0840108d70 0x7f084010f5a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:44.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.474+0000 7f084480a640 1 --2- 172.21.15.67:0/4164442831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0840108420 0x7f0840108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:44.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.474+0000 7f084480a640 1 --2- 172.21.15.67:0/4164442831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0840102ae0 0x7f0840102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:44.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.474+0000 7f084480a640 1 -- 172.21.15.67:0/4164442831 >> 172.21.15.67:0/4164442831 conn(0x7f0840079770 msgr2=0x7f0840079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:44.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.474+0000 7f084480a640 1 -- 172.21.15.67:0/4164442831 shutdown_connections 2024-09-21T06:33:44.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.474+0000 7f084480a640 1 -- 172.21.15.67:0/4164442831 wait complete. 2024-09-21T06:33:44.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.475+0000 7f084480a640 1 Processor -- start 2024-09-21T06:33:44.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.475+0000 7f084480a640 1 -- start start 2024-09-21T06:33:44.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.475+0000 7f084480a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0840102ae0 0x7f0840110080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:44.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.475+0000 7f084480a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0840108420 0x7f08401105c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:44.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.475+0000 7f084480a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0840108d70 0x7f0840110b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:44.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.475+0000 7f084480a640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0840113a20 con 0x7f0840108d70 2024-09-21T06:33:44.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.475+0000 7f084480a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f08401138a0 con 0x7f0840102ae0 2024-09-21T06:33:44.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.475+0000 7f084480a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0840113ba0 con 0x7f0840108420 2024-09-21T06:33:44.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.475+0000 7f083ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0840108d70 0x7f0840110b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:44.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.475+0000 7f083ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0840108d70 0x7f0840110b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59900/0 (socket says 172.21.15.67:59900) 2024-09-21T06:33:44.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.475+0000 7f083ed9d640 1 -- 172.21.15.67:0/863827007 learned_addr learned my addr 172.21.15.67:0/863827007 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:33:44.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.475+0000 7f083e59c640 1 --2- 172.21.15.67:0/863827007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0840102ae0 0x7f0840110080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:44.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.475+0000 7f083dd9b640 1 --2- 172.21.15.67:0/863827007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0840108420 0x7f08401105c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:44.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.476+0000 7f083ed9d640 1 -- 172.21.15.67:0/863827007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0840108420 msgr2=0x7f08401105c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:44.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.476+0000 7f083ed9d640 1 --2- 172.21.15.67:0/863827007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0840108420 0x7f08401105c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:44.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.476+0000 7f083ed9d640 1 -- 172.21.15.67:0/863827007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0840102ae0 msgr2=0x7f0840110080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:44.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.476+0000 7f083ed9d640 1 --2- 172.21.15.67:0/863827007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0840102ae0 0x7f0840110080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:44.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.476+0000 7f083ed9d640 1 -- 172.21.15.67:0/863827007 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08401a86c0 con 0x7f0840108d70 2024-09-21T06:33:44.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.476+0000 7f083dd9b640 1 --2- 172.21.15.67:0/863827007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0840108420 0x7f08401105c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:33:44.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.476+0000 7f083ed9d640 1 --2- 172.21.15.67:0/863827007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0840108d70 0x7f0840110b00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0838005ab0 tx=0x7f08380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:44.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.476+0000 7f082f7fe640 1 -- 172.21.15.67:0/863827007 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083803f030 con 0x7f0840108d70 2024-09-21T06:33:44.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.476+0000 7f082f7fe640 1 -- 172.21.15.67:0/863827007 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0838003c90 con 0x7f0840108d70 2024-09-21T06:33:44.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.476+0000 7f084480a640 1 -- 172.21.15.67:0/863827007 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08401a8950 con 0x7f0840108d70 2024-09-21T06:33:44.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.476+0000 7f082f7fe640 1 -- 172.21.15.67:0/863827007 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838035070 con 0x7f0840108d70 2024-09-21T06:33:44.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.476+0000 7f084480a640 1 -- 172.21.15.67:0/863827007 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08401a8e30 con 0x7f0840108d70 2024-09-21T06:33:44.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.477+0000 7f084480a640 1 -- 172.21.15.67:0/863827007 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0840103d60 con 0x7f0840108d70 2024-09-21T06:33:44.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.479+0000 7f082f7fe640 1 -- 172.21.15.67:0/863827007 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f0838042030 con 0x7f0840108d70 2024-09-21T06:33:44.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.479+0000 7f082f7fe640 1 --2- 172.21.15.67:0/863827007 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f08280b6510 0x7f08280b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:44.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.480+0000 7f083e59c640 1 --2- 172.21.15.67:0/863827007 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f08280b6510 0x7f08280b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:44.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.480+0000 7f082f7fe640 1 -- 172.21.15.67:0/863827007 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f0838107870 con 0x7f0840108d70 2024-09-21T06:33:44.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.480+0000 7f083e59c640 1 --2- 172.21.15.67:0/863827007 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f08280b6510 0x7f08280b8a20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0840111ad0 tx=0x7f083000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:44.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.483+0000 7f082f7fe640 1 -- 172.21.15.67:0/863827007 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f08380ce5a0 con 0x7f0840108d70 2024-09-21T06:33:44.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.672+0000 7f084480a640 1 -- 172.21.15.67:0/863827007 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0840102ef0 con 0x7f0840108d70 2024-09-21T06:33:44.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.673+0000 7f082f7fe640 1 -- 172.21.15.67:0/863827007 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+18494 (secure 0 0 0) 0x7f08380d2980 con 0x7f0840108d70 2024-09-21T06:33:44.676 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:33:44.676 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":70,"fsid":"2a791768-77e2-11ef-baf5-efdc52797490","created":"2024-09-21T06:26:15.222283+0000","modified":"2024-09-21T06:33:41.890082+0000","last_up_change":"2024-09-21T06:33:40.886310+0000","last_in_change":"2024-09-21T06:33:17.339486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:29:50.212825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"18778364-82ed-41c0-92da-ce44cdad7358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6803","nonce":2310759946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6805","nonce":2310759946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6809","nonce":2310759946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6807","nonce":2310759946}]},"public_addr":"172.21.15.67:6803/2310759946","cluster_addr":"172.21.15.67:6805/2310759946","heartbeat_back_addr":"172.21.15.67:6809/2310759946","heartbeat_front_addr":"172.21.15.67:6807/2310759946","state":["exists","up"]},{"osd":1,"uuid":"84e8d0c2-1c45-4673-ad49-f61df667d7f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6801","nonce":161807364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6803","nonce":161807364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6807","nonce":161807364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6805","nonce":161807364}]},"public_addr":"172.21.15.100:6801/161807364","cluster_addr":"172.21.15.100:6803/161807364","heartbeat_back_addr":"172.21.15.100:6807/161807364","heartbeat_front_addr":"172.21.15.100:6805/161807364","state":["exists","up"]},{"osd":2,"uuid":"e9a4d919-a779-4a1b-9b5b-d03a65f28ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6801","nonce":2754787169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6803","nonce":2754787169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6807","nonce":2754787169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6805","nonce":2754787169}]},"public_addr":"172.21.15.203:6801/2754787169","cluster_addr":"172.21.15.203:6803/2754787169","heartbeat_back_addr":"172.21.15.203:6807/2754787169","heartbeat_front_addr":"172.21.15.203:6805/2754787169","state":["exists","up"]},{"osd":3,"uuid":"3f7d9abd-587c-47ad-bf43-11f7dcdf71c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6811","nonce":4007517951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6813","nonce":4007517951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6817","nonce":4007517951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6815","nonce":4007517951}]},"public_addr":"172.21.15.67:6811/4007517951","cluster_addr":"172.21.15.67:6813/4007517951","heartbeat_back_addr":"172.21.15.67:6817/4007517951","heartbeat_front_addr":"172.21.15.67:6815/4007517951","state":["exists","up"]},{"osd":4,"uuid":"94f3a075-6d8a-4907-8418-a8513b35779a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6809","nonce":3773646501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6811","nonce":3773646501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6815","nonce":3773646501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6813","nonce":3773646501}]},"public_addr":"172.21.15.100:6809/3773646501","cluster_addr":"172.21.15.100:6811/3773646501","heartbeat_back_addr":"172.21.15.100:6815/3773646501","heartbeat_front_addr":"172.21.15.100:6813/3773646501","state":["exists","up"]},{"osd":5,"uuid":"5d842edf-06ad-49ae-ac71-863e5657270c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6809","nonce":2096171166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6811","nonce":2096171166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6815","nonce":2096171166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6813","nonce":2096171166}]},"public_addr":"172.21.15.203:6809/2096171166","cluster_addr":"172.21.15.203:6811/2096171166","heartbeat_back_addr":"172.21.15.203:6815/2096171166","heartbeat_front_addr":"172.21.15.203:6813/2096171166","state":["exists","up"]},{"osd":6,"uuid":"85b9beed-c93d-48ba-bc7a-c4cd816fbdff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6819","nonce":3229936280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6821","nonce":3229936280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6825","nonce":3229936280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6823","nonce":3229936280}]},"public_addr":"172.21.15.67:6819/3229936280","cluster_addr":"172.21.15.67:6821/3229936280","heartbeat_back_addr":"172.21.15.67:6825/3229936280","heartbeat_front_addr":"172.21.15.67:6823/3229936280","state":["exists","up"]},{"osd":7,"uuid":"86de30c2-b8ee-4c9d-9c17-8a56517934ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6817","nonce":1622917909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6819","nonce":1622917909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6823","nonce":1622917909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6821","nonce":1622917909}]},"public_addr":"172.21.15.100:6817/1622917909","cluster_addr":"172.21.15.100:6819/1622917909","heartbeat_back_addr":"172.21.15.100:6823/1622917909","heartbeat_front_addr":"172.21.15.100:6821/1622917909","state":["exists","up"]},{"osd":8,"uuid":"0e96cdad-a135-4115-9e2d-9a28306700e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6817","nonce":3452445943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6819","nonce":3452445943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6823","nonce":3452445943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6821","nonce":3452445943}]},"public_addr":"172.21.15.203:6817/3452445943","cluster_addr":"172.21.15.203:6819/3452445943","heartbeat_back_addr":"172.21.15.203:6823/3452445943","heartbeat_front_addr":"172.21.15.203:6821/3452445943","state":["exists","up"]},{"osd":9,"uuid":"95704ed6-6286-4094-9c6d-23919c7005ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6827","nonce":2998307533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6829","nonce":2998307533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6833","nonce":2998307533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6831","nonce":2998307533}]},"public_addr":"172.21.15.67:6827/2998307533","cluster_addr":"172.21.15.67:6829/2998307533","heartbeat_back_addr":"172.21.15.67:6833/2998307533","heartbeat_front_addr":"172.21.15.67:6831/2998307533","state":["exists","up"]},{"osd":10,"uuid":"bd2a7d33-b935-46c8-95c3-d76f28242ffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6825","nonce":2819057142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6827","nonce":2819057142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6831","nonce":2819057142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6829","nonce":2819057142}]},"public_addr":"172.21.15.100:6825/2819057142","cluster_addr":"172.21.15.100:6827/2819057142","heartbeat_back_addr":"172.21.15.100:6831/2819057142","heartbeat_front_addr":"172.21.15.100:6829/2819057142","state":["exists","up"]},{"osd":11,"uuid":"ea591507-10e2-4c08-8d29-d38ba9be96ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6825","nonce":566502600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6827","nonce":566502600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6831","nonce":566502600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6829","nonce":566502600}]},"public_addr":"172.21.15.203:6825/566502600","cluster_addr":"172.21.15.203:6827/566502600","heartbeat_back_addr":"172.21.15.203:6831/566502600","heartbeat_front_addr":"172.21.15.203:6829/566502600","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:28:58.945733+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:29:23.597118+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:29:48.303563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:30:14.139139+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:30:39.021508+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:04.981241+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:30.853841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:56.452967+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:32:22.239036+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:32:47.998803+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:33:13.850759+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:33:39.282150+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6800/1181476774":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/3100321138":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/2309364298":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/3458717076":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/4110206171":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6801/1181476774":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/2910038982":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6801/2622729103":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6800/2622729103":"2024-09-22T06:26:37.041377+0000","172.21.15.67:0/3346784528":"2024-09-22T06:26:37.041377+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:33:44.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.675+0000 7f084480a640 1 -- 172.21.15.67:0/863827007 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f08280b6510 msgr2=0x7f08280b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:44.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.675+0000 7f084480a640 1 --2- 172.21.15.67:0/863827007 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f08280b6510 0x7f08280b8a20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0840111ad0 tx=0x7f083000a2d0 comp rx=0 tx=0).stop 2024-09-21T06:33:44.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.675+0000 7f084480a640 1 -- 172.21.15.67:0/863827007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0840108d70 msgr2=0x7f0840110b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:44.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.675+0000 7f084480a640 1 --2- 172.21.15.67:0/863827007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0840108d70 0x7f0840110b00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0838005ab0 tx=0x7f08380042f0 comp rx=0 tx=0).stop 2024-09-21T06:33:44.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.675+0000 7f084480a640 1 -- 172.21.15.67:0/863827007 shutdown_connections 2024-09-21T06:33:44.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.675+0000 7f084480a640 1 --2- 172.21.15.67:0/863827007 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f08280b6510 0x7f08280b8a20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:44.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.675+0000 7f084480a640 1 --2- 172.21.15.67:0/863827007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0840108d70 0x7f0840110b00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:44.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.675+0000 7f084480a640 1 --2- 172.21.15.67:0/863827007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0840108420 0x7f08401105c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:44.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.675+0000 7f084480a640 1 --2- 172.21.15.67:0/863827007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0840102ae0 0x7f0840110080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:44.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.676+0000 7f084480a640 1 -- 172.21.15.67:0/863827007 >> 172.21.15.67:0/863827007 conn(0x7f0840079770 msgr2=0x7f0840106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:44.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.676+0000 7f084480a640 1 -- 172.21.15.67:0/863827007 shutdown_connections 2024-09-21T06:33:44.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:44.676+0000 7f084480a640 1 -- 172.21.15.67:0/863827007 wait complete. 2024-09-21T06:33:44.998 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:44 smithi067 ceph-mon[31232]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:33:44.998 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:44 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/863827007' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:33:45.234 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:44 smithi203 ceph-mon[33735]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:33:45.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:44 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/863827007' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:33:45.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:44 smithi100 ceph-mon[33401]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:33:45.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:44 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/863827007' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:33:45.411 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T06:29:50.212825+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T06:33:45.412 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd pool get .mgr pg_num 2024-09-21T06:33:45.666 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:33:46.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.154+0000 7fb8d5402640 1 -- 172.21.15.67:0/3425891849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8d010de10 msgr2=0x7fb8d0110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:46.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.154+0000 7fb8d5402640 1 --2- 172.21.15.67:0/3425891849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8d010de10 0x7fb8d0110300 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0009ca0 tx=0x7fb8c002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:33:46.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.154+0000 7fb8d5402640 1 -- 172.21.15.67:0/3425891849 shutdown_connections 2024-09-21T06:33:46.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.154+0000 7fb8d5402640 1 --2- 172.21.15.67:0/3425891849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8d010de10 0x7fb8d0110300 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:46.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.154+0000 7fb8d5402640 1 --2- 172.21.15.67:0/3425891849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8d0105300 0x7fb8d010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:46.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.154+0000 7fb8d5402640 1 --2- 172.21.15.67:0/3425891849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8d01049b0 0x7fb8d0104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:46.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.154+0000 7fb8d5402640 1 -- 172.21.15.67:0/3425891849 >> 172.21.15.67:0/3425891849 conn(0x7fb8d0079730 msgr2=0x7fb8d0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:46.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.154+0000 7fb8d5402640 1 -- 172.21.15.67:0/3425891849 shutdown_connections 2024-09-21T06:33:46.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.154+0000 7fb8d5402640 1 -- 172.21.15.67:0/3425891849 wait complete. 2024-09-21T06:33:46.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.154+0000 7fb8d5402640 1 Processor -- start 2024-09-21T06:33:46.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.155+0000 7fb8d5402640 1 -- start start 2024-09-21T06:33:46.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.155+0000 7fb8d5402640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8d01049b0 0x7fb8d019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:46.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.155+0000 7fb8d5402640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8d0105300 0x7fb8d019a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:46.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.155+0000 7fb8d5402640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8d010de10 0x7fb8d019fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:46.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.155+0000 7fb8d5402640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d0113970 con 0x7fb8d0105300 2024-09-21T06:33:46.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.155+0000 7fb8d5402640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d01137f0 con 0x7fb8d01049b0 2024-09-21T06:33:46.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.155+0000 7fb8d5402640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d0113af0 con 0x7fb8d010de10 2024-09-21T06:33:46.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.155+0000 7fb8ce7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8d0105300 0x7fb8d019a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:46.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.155+0000 7fb8ce7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8d0105300 0x7fb8d019a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59924/0 (socket says 172.21.15.67:59924) 2024-09-21T06:33:46.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.155+0000 7fb8ce7fc640 1 -- 172.21.15.67:0/2366885943 learned_addr learned my addr 172.21.15.67:0/2366885943 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:33:46.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.155+0000 7fb8ceffd640 1 --2- 172.21.15.67:0/2366885943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8d01049b0 0x7fb8d019a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:46.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.155+0000 7fb8cf7fe640 1 --2- 172.21.15.67:0/2366885943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8d010de10 0x7fb8d019fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:46.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.155+0000 7fb8ce7fc640 1 -- 172.21.15.67:0/2366885943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8d010de10 msgr2=0x7fb8d019fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:46.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.155+0000 7fb8ce7fc640 1 --2- 172.21.15.67:0/2366885943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8d010de10 0x7fb8d019fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:46.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.155+0000 7fb8ce7fc640 1 -- 172.21.15.67:0/2366885943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8d01049b0 msgr2=0x7fb8d019a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:46.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.155+0000 7fb8ce7fc640 1 --2- 172.21.15.67:0/2366885943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8d01049b0 0x7fb8d019a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:46.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.155+0000 7fb8ce7fc640 1 -- 172.21.15.67:0/2366885943 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d01a0210 con 0x7fb8d0105300 2024-09-21T06:33:46.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.156+0000 7fb8ce7fc640 1 --2- 172.21.15.67:0/2366885943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8d0105300 0x7fb8d019a900 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800c970 tx=0x7fb8b800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:46.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.156+0000 7fb8c7fff640 1 -- 172.21.15.67:0/2366885943 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8007bb0 con 0x7fb8d0105300 2024-09-21T06:33:46.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.156+0000 7fb8d5402640 1 -- 172.21.15.67:0/2366885943 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d01a0500 con 0x7fb8d0105300 2024-09-21T06:33:46.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.156+0000 7fb8c7fff640 1 -- 172.21.15.67:0/2366885943 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8b8007d50 con 0x7fb8d0105300 2024-09-21T06:33:46.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.156+0000 7fb8c7fff640 1 -- 172.21.15.67:0/2366885943 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8004d40 con 0x7fb8d0105300 2024-09-21T06:33:46.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.156+0000 7fb8d5402640 1 -- 172.21.15.67:0/2366885943 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d01a6d20 con 0x7fb8d0105300 2024-09-21T06:33:46.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.157+0000 7fb8d5402640 1 -- 172.21.15.67:0/2366885943 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8d0104dc0 con 0x7fb8d0105300 2024-09-21T06:33:46.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.159+0000 7fb8c7fff640 1 -- 172.21.15.67:0/2366885943 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fb8b8004fe0 con 0x7fb8d0105300 2024-09-21T06:33:46.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.159+0000 7fb8c7fff640 1 --2- 172.21.15.67:0/2366885943 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb8bc0b6540 0x7fb8bc0b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:46.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.160+0000 7fb8ceffd640 1 --2- 172.21.15.67:0/2366885943 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb8bc0b6540 0x7fb8bc0b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:46.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.160+0000 7fb8c7fff640 1 -- 172.21.15.67:0/2366885943 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fb8b80e6e30 con 0x7fb8d0105300 2024-09-21T06:33:46.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.160+0000 7fb8ceffd640 1 --2- 172.21.15.67:0/2366885943 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb8bc0b6540 0x7fb8bc0b8a50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb8d006bc60 tx=0x7fb8c8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:46.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.163+0000 7fb8c7fff640 1 -- 172.21.15.67:0/2366885943 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb8b8014030 con 0x7fb8d0105300 2024-09-21T06:33:46.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.351+0000 7fb8d5402640 1 -- 172.21.15.67:0/2366885943 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb8d0108980 con 0x7fb8d0105300 2024-09-21T06:33:46.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.351+0000 7fb8c7fff640 1 -- 172.21.15.67:0/2366885943 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) ==== 93+0+10 (secure 0 0 0) 0x7fb8b80b1f40 con 0x7fb8d0105300 2024-09-21T06:33:46.354 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-09-21T06:33:46.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.354+0000 7fb8d5402640 1 -- 172.21.15.67:0/2366885943 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb8bc0b6540 msgr2=0x7fb8bc0b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:46.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.354+0000 7fb8d5402640 1 --2- 172.21.15.67:0/2366885943 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb8bc0b6540 0x7fb8bc0b8a50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb8d006bc60 tx=0x7fb8c8005f90 comp rx=0 tx=0).stop 2024-09-21T06:33:46.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.354+0000 7fb8d5402640 1 -- 172.21.15.67:0/2366885943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8d0105300 msgr2=0x7fb8d019a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:46.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.354+0000 7fb8d5402640 1 --2- 172.21.15.67:0/2366885943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8d0105300 0x7fb8d019a900 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800c970 tx=0x7fb8b800ce30 comp rx=0 tx=0).stop 2024-09-21T06:33:46.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.354+0000 7fb8d5402640 1 -- 172.21.15.67:0/2366885943 shutdown_connections 2024-09-21T06:33:46.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.354+0000 7fb8d5402640 1 --2- 172.21.15.67:0/2366885943 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb8bc0b6540 0x7fb8bc0b8a50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:46.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.354+0000 7fb8d5402640 1 --2- 172.21.15.67:0/2366885943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8d010de10 0x7fb8d019fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:46.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.354+0000 7fb8d5402640 1 --2- 172.21.15.67:0/2366885943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8d0105300 0x7fb8d019a900 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:46.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.354+0000 7fb8d5402640 1 --2- 172.21.15.67:0/2366885943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8d01049b0 0x7fb8d019a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:46.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.354+0000 7fb8d5402640 1 -- 172.21.15.67:0/2366885943 >> 172.21.15.67:0/2366885943 conn(0x7fb8d0079730 msgr2=0x7fb8d0100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:46.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.354+0000 7fb8d5402640 1 -- 172.21.15.67:0/2366885943 shutdown_connections 2024-09-21T06:33:46.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:46.354+0000 7fb8d5402640 1 -- 172.21.15.67:0/2366885943 wait complete. 2024-09-21T06:33:47.061 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:46 smithi067 ceph-mon[31232]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:33:47.061 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:46 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2366885943' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:33:47.091 INFO:tasks.cephadm:Adding mds.a on smithi067 2024-09-21T06:33:47.091 INFO:tasks.cephadm:Adding mds.d on smithi067 2024-09-21T06:33:47.091 INFO:tasks.cephadm:Adding mds.g on smithi067 2024-09-21T06:33:47.091 INFO:tasks.cephadm:Adding mds.j on smithi067 2024-09-21T06:33:47.091 INFO:tasks.cephadm:Adding mds.b on smithi100 2024-09-21T06:33:47.091 INFO:tasks.cephadm:Adding mds.e on smithi100 2024-09-21T06:33:47.092 INFO:tasks.cephadm:Adding mds.h on smithi100 2024-09-21T06:33:47.092 INFO:tasks.cephadm:Adding mds.k on smithi100 2024-09-21T06:33:47.092 INFO:tasks.cephadm:Adding mds.c on smithi203 2024-09-21T06:33:47.092 INFO:tasks.cephadm:Adding mds.f on smithi203 2024-09-21T06:33:47.092 INFO:tasks.cephadm:Adding mds.i on smithi203 2024-09-21T06:33:47.092 INFO:tasks.cephadm:Adding mds.l on smithi203 2024-09-21T06:33:47.092 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph orch apply mds all '12;smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l' 2024-09-21T06:33:47.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:46 smithi203 ceph-mon[33735]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:33:47.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:46 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2366885943' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:33:47.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:46 smithi100 ceph-mon[33401]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:33:47.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:46 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2366885943' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:33:47.354 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:33:47.911 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.904+0000 7f5b7422a640 1 -- 172.21.15.203:0/3944311821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b6c103750 msgr2=0x7f5b6c103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:47.912 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.904+0000 7f5b7422a640 1 --2- 172.21.15.203:0/3944311821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b6c103750 0x7f5b6c103bc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009900 tx=0x7f5b6802a480 comp rx=0 tx=0).stop 2024-09-21T06:33:47.912 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.904+0000 7f5b7422a640 1 -- 172.21.15.203:0/3944311821 shutdown_connections 2024-09-21T06:33:47.912 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.904+0000 7f5b7422a640 1 --2- 172.21.15.203:0/3944311821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b6c1047e0 0x7f5b6c104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:47.912 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.904+0000 7f5b7422a640 1 --2- 172.21.15.203:0/3944311821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b6c105180 0x7f5b6c10bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:47.912 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.904+0000 7f5b7422a640 1 --2- 172.21.15.203:0/3944311821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b6c103750 0x7f5b6c103bc0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:47.912 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.904+0000 7f5b7422a640 1 -- 172.21.15.203:0/3944311821 >> 172.21.15.203:0/3944311821 conn(0x7f5b6c0ff0e0 msgr2=0x7f5b6c101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:47.912 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.904+0000 7f5b7422a640 1 -- 172.21.15.203:0/3944311821 shutdown_connections 2024-09-21T06:33:47.912 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.904+0000 7f5b7422a640 1 -- 172.21.15.203:0/3944311821 wait complete. 2024-09-21T06:33:47.912 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.905+0000 7f5b7422a640 1 Processor -- start 2024-09-21T06:33:47.912 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.905+0000 7f5b7422a640 1 -- start start 2024-09-21T06:33:47.912 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.905+0000 7f5b7422a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b6c103750 0x7f5b6c199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:47.912 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.905+0000 7f5b7422a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b6c1047e0 0x7f5b6c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:47.913 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.905+0000 7f5b7422a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b6c105180 0x7f5b6c19b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:47.913 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.905+0000 7f5b7422a640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c10f2a0 con 0x7f5b6c1047e0 2024-09-21T06:33:47.913 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.905+0000 7f5b7422a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c10f120 con 0x7f5b6c105180 2024-09-21T06:33:47.913 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.905+0000 7f5b7422a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c10f420 con 0x7f5b6c103750 2024-09-21T06:33:47.913 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.905+0000 7f5b71fc6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b6c103750 0x7f5b6c199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:47.913 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.905+0000 7f5b71fc6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b6c103750 0x7f5b6c199f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:41018/0 (socket says 172.21.15.203:41018) 2024-09-21T06:33:47.913 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.905+0000 7f5b71fc6640 1 -- 172.21.15.203:0/22126372 learned_addr learned my addr 172.21.15.203:0/22126372 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-09-21T06:33:47.913 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.905+0000 7f5b717c5640 1 --2- 172.21.15.203:0/22126372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b6c1047e0 0x7f5b6c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:47.913 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.905+0000 7f5b727c7640 1 --2- 172.21.15.203:0/22126372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b6c105180 0x7f5b6c19b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:47.913 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.906+0000 7f5b71fc6640 1 -- 172.21.15.203:0/22126372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b6c105180 msgr2=0x7f5b6c19b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:47.913 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.906+0000 7f5b71fc6640 1 --2- 172.21.15.203:0/22126372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b6c105180 0x7f5b6c19b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:47.913 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.906+0000 7f5b71fc6640 1 -- 172.21.15.203:0/22126372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b6c1047e0 msgr2=0x7f5b6c19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:47.913 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.906+0000 7f5b71fc6640 1 --2- 172.21.15.203:0/22126372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b6c1047e0 0x7f5b6c19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:47.913 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.906+0000 7f5b71fc6640 1 -- 172.21.15.203:0/22126372 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6c19b9b0 con 0x7f5b6c103750 2024-09-21T06:33:47.914 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.906+0000 7f5b71fc6640 1 --2- 172.21.15.203:0/22126372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b6c103750 0x7f5b6c199f80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009230 tx=0x7f5b680042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:47.914 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.909+0000 7f5b5effd640 1 -- 172.21.15.203:0/22126372 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6803f030 con 0x7f5b6c103750 2024-09-21T06:33:47.914 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.909+0000 7f5b5effd640 1 -- 172.21.15.203:0/22126372 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5b68003c90 con 0x7f5b6c103750 2024-09-21T06:33:47.914 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.909+0000 7f5b7422a640 1 -- 172.21.15.203:0/22126372 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b6c19bc40 con 0x7f5b6c103750 2024-09-21T06:33:47.914 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.909+0000 7f5b5effd640 1 -- 172.21.15.203:0/22126372 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68035070 con 0x7f5b6c103750 2024-09-21T06:33:47.914 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.909+0000 7f5b7422a640 1 -- 172.21.15.203:0/22126372 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b6c075670 con 0x7f5b6c103750 2024-09-21T06:33:47.914 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.910+0000 7f5b7422a640 1 -- 172.21.15.203:0/22126372 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b6c10a930 con 0x7f5b6c103750 2024-09-21T06:33:47.915 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.911+0000 7f5b5effd640 1 -- 172.21.15.203:0/22126372 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f5b68042030 con 0x7f5b6c103750 2024-09-21T06:33:47.915 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.912+0000 7f5b5effd640 1 --2- 172.21.15.203:0/22126372 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5b580b6510 0x7f5b580b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:47.915 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.912+0000 7f5b5effd640 1 -- 172.21.15.203:0/22126372 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f5b681073e0 con 0x7f5b6c103750 2024-09-21T06:33:47.915 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.912+0000 7f5b717c5640 1 --2- 172.21.15.203:0/22126372 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5b580b6510 0x7f5b580b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:47.915 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.913+0000 7f5b717c5640 1 --2- 172.21.15.203:0/22126372 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5b580b6510 0x7f5b580b8a20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5b6000bbe0 tx=0x7f5b6000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:47.918 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:47.916+0000 7f5b5effd640 1 -- 172.21.15.203:0/22126372 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5b680d24f0 con 0x7f5b6c103750 2024-09-21T06:33:48.114 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:48.113+0000 7f5b7422a640 1 -- 172.21.15.203:0/22126372 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}) -- 0x7f5b6c063900 con 0x7f5b580b6510 2024-09-21T06:33:48.123 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:48.121+0000 7f5b5effd640 1 -- 172.21.15.203:0/22126372 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f5b6c063900 con 0x7f5b580b6510 2024-09-21T06:33:48.123 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled mds.all update... 2024-09-21T06:33:48.125 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:48.123+0000 7f5b7422a640 1 -- 172.21.15.203:0/22126372 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5b580b6510 msgr2=0x7f5b580b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:48.125 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:48.123+0000 7f5b7422a640 1 --2- 172.21.15.203:0/22126372 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5b580b6510 0x7f5b580b8a20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5b6000bbe0 tx=0x7f5b6000b390 comp rx=0 tx=0).stop 2024-09-21T06:33:48.125 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:48.123+0000 7f5b7422a640 1 -- 172.21.15.203:0/22126372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b6c103750 msgr2=0x7f5b6c199f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:48.126 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:48.123+0000 7f5b7422a640 1 --2- 172.21.15.203:0/22126372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b6c103750 0x7f5b6c199f80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009230 tx=0x7f5b680042f0 comp rx=0 tx=0).stop 2024-09-21T06:33:48.126 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:48.124+0000 7f5b7422a640 1 -- 172.21.15.203:0/22126372 shutdown_connections 2024-09-21T06:33:48.126 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:48.124+0000 7f5b7422a640 1 --2- 172.21.15.203:0/22126372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b6c105180 0x7f5b6c19b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:48.126 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:48.124+0000 7f5b7422a640 1 --2- 172.21.15.203:0/22126372 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5b580b6510 0x7f5b580b8a20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:48.126 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:48.124+0000 7f5b7422a640 1 --2- 172.21.15.203:0/22126372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b6c1047e0 0x7f5b6c19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:48.126 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:48.124+0000 7f5b7422a640 1 --2- 172.21.15.203:0/22126372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b6c103750 0x7f5b6c199f80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:48.126 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:48.124+0000 7f5b7422a640 1 -- 172.21.15.203:0/22126372 >> 172.21.15.203:0/22126372 conn(0x7f5b6c0ff0e0 msgr2=0x7f5b6c107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:48.126 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:48.124+0000 7f5b7422a640 1 -- 172.21.15.203:0/22126372 shutdown_connections 2024-09-21T06:33:48.126 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T06:33:48.124+0000 7f5b7422a640 1 -- 172.21.15.203:0/22126372 wait complete. 2024-09-21T06:33:48.802 DEBUG:teuthology.orchestra.run.smithi067:mds.a> sudo journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.a.service 2024-09-21T06:33:48.804 DEBUG:teuthology.orchestra.run.smithi067:mds.d> sudo journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.d.service 2024-09-21T06:33:48.847 DEBUG:teuthology.orchestra.run.smithi067:mds.g> sudo journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.g.service 2024-09-21T06:33:48.890 DEBUG:teuthology.orchestra.run.smithi067:mds.j> sudo journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.j.service 2024-09-21T06:33:48.933 DEBUG:teuthology.orchestra.run.smithi100:mds.b> sudo journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.b.service 2024-09-21T06:33:48.936 DEBUG:teuthology.orchestra.run.smithi100:mds.e> sudo journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.e.service 2024-09-21T06:33:48.979 DEBUG:teuthology.orchestra.run.smithi100:mds.h> sudo journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.h.service 2024-09-21T06:33:49.023 DEBUG:teuthology.orchestra.run.smithi100:mds.k> sudo journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.k.service 2024-09-21T06:33:49.065 DEBUG:teuthology.orchestra.run.smithi203:mds.c> sudo journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.c.service 2024-09-21T06:33:49.068 DEBUG:teuthology.orchestra.run.smithi203:mds.f> sudo journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.f.service 2024-09-21T06:33:49.110 DEBUG:teuthology.orchestra.run.smithi203:mds.i> sudo journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.i.service 2024-09-21T06:33:49.154 DEBUG:teuthology.orchestra.run.smithi203:mds.l> sudo journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.l.service 2024-09-21T06:33:49.197 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-21T06:33:49.197 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:33:49.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:49 smithi203 ceph-mon[33735]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:33:49.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:49 smithi203 ceph-mon[33735]: from='client.24565 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:33:49.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:49 smithi203 ceph-mon[33735]: Saving service mds.all spec with placement smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2024-09-21T06:33:49.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:49 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:49.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:49 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:33:49.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:49 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:49.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:49 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:33:49.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:49 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:49.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:49 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:33:49.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:49 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:33:49.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:49 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:49.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:49 smithi203 ceph-mon[33735]: Deploying daemon mds.l on smithi203 2024-09-21T06:33:49.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:49 smithi100 ceph-mon[33401]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:33:49.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:49 smithi100 ceph-mon[33401]: from='client.24565 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:33:49.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:49 smithi100 ceph-mon[33401]: Saving service mds.all spec with placement smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2024-09-21T06:33:49.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:49 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:49.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:49 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:33:49.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:49 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:49.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:49 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:33:49.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:49 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:49.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:49 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:33:49.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:49 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:33:49.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:49 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:49.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:49 smithi100 ceph-mon[33401]: Deploying daemon mds.l on smithi203 2024-09-21T06:33:49.454 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:33:49.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:49 smithi067 ceph-mon[31232]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:33:49.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:49 smithi067 ceph-mon[31232]: from='client.24565 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:33:49.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:49 smithi067 ceph-mon[31232]: Saving service mds.all spec with placement smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2024-09-21T06:33:49.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:49 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:49.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:49 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:33:49.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:49 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:49.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:49 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:33:49.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:49 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:49.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:49 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:33:49.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:49 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:33:49.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:49 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:49.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:49 smithi067 ceph-mon[31232]: Deploying daemon mds.l on smithi203 2024-09-21T06:33:49.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.927+0000 7f0aa5706640 1 -- 172.21.15.67:0/3106003946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0aa0102ae0 msgr2=0x7f0aa0102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:49.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.927+0000 7f0aa5706640 1 --2- 172.21.15.67:0/3106003946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0aa0102ae0 0x7f0aa0102ef0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0a90009900 tx=0x7f0a9002a410 comp rx=0 tx=0).stop 2024-09-21T06:33:49.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.927+0000 7f0aa5706640 1 -- 172.21.15.67:0/3106003946 shutdown_connections 2024-09-21T06:33:49.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.927+0000 7f0aa5706640 1 --2- 172.21.15.67:0/3106003946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aa0108d20 0x7f0aa010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:49.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.927+0000 7f0aa5706640 1 --2- 172.21.15.67:0/3106003946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aa01083d0 0x7f0aa01087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:49.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.927+0000 7f0aa5706640 1 --2- 172.21.15.67:0/3106003946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0aa0102ae0 0x7f0aa0102ef0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:49.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.927+0000 7f0aa5706640 1 -- 172.21.15.67:0/3106003946 >> 172.21.15.67:0/3106003946 conn(0x7f0aa0079760 msgr2=0x7f0aa0079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:49.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.927+0000 7f0aa5706640 1 -- 172.21.15.67:0/3106003946 shutdown_connections 2024-09-21T06:33:49.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.927+0000 7f0aa5706640 1 -- 172.21.15.67:0/3106003946 wait complete. 2024-09-21T06:33:49.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.927+0000 7f0aa5706640 1 Processor -- start 2024-09-21T06:33:49.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.928+0000 7f0aa5706640 1 -- start start 2024-09-21T06:33:49.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.928+0000 7f0aa5706640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aa0102ae0 0x7f0aa019a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:49.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.928+0000 7f0aa5706640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0aa01083d0 0x7f0aa019a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:49.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.928+0000 7f0aa5706640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aa0108d20 0x7f0aa019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:49.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.928+0000 7f0aa5706640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa01139a0 con 0x7f0aa01083d0 2024-09-21T06:33:49.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.928+0000 7f0aa5706640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa0113820 con 0x7f0aa0102ae0 2024-09-21T06:33:49.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.928+0000 7f0aa5706640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa0113b20 con 0x7f0aa0108d20 2024-09-21T06:33:49.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.928+0000 7f0a9e7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0aa01083d0 0x7f0aa019a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:49.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.928+0000 7f0a9e7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0aa01083d0 0x7f0aa019a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39614/0 (socket says 172.21.15.67:39614) 2024-09-21T06:33:49.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.928+0000 7f0a9e7fc640 1 -- 172.21.15.67:0/47877626 learned_addr learned my addr 172.21.15.67:0/47877626 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:33:49.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.928+0000 7f0a9f7fe640 1 --2- 172.21.15.67:0/47877626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aa0108d20 0x7f0aa019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:49.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.928+0000 7f0a9effd640 1 --2- 172.21.15.67:0/47877626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aa0102ae0 0x7f0aa019a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:49.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.928+0000 7f0a9e7fc640 1 -- 172.21.15.67:0/47877626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aa0108d20 msgr2=0x7f0aa019fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:49.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.928+0000 7f0a9e7fc640 1 --2- 172.21.15.67:0/47877626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aa0108d20 0x7f0aa019fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:49.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.928+0000 7f0a9e7fc640 1 -- 172.21.15.67:0/47877626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aa0102ae0 msgr2=0x7f0aa019a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:49.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.928+0000 7f0a9e7fc640 1 --2- 172.21.15.67:0/47877626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aa0102ae0 0x7f0aa019a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:49.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.928+0000 7f0a9e7fc640 1 -- 172.21.15.67:0/47877626 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa01a0340 con 0x7f0aa01083d0 2024-09-21T06:33:49.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.929+0000 7f0a9e7fc640 1 --2- 172.21.15.67:0/47877626 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0aa01083d0 0x7f0aa019a970 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400b450 tx=0x7f0a9400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:49.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.929+0000 7f0a87fff640 1 -- 172.21.15.67:0/47877626 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a94015020 con 0x7f0aa01083d0 2024-09-21T06:33:49.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.929+0000 7f0aa5706640 1 -- 172.21.15.67:0/47877626 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aa01a0630 con 0x7f0aa01083d0 2024-09-21T06:33:49.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.929+0000 7f0a87fff640 1 -- 172.21.15.67:0/47877626 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0a94011d20 con 0x7f0aa01083d0 2024-09-21T06:33:49.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.929+0000 7f0a87fff640 1 -- 172.21.15.67:0/47877626 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a94010e10 con 0x7f0aa01083d0 2024-09-21T06:33:49.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.929+0000 7f0aa5706640 1 -- 172.21.15.67:0/47877626 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aa01a6e10 con 0x7f0aa01083d0 2024-09-21T06:33:49.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.930+0000 7f0aa5706640 1 -- 172.21.15.67:0/47877626 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aa0103d10 con 0x7f0aa01083d0 2024-09-21T06:33:49.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.931+0000 7f0a87fff640 1 -- 172.21.15.67:0/47877626 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f0a94018020 con 0x7f0aa01083d0 2024-09-21T06:33:49.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.932+0000 7f0a87fff640 1 --2- 172.21.15.67:0/47877626 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f0a880b6510 0x7f0a880b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:49.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.932+0000 7f0a9effd640 1 --2- 172.21.15.67:0/47877626 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f0a880b6510 0x7f0a880b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:49.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.932+0000 7f0a87fff640 1 -- 172.21.15.67:0/47877626 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f0a940e6d80 con 0x7f0aa01083d0 2024-09-21T06:33:49.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.933+0000 7f0a9effd640 1 --2- 172.21.15.67:0/47877626 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f0a880b6510 0x7f0a880b8a20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0a90000c00 tx=0x7f0a90009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:49.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:49.936+0000 7f0a87fff640 1 -- 172.21.15.67:0/47877626 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0a9401f020 con 0x7f0aa01083d0 2024-09-21T06:33:50.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:50.164+0000 7f0aa5706640 1 -- 172.21.15.67:0/47877626 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0aa0102ef0 con 0x7f0aa01083d0 2024-09-21T06:33:50.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:50.164+0000 7f0a87fff640 1 -- 172.21.15.67:0/47877626 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f0aa0102ef0 con 0x7f0aa01083d0 2024-09-21T06:33:50.167 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 1 2024-09-21T06:33:50.168 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:33:50.168 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":1,"btime":"2024-09-21T06:26:15:222052+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-21T06:33:50.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:50.168+0000 7f0aa5706640 1 -- 172.21.15.67:0/47877626 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f0a880b6510 msgr2=0x7f0a880b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:50.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:50.168+0000 7f0aa5706640 1 --2- 172.21.15.67:0/47877626 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f0a880b6510 0x7f0a880b8a20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0a90000c00 tx=0x7f0a90009680 comp rx=0 tx=0).stop 2024-09-21T06:33:50.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:50.168+0000 7f0aa5706640 1 -- 172.21.15.67:0/47877626 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0aa01083d0 msgr2=0x7f0aa019a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:50.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:50.168+0000 7f0aa5706640 1 --2- 172.21.15.67:0/47877626 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0aa01083d0 0x7f0aa019a970 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400b450 tx=0x7f0a9400b910 comp rx=0 tx=0).stop 2024-09-21T06:33:50.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:50.168+0000 7f0aa5706640 1 -- 172.21.15.67:0/47877626 shutdown_connections 2024-09-21T06:33:50.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:50.168+0000 7f0aa5706640 1 --2- 172.21.15.67:0/47877626 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f0a880b6510 0x7f0a880b8a20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:50.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:50.168+0000 7f0aa5706640 1 --2- 172.21.15.67:0/47877626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aa0108d20 0x7f0aa019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:50.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:50.168+0000 7f0aa5706640 1 --2- 172.21.15.67:0/47877626 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0aa01083d0 0x7f0aa019a970 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:50.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:50.168+0000 7f0aa5706640 1 --2- 172.21.15.67:0/47877626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aa0102ae0 0x7f0aa019a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:50.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:50.168+0000 7f0aa5706640 1 -- 172.21.15.67:0/47877626 >> 172.21.15.67:0/47877626 conn(0x7f0aa0079760 msgr2=0x7f0aa01015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:50.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:50.168+0000 7f0aa5706640 1 -- 172.21.15.67:0/47877626 shutdown_connections 2024-09-21T06:33:50.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:50.168+0000 7f0aa5706640 1 -- 172.21.15.67:0/47877626 wait complete. 2024-09-21T06:33:50.732 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 21 06:33:50 smithi203 systemd[1]: Starting Ceph mds.l for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:33:51.087 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 21 06:33:50 smithi203 podman[61115]: 2024-09-21 06:33:50.726009474 +0000 UTC m=+0.015883295 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:33:51.088 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 21 06:33:50 smithi203 podman[61115]: 2024-09-21 06:33:50.825446218 +0000 UTC m=+0.115320066 container create 1e20586e4e96774ac74be17325b5a100c14135cf805d02ee4433dee6ffad55a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-l, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-21T06:33:51.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:51 smithi067 ceph-mon[31232]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:33:51.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:51 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/47877626' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:33:51.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:51 smithi203 ceph-mon[33735]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:33:51.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:51 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/47877626' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:33:51.482 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 21 06:33:51 smithi203 podman[61115]: 2024-09-21 06:33:51.086904406 +0000 UTC m=+0.376778219 container init 1e20586e4e96774ac74be17325b5a100c14135cf805d02ee4433dee6ffad55a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-l, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-21T06:33:51.482 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 21 06:33:51 smithi203 podman[61115]: 2024-09-21 06:33:51.08968831 +0000 UTC m=+0.379562121 container start 1e20586e4e96774ac74be17325b5a100c14135cf805d02ee4433dee6ffad55a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-l, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-21T06:33:51.482 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 21 06:33:51 smithi203 ceph-mds[61134]: -- 172.21.15.203:0/2532788060 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563b0868d860 con 0x563b0c835180 2024-09-21T06:33:51.483 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 21 06:33:51 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-l[61130]: starting mds.l at 2024-09-21T06:33:51.483 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 21 06:33:51 smithi203 bash[61115]: 1e20586e4e96774ac74be17325b5a100c14135cf805d02ee4433dee6ffad55a2 2024-09-21T06:33:51.483 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 21 06:33:51 smithi203 systemd[1]: Started Ceph mds.l for 2a791768-77e2-11ef-baf5-efdc52797490. 2024-09-21T06:33:51.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:51 smithi100 ceph-mon[33401]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:33:51.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:51 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/47877626' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:33:52.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:52 smithi100 ceph-mon[33401]: mds.? [v2:172.21.15.203:6832/3852485418,v1:172.21.15.203:6833/3852485418] up:boot 2024-09-21T06:33:52.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:52 smithi100 ceph-mon[33401]: fsmap 1 up:standby 2024-09-21T06:33:52.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:52 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T06:33:52.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:52 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:52.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:52 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:52.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:52 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:52.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:52 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:33:52.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:52 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:33:52.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:52 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:52.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:52 smithi100 ceph-mon[33401]: Deploying daemon mds.k on smithi100 2024-09-21T06:33:52.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:52 smithi203 ceph-mon[33735]: mds.? [v2:172.21.15.203:6832/3852485418,v1:172.21.15.203:6833/3852485418] up:boot 2024-09-21T06:33:52.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:52 smithi203 ceph-mon[33735]: fsmap 1 up:standby 2024-09-21T06:33:52.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:52 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T06:33:52.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:52 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:52.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:52 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:52.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:52 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:52.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:52 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:33:52.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:52 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:33:52.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:52 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:52.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:52 smithi203 ceph-mon[33735]: Deploying daemon mds.k on smithi100 2024-09-21T06:33:52.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:52 smithi067 ceph-mon[31232]: mds.? [v2:172.21.15.203:6832/3852485418,v1:172.21.15.203:6833/3852485418] up:boot 2024-09-21T06:33:52.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:52 smithi067 ceph-mon[31232]: fsmap 1 up:standby 2024-09-21T06:33:52.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:52 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T06:33:52.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:52 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:52.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:52 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:52.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:52 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:52.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:52 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:33:52.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:52 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:33:52.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:52 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:52.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:52 smithi067 ceph-mon[31232]: Deploying daemon mds.k on smithi100 2024-09-21T06:33:52.911 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:33:53.181 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:33:53.250 INFO:journalctl@ceph.mds.k.smithi100.stdout:Sep 21 06:33:53 smithi100 systemd[1]: Starting Ceph mds.k for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:33:53.541 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:53 smithi067 ceph-mon[31232]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:33:53.544 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:53 smithi100 ceph-mon[33401]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:33:53.544 INFO:journalctl@ceph.mds.k.smithi100.stdout:Sep 21 06:33:53 smithi100 podman[60240]: 2024-09-21 06:33:53.315746728 +0000 UTC m=+0.110538852 container create 9195a712aedbacd2add04f89e453bc318c16e836133c7bd5b5727729bfbefb73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-k, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-21T06:33:53.544 INFO:journalctl@ceph.mds.k.smithi100.stdout:Sep 21 06:33:53 smithi100 podman[60240]: 2024-09-21 06:33:53.221059403 +0000 UTC m=+0.015851538 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:33:53.545 INFO:journalctl@ceph.mds.k.smithi100.stdout:Sep 21 06:33:53 smithi100 podman[60240]: 2024-09-21 06:33:53.543746563 +0000 UTC m=+0.338538686 container init 9195a712aedbacd2add04f89e453bc318c16e836133c7bd5b5727729bfbefb73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-k, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-21T06:33:53.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.686+0000 7f01b33cd640 1 -- 172.21.15.67:0/1112041506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01ac110200 msgr2=0x7f01ac112620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:53.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.686+0000 7f01b33cd640 1 --2- 172.21.15.67:0/1112041506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01ac110200 0x7f01ac112620 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009ca0 tx=0x7f01a402a950 comp rx=0 tx=0).stop 2024-09-21T06:33:53.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.686+0000 7f01b33cd640 1 -- 172.21.15.67:0/1112041506 shutdown_connections 2024-09-21T06:33:53.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.686+0000 7f01b33cd640 1 --2- 172.21.15.67:0/1112041506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01ac110200 0x7f01ac112620 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.686+0000 7f01b33cd640 1 --2- 172.21.15.67:0/1112041506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01ac102fa0 0x7f01ac10fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.686+0000 7f01b33cd640 1 --2- 172.21.15.67:0/1112041506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01ac102650 0x7f01ac102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.686+0000 7f01b33cd640 1 -- 172.21.15.67:0/1112041506 >> 172.21.15.67:0/1112041506 conn(0x7f01ac079760 msgr2=0x7f01ac079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:53.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.686+0000 7f01b33cd640 1 -- 172.21.15.67:0/1112041506 shutdown_connections 2024-09-21T06:33:53.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.686+0000 7f01b33cd640 1 -- 172.21.15.67:0/1112041506 wait complete. 2024-09-21T06:33:53.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.687+0000 7f01b33cd640 1 Processor -- start 2024-09-21T06:33:53.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.687+0000 7f01b33cd640 1 -- start start 2024-09-21T06:33:53.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.687+0000 7f01b33cd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01ac102650 0x7f01ac19c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:53.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.687+0000 7f01b33cd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01ac102fa0 0x7f01ac19cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:53.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.687+0000 7f01b33cd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01ac110200 0x7f01ac1a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:53.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.687+0000 7f01b33cd640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac115b50 con 0x7f01ac110200 2024-09-21T06:33:53.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.687+0000 7f01b33cd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac1159d0 con 0x7f01ac102650 2024-09-21T06:33:53.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.687+0000 7f01b33cd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac115cd0 con 0x7f01ac102fa0 2024-09-21T06:33:53.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.687+0000 7f01b196a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01ac110200 0x7f01ac1a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:53.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.687+0000 7f01b196a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01ac110200 0x7f01ac1a1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39644/0 (socket says 172.21.15.67:39644) 2024-09-21T06:33:53.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.687+0000 7f01b196a640 1 -- 172.21.15.67:0/265689764 learned_addr learned my addr 172.21.15.67:0/265689764 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:33:53.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.688+0000 7f01b1169640 1 --2- 172.21.15.67:0/265689764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01ac102650 0x7f01ac19c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:53.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.688+0000 7f01b0968640 1 --2- 172.21.15.67:0/265689764 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01ac102fa0 0x7f01ac19cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:53.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.688+0000 7f01b196a640 1 -- 172.21.15.67:0/265689764 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01ac102fa0 msgr2=0x7f01ac19cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:53.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.688+0000 7f01b196a640 1 --2- 172.21.15.67:0/265689764 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01ac102fa0 0x7f01ac19cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.688+0000 7f01b196a640 1 -- 172.21.15.67:0/265689764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01ac102650 msgr2=0x7f01ac19c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:53.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.688+0000 7f01b196a640 1 --2- 172.21.15.67:0/265689764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01ac102650 0x7f01ac19c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.688+0000 7f01b196a640 1 -- 172.21.15.67:0/265689764 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ac1a24f0 con 0x7f01ac110200 2024-09-21T06:33:53.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.688+0000 7f01b196a640 1 --2- 172.21.15.67:0/265689764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01ac110200 0x7f01ac1a1dc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f01a402ae70 tx=0x7f01a4003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:53.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.688+0000 7f01a27fc640 1 -- 172.21.15.67:0/265689764 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a4034040 con 0x7f01ac110200 2024-09-21T06:33:53.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.688+0000 7f01b33cd640 1 -- 172.21.15.67:0/265689764 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01ac1a2780 con 0x7f01ac110200 2024-09-21T06:33:53.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.688+0000 7f01a27fc640 1 -- 172.21.15.67:0/265689764 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01a4030850 con 0x7f01ac110200 2024-09-21T06:33:53.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.688+0000 7f01a27fc640 1 -- 172.21.15.67:0/265689764 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a4004600 con 0x7f01ac110200 2024-09-21T06:33:53.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.688+0000 7f01b33cd640 1 -- 172.21.15.67:0/265689764 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01ac1a8f40 con 0x7f01ac110200 2024-09-21T06:33:53.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.689+0000 7f01b33cd640 1 -- 172.21.15.67:0/265689764 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01ac06c1f0 con 0x7f01ac110200 2024-09-21T06:33:53.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.691+0000 7f01a27fc640 1 -- 172.21.15.67:0/265689764 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f01a4037020 con 0x7f01ac110200 2024-09-21T06:33:53.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.692+0000 7f01a27fc640 1 --2- 172.21.15.67:0/265689764 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f01980b6510 0x7f01980b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:53.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.692+0000 7f01b1169640 1 --2- 172.21.15.67:0/265689764 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f01980b6510 0x7f01980b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:53.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.692+0000 7f01a27fc640 1 -- 172.21.15.67:0/265689764 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f01a4107d60 con 0x7f01ac110200 2024-09-21T06:33:53.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.692+0000 7f01b1169640 1 --2- 172.21.15.67:0/265689764 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f01980b6510 0x7f01980b8a20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f01a8009bd0 tx=0x7f01a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:53.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.695+0000 7f01a27fc640 1 -- 172.21.15.67:0/265689764 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f01a4083710 con 0x7f01ac110200 2024-09-21T06:33:53.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:53 smithi203 ceph-mon[33735]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:33:53.803 INFO:journalctl@ceph.mds.k.smithi100.stdout:Sep 21 06:33:53 smithi100 podman[60240]: 2024-09-21 06:33:53.546731231 +0000 UTC m=+0.341523355 container start 9195a712aedbacd2add04f89e453bc318c16e836133c7bd5b5727729bfbefb73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-k, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-21T06:33:53.804 INFO:journalctl@ceph.mds.k.smithi100.stdout:Sep 21 06:33:53 smithi100 ceph-mds[60260]: -- 172.21.15.100:0/2815778583 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557e8a56b860 con 0x557e8e713a80 2024-09-21T06:33:53.804 INFO:journalctl@ceph.mds.k.smithi100.stdout:Sep 21 06:33:53 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-k[60256]: starting mds.k at 2024-09-21T06:33:53.804 INFO:journalctl@ceph.mds.k.smithi100.stdout:Sep 21 06:33:53 smithi100 bash[60240]: 9195a712aedbacd2add04f89e453bc318c16e836133c7bd5b5727729bfbefb73 2024-09-21T06:33:53.804 INFO:journalctl@ceph.mds.k.smithi100.stdout:Sep 21 06:33:53 smithi100 systemd[1]: Started Ceph mds.k for 2a791768-77e2-11ef-baf5-efdc52797490. 2024-09-21T06:33:53.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.927+0000 7f01b33cd640 1 -- 172.21.15.67:0/265689764 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01ac102a60 con 0x7f01ac110200 2024-09-21T06:33:53.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.927+0000 7f01a27fc640 1 -- 172.21.15.67:0/265689764 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f01ac102a60 con 0x7f01ac110200 2024-09-21T06:33:53.930 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:33:53.930 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":2,"btime":"2024-09-21T06:33:51:132008+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T06:33:53.931 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 2 2024-09-21T06:33:53.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.930+0000 7f01b33cd640 1 -- 172.21.15.67:0/265689764 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f01980b6510 msgr2=0x7f01980b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:53.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.930+0000 7f01b33cd640 1 --2- 172.21.15.67:0/265689764 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f01980b6510 0x7f01980b8a20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f01a8009bd0 tx=0x7f01a8005f90 comp rx=0 tx=0).stop 2024-09-21T06:33:53.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.930+0000 7f01b33cd640 1 -- 172.21.15.67:0/265689764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01ac110200 msgr2=0x7f01ac1a1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:53.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.930+0000 7f01b33cd640 1 --2- 172.21.15.67:0/265689764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01ac110200 0x7f01ac1a1dc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f01a402ae70 tx=0x7f01a4003ce0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.930+0000 7f01b33cd640 1 -- 172.21.15.67:0/265689764 shutdown_connections 2024-09-21T06:33:53.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.930+0000 7f01b33cd640 1 --2- 172.21.15.67:0/265689764 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f01980b6510 0x7f01980b8a20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.930+0000 7f01b33cd640 1 --2- 172.21.15.67:0/265689764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01ac110200 0x7f01ac1a1dc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.930+0000 7f01b33cd640 1 --2- 172.21.15.67:0/265689764 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01ac102fa0 0x7f01ac19cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.930+0000 7f01b33cd640 1 --2- 172.21.15.67:0/265689764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01ac102650 0x7f01ac19c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:53.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.930+0000 7f01b33cd640 1 -- 172.21.15.67:0/265689764 >> 172.21.15.67:0/265689764 conn(0x7f01ac079760 msgr2=0x7f01ac108c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:53.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.930+0000 7f01b33cd640 1 -- 172.21.15.67:0/265689764 shutdown_connections 2024-09-21T06:33:53.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:53.930+0000 7f01b33cd640 1 -- 172.21.15.67:0/265689764 wait complete. 2024-09-21T06:33:54.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:54 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:54.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:54 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:54.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:54 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:54.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:54 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:33:54.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:54 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:33:54.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:54 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:54.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:54 smithi203 ceph-mon[33735]: Deploying daemon mds.j on smithi067 2024-09-21T06:33:54.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:54 smithi203 ceph-mon[33735]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:33:54.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:54 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/265689764' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:33:54.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:54 smithi203 ceph-mon[33735]: mds.? [v2:172.21.15.100:6832/4088716909,v1:172.21.15.100:6833/4088716909] up:boot 2024-09-21T06:33:54.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:54 smithi203 ceph-mon[33735]: fsmap 2 up:standby 2024-09-21T06:33:54.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:54 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T06:33:54.984 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:54 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:54.984 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:54 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:54.984 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:54 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:54.984 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:54 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:33:54.984 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:54 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:33:54.984 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:54 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:54.984 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:54 smithi067 ceph-mon[31232]: Deploying daemon mds.j on smithi067 2024-09-21T06:33:54.984 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:54 smithi067 ceph-mon[31232]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:33:54.984 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:54 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/265689764' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:33:54.984 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:54 smithi067 ceph-mon[31232]: mds.? [v2:172.21.15.100:6832/4088716909,v1:172.21.15.100:6833/4088716909] up:boot 2024-09-21T06:33:54.985 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:54 smithi067 ceph-mon[31232]: fsmap 2 up:standby 2024-09-21T06:33:54.985 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:54 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T06:33:55.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:54 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:55.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:54 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:55.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:54 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:55.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:54 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:33:55.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:54 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:33:55.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:54 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:55.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:54 smithi100 ceph-mon[33401]: Deploying daemon mds.j on smithi067 2024-09-21T06:33:55.055 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:54 smithi100 ceph-mon[33401]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:33:55.055 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:54 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/265689764' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:33:55.055 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:54 smithi100 ceph-mon[33401]: mds.? [v2:172.21.15.100:6832/4088716909,v1:172.21.15.100:6833/4088716909] up:boot 2024-09-21T06:33:55.055 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:54 smithi100 ceph-mon[33401]: fsmap 2 up:standby 2024-09-21T06:33:55.055 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:54 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T06:33:56.542 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 21 06:33:56 smithi067 systemd[1]: Starting Ceph mds.j for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:33:56.694 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:33:56.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:56 smithi067 ceph-mon[31232]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:33:56.924 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 21 06:33:56 smithi067 podman[70615]: 2024-09-21 06:33:56.56547081 +0000 UTC m=+0.016178415 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:33:56.924 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 21 06:33:56 smithi067 podman[70615]: 2024-09-21 06:33:56.678707288 +0000 UTC m=+0.129414883 container create 97060575a224078261c95eda1d9c6fc9690ae15c76f56c5dd2a3e356321e3e1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:33:56.924 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 21 06:33:56 smithi067 podman[70615]: 2024-09-21 06:33:56.922510069 +0000 UTC m=+0.373217677 container init 97060575a224078261c95eda1d9c6fc9690ae15c76f56c5dd2a3e356321e3e1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-21T06:33:57.141 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:33:57.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:56 smithi203 ceph-mon[33735]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:33:57.265 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 21 06:33:56 smithi067 podman[70615]: 2024-09-21 06:33:56.925814403 +0000 UTC m=+0.376521999 container start 97060575a224078261c95eda1d9c6fc9690ae15c76f56c5dd2a3e356321e3e1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-j, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-21T06:33:57.265 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 21 06:33:56 smithi067 ceph-mds[70664]: -- 172.21.15.67:0/3246868881 <== mon.0 v2:172.21.15.67:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56031f817860 con 0x5603239bfa80 2024-09-21T06:33:57.265 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 21 06:33:56 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-j[70648]: starting mds.j at 2024-09-21T06:33:57.265 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 21 06:33:57 smithi067 bash[70615]: 97060575a224078261c95eda1d9c6fc9690ae15c76f56c5dd2a3e356321e3e1c 2024-09-21T06:33:57.266 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 21 06:33:57 smithi067 systemd[1]: Started Ceph mds.j for 2a791768-77e2-11ef-baf5-efdc52797490. 2024-09-21T06:33:57.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:56 smithi100 ceph-mon[33401]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:33:57.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.619+0000 7f146f1b3640 1 -- 172.21.15.67:0/1202321652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1468102ae0 msgr2=0x7f1468102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:57.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.619+0000 7f146f1b3640 1 --2- 172.21.15.67:0/1202321652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1468102ae0 0x7f1468102ef0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1450009900 tx=0x7f145002a410 comp rx=0 tx=0).stop 2024-09-21T06:33:57.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.619+0000 7f146f1b3640 1 -- 172.21.15.67:0/1202321652 shutdown_connections 2024-09-21T06:33:57.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.619+0000 7f146f1b3640 1 --2- 172.21.15.67:0/1202321652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14681083d0 0x7f14681087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:57.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.619+0000 7f146f1b3640 1 --2- 172.21.15.67:0/1202321652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1468108d20 0x7f146810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:57.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.619+0000 7f146f1b3640 1 --2- 172.21.15.67:0/1202321652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1468102ae0 0x7f1468102ef0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:57.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.619+0000 7f146f1b3640 1 -- 172.21.15.67:0/1202321652 >> 172.21.15.67:0/1202321652 conn(0x7f1468079760 msgr2=0x7f1468079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:57.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.619+0000 7f146f1b3640 1 -- 172.21.15.67:0/1202321652 shutdown_connections 2024-09-21T06:33:57.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.619+0000 7f146f1b3640 1 -- 172.21.15.67:0/1202321652 wait complete. 2024-09-21T06:33:57.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.620+0000 7f146f1b3640 1 Processor -- start 2024-09-21T06:33:57.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.620+0000 7f146f1b3640 1 -- start start 2024-09-21T06:33:57.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.620+0000 7f146f1b3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1468102ae0 0x7f146819a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:57.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.620+0000 7f146f1b3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14681083d0 0x7f146819a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:57.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.620+0000 7f146f1b3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1468108d20 0x7f146819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:57.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.620+0000 7f146f1b3640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f14681128a0 con 0x7f14681083d0 2024-09-21T06:33:57.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.620+0000 7f146f1b3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1468112720 con 0x7f1468108d20 2024-09-21T06:33:57.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.620+0000 7f146f1b3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1468112a20 con 0x7f1468102ae0 2024-09-21T06:33:57.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.620+0000 7f1467fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14681083d0 0x7f146819a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:57.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.620+0000 7f1467fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14681083d0 0x7f146819a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57688/0 (socket says 172.21.15.67:57688) 2024-09-21T06:33:57.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.620+0000 7f1467fff640 1 -- 172.21.15.67:0/3667671619 learned_addr learned my addr 172.21.15.67:0/3667671619 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:33:57.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.620+0000 7f146cf4f640 1 --2- 172.21.15.67:0/3667671619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1468102ae0 0x7f146819a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:57.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.620+0000 7f146d750640 1 --2- 172.21.15.67:0/3667671619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1468108d20 0x7f146819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:57.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.620+0000 7f1467fff640 1 -- 172.21.15.67:0/3667671619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1468102ae0 msgr2=0x7f146819a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:57.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.620+0000 7f1467fff640 1 --2- 172.21.15.67:0/3667671619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1468102ae0 0x7f146819a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:57.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.620+0000 7f1467fff640 1 -- 172.21.15.67:0/3667671619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1468108d20 msgr2=0x7f146819fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:57.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.620+0000 7f1467fff640 1 --2- 172.21.15.67:0/3667671619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1468108d20 0x7f146819fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:57.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.620+0000 7f1467fff640 1 -- 172.21.15.67:0/3667671619 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14681a0270 con 0x7f14681083d0 2024-09-21T06:33:57.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.621+0000 7f1467fff640 1 --2- 172.21.15.67:0/3667671619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14681083d0 0x7f146819a960 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f145800ccb0 tx=0x7f145800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:57.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.621+0000 7f1465ffb640 1 -- 172.21.15.67:0/3667671619 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145800ce60 con 0x7f14681083d0 2024-09-21T06:33:57.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.621+0000 7f146f1b3640 1 -- 172.21.15.67:0/3667671619 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14681a0560 con 0x7f14681083d0 2024-09-21T06:33:57.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.621+0000 7f1465ffb640 1 -- 172.21.15.67:0/3667671619 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f145800fae0 con 0x7f14681083d0 2024-09-21T06:33:57.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.621+0000 7f1465ffb640 1 -- 172.21.15.67:0/3667671619 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1458018aa0 con 0x7f14681083d0 2024-09-21T06:33:57.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.621+0000 7f146f1b3640 1 -- 172.21.15.67:0/3667671619 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14681a6d80 con 0x7f14681083d0 2024-09-21T06:33:57.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.622+0000 7f146f1b3640 1 -- 172.21.15.67:0/3667671619 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1468103d10 con 0x7f14681083d0 2024-09-21T06:33:57.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.624+0000 7f1465ffb640 1 -- 172.21.15.67:0/3667671619 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f1458018c40 con 0x7f14681083d0 2024-09-21T06:33:57.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.624+0000 7f1465ffb640 1 --2- 172.21.15.67:0/3667671619 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f14540b6540 0x7f14540b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:33:57.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.624+0000 7f146cf4f640 1 --2- 172.21.15.67:0/3667671619 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f14540b6540 0x7f14540b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:33:57.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.625+0000 7f1465ffb640 1 -- 172.21.15.67:0/3667671619 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f14580e7590 con 0x7f14681083d0 2024-09-21T06:33:57.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.625+0000 7f146cf4f640 1 --2- 172.21.15.67:0/3667671619 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f14540b6540 0x7f14540b8a50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1450000c00 tx=0x7f1450009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:33:57.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.628+0000 7f1465ffb640 1 -- 172.21.15.67:0/3667671619 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1458014030 con 0x7f14681083d0 2024-09-21T06:33:57.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.859+0000 7f146f1b3640 1 -- 172.21.15.67:0/3667671619 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f146819b4f0 con 0x7f14681083d0 2024-09-21T06:33:57.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.860+0000 7f1465ffb640 1 -- 172.21.15.67:0/3667671619 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2266 (secure 0 0 0) 0x7f146819b4f0 con 0x7f14681083d0 2024-09-21T06:33:57.863 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:33:57.863 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":3,"btime":"2024-09-21T06:33:54:249164+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-21T06:33:57.863 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 3 2024-09-21T06:33:57.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.862+0000 7f146f1b3640 1 -- 172.21.15.67:0/3667671619 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f14540b6540 msgr2=0x7f14540b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:57.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.862+0000 7f146f1b3640 1 --2- 172.21.15.67:0/3667671619 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f14540b6540 0x7f14540b8a50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1450000c00 tx=0x7f1450009680 comp rx=0 tx=0).stop 2024-09-21T06:33:57.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.862+0000 7f146f1b3640 1 -- 172.21.15.67:0/3667671619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14681083d0 msgr2=0x7f146819a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:33:57.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.862+0000 7f146f1b3640 1 --2- 172.21.15.67:0/3667671619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14681083d0 0x7f146819a960 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f145800ccb0 tx=0x7f145800a620 comp rx=0 tx=0).stop 2024-09-21T06:33:57.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.862+0000 7f146f1b3640 1 -- 172.21.15.67:0/3667671619 shutdown_connections 2024-09-21T06:33:57.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.862+0000 7f146f1b3640 1 --2- 172.21.15.67:0/3667671619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1468108d20 0x7f146819fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:57.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.862+0000 7f146f1b3640 1 --2- 172.21.15.67:0/3667671619 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f14540b6540 0x7f14540b8a50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:57.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.862+0000 7f146f1b3640 1 --2- 172.21.15.67:0/3667671619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14681083d0 0x7f146819a960 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:57.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.862+0000 7f146f1b3640 1 --2- 172.21.15.67:0/3667671619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1468102ae0 0x7f146819a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:33:57.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.862+0000 7f146f1b3640 1 -- 172.21.15.67:0/3667671619 >> 172.21.15.67:0/3667671619 conn(0x7f1468079760 msgr2=0x7f1468106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:33:57.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.862+0000 7f146f1b3640 1 -- 172.21.15.67:0/3667671619 shutdown_connections 2024-09-21T06:33:57.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:33:57.862+0000 7f146f1b3640 1 -- 172.21.15.67:0/3667671619 wait complete. 2024-09-21T06:33:58.394 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:58 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:58.394 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:58 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:58.394 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:58 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:58.394 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:58 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:33:58.394 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:58 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:33:58.394 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:58 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:58.394 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:58 smithi203 ceph-mon[33735]: Deploying daemon mds.d on smithi067 2024-09-21T06:33:58.394 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:58 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3667671619' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:33:58.394 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:58 smithi203 ceph-mon[33735]: mds.? [v2:172.21.15.67:6834/4145954943,v1:172.21.15.67:6835/4145954943] up:boot 2024-09-21T06:33:58.394 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:58 smithi203 ceph-mon[33735]: fsmap 3 up:standby 2024-09-21T06:33:58.394 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:58 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T06:33:58.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:58 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:58.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:58 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:58.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:58 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:58.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:58 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:33:58.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:58 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:33:58.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:58 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:58.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:58 smithi067 ceph-mon[31232]: Deploying daemon mds.d on smithi067 2024-09-21T06:33:58.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:58 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3667671619' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:33:58.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:58 smithi067 ceph-mon[31232]: mds.? [v2:172.21.15.67:6834/4145954943,v1:172.21.15.67:6835/4145954943] up:boot 2024-09-21T06:33:58.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:58 smithi067 ceph-mon[31232]: fsmap 3 up:standby 2024-09-21T06:33:58.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:58 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T06:33:58.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:58 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:58.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:58 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:58.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:58 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:33:58.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:58 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:33:58.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:58 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:33:58.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:58 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:33:58.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:58 smithi100 ceph-mon[33401]: Deploying daemon mds.d on smithi067 2024-09-21T06:33:58.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:58 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3667671619' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:33:58.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:58 smithi100 ceph-mon[33401]: mds.? [v2:172.21.15.67:6834/4145954943,v1:172.21.15.67:6835/4145954943] up:boot 2024-09-21T06:33:58.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:58 smithi100 ceph-mon[33401]: fsmap 3 up:standby 2024-09-21T06:33:58.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:58 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T06:33:59.291 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:33:59 smithi067 ceph-mon[31232]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:33:59.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:33:59 smithi203 ceph-mon[33735]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:33:59.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:33:59 smithi100 ceph-mon[33401]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:00.292 INFO:journalctl@ceph.mds.d.smithi067.stdout:Sep 21 06:34:00 smithi067 systemd[1]: Starting Ceph mds.d for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:34:00.717 INFO:journalctl@ceph.mds.d.smithi067.stdout:Sep 21 06:34:00 smithi067 podman[70996]: 2024-09-21 06:34:00.33488654 +0000 UTC m=+0.016046650 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:34:00.717 INFO:journalctl@ceph.mds.d.smithi067.stdout:Sep 21 06:34:00 smithi067 podman[70996]: 2024-09-21 06:34:00.442701753 +0000 UTC m=+0.123861853 container create 4744b69f185ddbab1717dfe553b581b149f39739ad2bc775b305094d002d87f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-d, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:34:00.717 INFO:journalctl@ceph.mds.d.smithi067.stdout:Sep 21 06:34:00 smithi067 podman[70996]: 2024-09-21 06:34:00.67870267 +0000 UTC m=+0.359862781 container init 4744b69f185ddbab1717dfe553b581b149f39739ad2bc775b305094d002d87f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True) 2024-09-21T06:34:00.717 INFO:journalctl@ceph.mds.d.smithi067.stdout:Sep 21 06:34:00 smithi067 podman[70996]: 2024-09-21 06:34:00.681433342 +0000 UTC m=+0.362593447 container start 4744b69f185ddbab1717dfe553b581b149f39739ad2bc775b305094d002d87f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-21T06:34:00.777 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:34:01.041 INFO:journalctl@ceph.mds.d.smithi067.stdout:Sep 21 06:34:00 smithi067 ceph-mds[71015]: -- 172.21.15.67:0/3449438177 <== mon.0 v2:172.21.15.67:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e743715860 con 0x55e7478bda80 2024-09-21T06:34:01.042 INFO:journalctl@ceph.mds.d.smithi067.stdout:Sep 21 06:34:00 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-d[71011]: starting mds.d at 2024-09-21T06:34:01.042 INFO:journalctl@ceph.mds.d.smithi067.stdout:Sep 21 06:34:00 smithi067 bash[70996]: 4744b69f185ddbab1717dfe553b581b149f39739ad2bc775b305094d002d87f1 2024-09-21T06:34:01.042 INFO:journalctl@ceph.mds.d.smithi067.stdout:Sep 21 06:34:00 smithi067 systemd[1]: Started Ceph mds.d for 2a791768-77e2-11ef-baf5-efdc52797490. 2024-09-21T06:34:01.042 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:34:01.442 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:01 smithi067 ceph-mon[31232]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:01.442 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:01 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:01.442 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:01 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:01.442 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:01 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:01.442 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:01 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:34:01.442 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:01 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:34:01.443 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:01 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:34:01.456 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:01 smithi100 ceph-mon[33401]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:01.456 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:01 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:01.456 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:01 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:01.456 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:01 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:01.457 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:01 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:34:01.457 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:01 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:34:01.457 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:01 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:34:01.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:01 smithi203 ceph-mon[33735]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:01.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:01 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:01.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:01 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:01.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:01 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:01.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:01 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:34:01.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:01 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:34:01.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:01 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:34:01.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.533+0000 7ff090860640 1 -- 172.21.15.67:0/368511500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff08c110160 msgr2=0x7ff08c112580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:01.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.533+0000 7ff090860640 1 --2- 172.21.15.67:0/368511500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff08c110160 0x7ff08c112580 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff084009ca0 tx=0x7ff08402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:34:01.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.534+0000 7ff090860640 1 -- 172.21.15.67:0/368511500 shutdown_connections 2024-09-21T06:34:01.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.534+0000 7ff090860640 1 --2- 172.21.15.67:0/368511500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff08c076080 0x7ff08c10fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:01.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.534+0000 7ff090860640 1 --2- 172.21.15.67:0/368511500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff08c110160 0x7ff08c112580 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:01.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.534+0000 7ff090860640 1 --2- 172.21.15.67:0/368511500 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff08c075730 0x7ff08c075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:01.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.534+0000 7ff090860640 1 -- 172.21.15.67:0/368511500 >> 172.21.15.67:0/368511500 conn(0x7ff08c0ff610 msgr2=0x7ff08c101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:01.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.534+0000 7ff090860640 1 -- 172.21.15.67:0/368511500 shutdown_connections 2024-09-21T06:34:01.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.534+0000 7ff090860640 1 -- 172.21.15.67:0/368511500 wait complete. 2024-09-21T06:34:01.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.534+0000 7ff090860640 1 Processor -- start 2024-09-21T06:34:01.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.535+0000 7ff090860640 1 -- start start 2024-09-21T06:34:01.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.535+0000 7ff090860640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff08c075730 0x7ff08c19c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:01.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.535+0000 7ff090860640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff08c076080 0x7ff08c19cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:01.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.535+0000 7ff090860640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff08c110160 0x7ff08c1a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:01.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.535+0000 7ff090860640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c115bc0 con 0x7ff08c110160 2024-09-21T06:34:01.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.535+0000 7ff090860640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c115a40 con 0x7ff08c075730 2024-09-21T06:34:01.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.535+0000 7ff090860640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c115d40 con 0x7ff08c076080 2024-09-21T06:34:01.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.535+0000 7ff08ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff08c110160 0x7ff08c1a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:01.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.535+0000 7ff08ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff08c110160 0x7ff08c1a1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57724/0 (socket says 172.21.15.67:57724) 2024-09-21T06:34:01.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.535+0000 7ff08ad9d640 1 -- 172.21.15.67:0/1897603550 learned_addr learned my addr 172.21.15.67:0/1897603550 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:34:01.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.535+0000 7ff08a59c640 1 --2- 172.21.15.67:0/1897603550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff08c075730 0x7ff08c19c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:01.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.535+0000 7ff089d9b640 1 --2- 172.21.15.67:0/1897603550 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff08c076080 0x7ff08c19cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:01.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.535+0000 7ff08ad9d640 1 -- 172.21.15.67:0/1897603550 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff08c076080 msgr2=0x7ff08c19cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:01.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.535+0000 7ff08ad9d640 1 --2- 172.21.15.67:0/1897603550 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff08c076080 0x7ff08c19cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:01.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.535+0000 7ff08ad9d640 1 -- 172.21.15.67:0/1897603550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff08c075730 msgr2=0x7ff08c19c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:01.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.535+0000 7ff08ad9d640 1 --2- 172.21.15.67:0/1897603550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff08c075730 0x7ff08c19c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:01.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.535+0000 7ff08ad9d640 1 -- 172.21.15.67:0/1897603550 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08c1a24e0 con 0x7ff08c110160 2024-09-21T06:34:01.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.536+0000 7ff08ad9d640 1 --2- 172.21.15.67:0/1897603550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff08c110160 0x7ff08c1a1db0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff084005cc0 tx=0x7ff084032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:01.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.536+0000 7ff0837fe640 1 -- 172.21.15.67:0/1897603550 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff084032e00 con 0x7ff08c110160 2024-09-21T06:34:01.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.536+0000 7ff0837fe640 1 -- 172.21.15.67:0/1897603550 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff0840044d0 con 0x7ff08c110160 2024-09-21T06:34:01.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.536+0000 7ff090860640 1 -- 172.21.15.67:0/1897603550 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff08c1a2770 con 0x7ff08c110160 2024-09-21T06:34:01.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.536+0000 7ff0837fe640 1 -- 172.21.15.67:0/1897603550 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff084030b70 con 0x7ff08c110160 2024-09-21T06:34:01.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.536+0000 7ff090860640 1 -- 172.21.15.67:0/1897603550 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff08c1a8f30 con 0x7ff08c110160 2024-09-21T06:34:01.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.537+0000 7ff090860640 1 -- 172.21.15.67:0/1897603550 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff08c076c40 con 0x7ff08c110160 2024-09-21T06:34:01.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.538+0000 7ff0837fe640 1 -- 172.21.15.67:0/1897603550 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7ff084030d10 con 0x7ff08c110160 2024-09-21T06:34:01.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.539+0000 7ff0837fe640 1 --2- 172.21.15.67:0/1897603550 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7ff0780b6510 0x7ff0780b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:01.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.540+0000 7ff08a59c640 1 --2- 172.21.15.67:0/1897603550 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7ff0780b6510 0x7ff0780b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:01.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.540+0000 7ff0837fe640 1 -- 172.21.15.67:0/1897603550 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7ff084108240 con 0x7ff08c110160 2024-09-21T06:34:01.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.540+0000 7ff08a59c640 1 --2- 172.21.15.67:0/1897603550 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7ff0780b6510 0x7ff0780b8a20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff07c005fd0 tx=0x7ff07c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:01.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.543+0000 7ff0837fe640 1 -- 172.21.15.67:0/1897603550 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff0840cef70 con 0x7ff08c110160 2024-09-21T06:34:01.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.768+0000 7ff090860640 1 -- 172.21.15.67:0/1897603550 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff08c075b40 con 0x7ff08c110160 2024-09-21T06:34:01.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.769+0000 7ff0837fe640 1 -- 172.21.15.67:0/1897603550 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3916 (secure 0 0 0) 0x7ff0840d3350 con 0x7ff08c110160 2024-09-21T06:34:01.771 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:34:01.772 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":5,"btime":"2024-09-21T06:34:01:128542+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-21T06:34:01.772 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 5 2024-09-21T06:34:01.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.771+0000 7ff090860640 1 -- 172.21.15.67:0/1897603550 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7ff0780b6510 msgr2=0x7ff0780b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:01.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.771+0000 7ff090860640 1 --2- 172.21.15.67:0/1897603550 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7ff0780b6510 0x7ff0780b8a20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff07c005fd0 tx=0x7ff07c00b390 comp rx=0 tx=0).stop 2024-09-21T06:34:01.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.771+0000 7ff090860640 1 -- 172.21.15.67:0/1897603550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff08c110160 msgr2=0x7ff08c1a1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:01.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.771+0000 7ff090860640 1 --2- 172.21.15.67:0/1897603550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff08c110160 0x7ff08c1a1db0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff084005cc0 tx=0x7ff084032a10 comp rx=0 tx=0).stop 2024-09-21T06:34:01.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.771+0000 7ff090860640 1 -- 172.21.15.67:0/1897603550 shutdown_connections 2024-09-21T06:34:01.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.771+0000 7ff090860640 1 --2- 172.21.15.67:0/1897603550 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7ff0780b6510 0x7ff0780b8a20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:01.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.771+0000 7ff090860640 1 --2- 172.21.15.67:0/1897603550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff08c110160 0x7ff08c1a1db0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:01.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.771+0000 7ff090860640 1 --2- 172.21.15.67:0/1897603550 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff08c076080 0x7ff08c19cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:01.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.771+0000 7ff090860640 1 --2- 172.21.15.67:0/1897603550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff08c075730 0x7ff08c19c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:01.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.771+0000 7ff090860640 1 -- 172.21.15.67:0/1897603550 >> 172.21.15.67:0/1897603550 conn(0x7ff08c0ff610 msgr2=0x7ff08c109c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:01.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.772+0000 7ff090860640 1 -- 172.21.15.67:0/1897603550 shutdown_connections 2024-09-21T06:34:01.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:01.772+0000 7ff090860640 1 -- 172.21.15.67:0/1897603550 wait complete. 2024-09-21T06:34:02.421 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:02 smithi067 ceph-mon[31232]: Deploying daemon mds.e on smithi100 2024-09-21T06:34:02.422 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:02 smithi067 ceph-mon[31232]: mds.? [v2:172.21.15.67:6836/1744346240,v1:172.21.15.67:6837/1744346240] up:boot 2024-09-21T06:34:02.422 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:02 smithi067 ceph-mon[31232]: fsmap 4 up:standby 2024-09-21T06:34:02.422 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:02 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T06:34:02.422 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:02 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1897603550' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:34:02.436 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:02 smithi100 ceph-mon[33401]: Deploying daemon mds.e on smithi100 2024-09-21T06:34:02.437 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:02 smithi100 ceph-mon[33401]: mds.? [v2:172.21.15.67:6836/1744346240,v1:172.21.15.67:6837/1744346240] up:boot 2024-09-21T06:34:02.437 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:02 smithi100 ceph-mon[33401]: fsmap 4 up:standby 2024-09-21T06:34:02.437 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:02 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T06:34:02.437 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:02 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1897603550' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:34:02.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:02 smithi203 ceph-mon[33735]: Deploying daemon mds.e on smithi100 2024-09-21T06:34:02.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:02 smithi203 ceph-mon[33735]: mds.? [v2:172.21.15.67:6836/1744346240,v1:172.21.15.67:6837/1744346240] up:boot 2024-09-21T06:34:02.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:02 smithi203 ceph-mon[33735]: fsmap 4 up:standby 2024-09-21T06:34:02.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:02 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T06:34:02.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:02 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1897603550' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:34:03.143 INFO:journalctl@ceph.mds.e.smithi100.stdout:Sep 21 06:34:03 smithi100 podman[60514]: 2024-09-21 06:34:03.072482739 +0000 UTC m=+0.147402913 container create 21813274ce3b7459492d7567515ebeaf51d3705b2577cd7a60a6f34e8e6515d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-e, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-21T06:34:03.452 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:03 smithi100 ceph-mon[33401]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:03.452 INFO:journalctl@ceph.mds.e.smithi100.stdout:Sep 21 06:34:03 smithi100 podman[60514]: 2024-09-21 06:34:03.32553573 +0000 UTC m=+0.400455910 container init 21813274ce3b7459492d7567515ebeaf51d3705b2577cd7a60a6f34e8e6515d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-e, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:34:03.452 INFO:journalctl@ceph.mds.e.smithi100.stdout:Sep 21 06:34:03 smithi100 podman[60514]: 2024-09-21 06:34:03.328437136 +0000 UTC m=+0.403357309 container start 21813274ce3b7459492d7567515ebeaf51d3705b2577cd7a60a6f34e8e6515d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-e, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-21T06:34:03.452 INFO:journalctl@ceph.mds.e.smithi100.stdout:Sep 21 06:34:03 smithi100 ceph-mds[60534]: -- 172.21.15.100:0/3399879368 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a9b92bf860 con 0x55a9bd467a80 2024-09-21T06:34:03.452 INFO:journalctl@ceph.mds.e.smithi100.stdout:Sep 21 06:34:03 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-e[60530]: starting mds.e at 2024-09-21T06:34:03.452 INFO:journalctl@ceph.mds.e.smithi100.stdout:Sep 21 06:34:03 smithi100 bash[60514]: 21813274ce3b7459492d7567515ebeaf51d3705b2577cd7a60a6f34e8e6515d0 2024-09-21T06:34:03.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:03 smithi203 ceph-mon[33735]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:03.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:03 smithi067 ceph-mon[31232]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:03.803 INFO:journalctl@ceph.mds.e.smithi100.stdout:Sep 21 06:34:03 smithi100 systemd[1]: Started Ceph mds.e for 2a791768-77e2-11ef-baf5-efdc52797490. 2024-09-21T06:34:04.457 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:34:04.730 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:34:04.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:04 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:04.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:04 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:04.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:04 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:04.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:04 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:34:04.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:04 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:34:04.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:04 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:34:04.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:04 smithi067 ceph-mon[31232]: Deploying daemon mds.f on smithi203 2024-09-21T06:34:04.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:04 smithi067 ceph-mon[31232]: mds.? [v2:172.21.15.100:6834/1518007169,v1:172.21.15.100:6835/1518007169] up:boot 2024-09-21T06:34:04.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:04 smithi067 ceph-mon[31232]: fsmap 5 up:standby 2024-09-21T06:34:04.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:04 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T06:34:04.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:04 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:04.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:04 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:04.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:04 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:04.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:04 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:34:04.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:04 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:34:04.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:04 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:34:04.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:04 smithi100 ceph-mon[33401]: Deploying daemon mds.f on smithi203 2024-09-21T06:34:04.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:04 smithi100 ceph-mon[33401]: mds.? [v2:172.21.15.100:6834/1518007169,v1:172.21.15.100:6835/1518007169] up:boot 2024-09-21T06:34:04.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:04 smithi100 ceph-mon[33401]: fsmap 5 up:standby 2024-09-21T06:34:04.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:04 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T06:34:04.953 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:04 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:04.953 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:04 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:04.953 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:04 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:04.953 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:04 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:34:04.953 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:04 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:34:04.953 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:04 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:34:04.953 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:04 smithi203 ceph-mon[33735]: Deploying daemon mds.f on smithi203 2024-09-21T06:34:04.954 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:04 smithi203 ceph-mon[33735]: mds.? [v2:172.21.15.100:6834/1518007169,v1:172.21.15.100:6835/1518007169] up:boot 2024-09-21T06:34:04.954 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:04 smithi203 ceph-mon[33735]: fsmap 5 up:standby 2024-09-21T06:34:04.954 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:04 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T06:34:05.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.234+0000 7fc563f70640 1 -- 172.21.15.67:0/137957680 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc55c1083a0 msgr2=0x7fc55c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.234+0000 7fc563f70640 1 --2- 172.21.15.67:0/137957680 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc55c1083a0 0x7fc55c1087b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc54c009900 tx=0x7fc54c02a3f0 comp rx=0 tx=0).stop 2024-09-21T06:34:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.235+0000 7fc563f70640 1 -- 172.21.15.67:0/137957680 shutdown_connections 2024-09-21T06:34:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.235+0000 7fc563f70640 1 --2- 172.21.15.67:0/137957680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc55c108cf0 0x7fc55c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.235+0000 7fc563f70640 1 --2- 172.21.15.67:0/137957680 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc55c1083a0 0x7fc55c1087b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.235+0000 7fc563f70640 1 --2- 172.21.15.67:0/137957680 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc55c102a60 0x7fc55c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.235+0000 7fc563f70640 1 -- 172.21.15.67:0/137957680 >> 172.21.15.67:0/137957680 conn(0x7fc55c079730 msgr2=0x7fc55c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.235+0000 7fc563f70640 1 -- 172.21.15.67:0/137957680 shutdown_connections 2024-09-21T06:34:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.235+0000 7fc563f70640 1 -- 172.21.15.67:0/137957680 wait complete. 2024-09-21T06:34:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.235+0000 7fc563f70640 1 Processor -- start 2024-09-21T06:34:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.235+0000 7fc563f70640 1 -- start start 2024-09-21T06:34:05.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.236+0000 7fc563f70640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc55c102a60 0x7fc55c10fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:05.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.236+0000 7fc563f70640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc55c1083a0 0x7fc55c1102f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:05.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.236+0000 7fc563f70640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc55c108cf0 0x7fc55c110830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:05.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.236+0000 7fc563f70640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc55c113920 con 0x7fc55c1083a0 2024-09-21T06:34:05.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.236+0000 7fc563f70640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc55c1137a0 con 0x7fc55c102a60 2024-09-21T06:34:05.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.236+0000 7fc563f70640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc55c113aa0 con 0x7fc55c108cf0 2024-09-21T06:34:05.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.236+0000 7fc56150b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc55c1083a0 0x7fc55c1102f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:05.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.236+0000 7fc56150b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc55c1083a0 0x7fc55c1102f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57740/0 (socket says 172.21.15.67:57740) 2024-09-21T06:34:05.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.236+0000 7fc56150b640 1 -- 172.21.15.67:0/3724743071 learned_addr learned my addr 172.21.15.67:0/3724743071 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:34:05.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.236+0000 7fc561d0c640 1 --2- 172.21.15.67:0/3724743071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc55c102a60 0x7fc55c10fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:05.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.236+0000 7fc56250d640 1 --2- 172.21.15.67:0/3724743071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc55c108cf0 0x7fc55c110830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:05.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.236+0000 7fc56150b640 1 -- 172.21.15.67:0/3724743071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc55c108cf0 msgr2=0x7fc55c110830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:05.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.236+0000 7fc56150b640 1 --2- 172.21.15.67:0/3724743071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc55c108cf0 0x7fc55c110830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:05.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.236+0000 7fc56150b640 1 -- 172.21.15.67:0/3724743071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc55c102a60 msgr2=0x7fc55c10fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:05.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.236+0000 7fc56150b640 1 --2- 172.21.15.67:0/3724743071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc55c102a60 0x7fc55c10fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:05.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.236+0000 7fc56150b640 1 -- 172.21.15.67:0/3724743071 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc55c1a8600 con 0x7fc55c1083a0 2024-09-21T06:34:05.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.237+0000 7fc56150b640 1 --2- 172.21.15.67:0/3724743071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc55c1083a0 0x7fc55c1102f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc54c005940 tx=0x7fc54c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:05.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.237+0000 7fc552ffd640 1 -- 172.21.15.67:0/3724743071 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54c03f030 con 0x7fc55c1083a0 2024-09-21T06:34:05.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.237+0000 7fc563f70640 1 -- 172.21.15.67:0/3724743071 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc55c1a8890 con 0x7fc55c1083a0 2024-09-21T06:34:05.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.237+0000 7fc552ffd640 1 -- 172.21.15.67:0/3724743071 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc54c003c90 con 0x7fc55c1083a0 2024-09-21T06:34:05.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.237+0000 7fc552ffd640 1 -- 172.21.15.67:0/3724743071 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54c035070 con 0x7fc55c1083a0 2024-09-21T06:34:05.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.237+0000 7fc563f70640 1 -- 172.21.15.67:0/3724743071 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc55c1a8d70 con 0x7fc55c1083a0 2024-09-21T06:34:05.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.238+0000 7fc563f70640 1 -- 172.21.15.67:0/3724743071 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc55c103ce0 con 0x7fc55c1083a0 2024-09-21T06:34:05.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.240+0000 7fc552ffd640 1 -- 172.21.15.67:0/3724743071 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fc54c042030 con 0x7fc55c1083a0 2024-09-21T06:34:05.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.241+0000 7fc552ffd640 1 --2- 172.21.15.67:0/3724743071 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc5480b6510 0x7fc5480b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:05.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.241+0000 7fc561d0c640 1 --2- 172.21.15.67:0/3724743071 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc5480b6510 0x7fc5480b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:05.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.241+0000 7fc552ffd640 1 -- 172.21.15.67:0/3724743071 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fc54c1072c0 con 0x7fc55c1083a0 2024-09-21T06:34:05.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.241+0000 7fc561d0c640 1 --2- 172.21.15.67:0/3724743071 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc5480b6510 0x7fc5480b8a20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc55c111800 tx=0x7fc55800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:05.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.245+0000 7fc552ffd640 1 -- 172.21.15.67:0/3724743071 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc54c0cdff0 con 0x7fc55c1083a0 2024-09-21T06:34:05.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.470+0000 7fc563f70640 1 -- 172.21.15.67:0/3724743071 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc55c102e70 con 0x7fc55c1083a0 2024-09-21T06:34:05.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.470+0000 7fc552ffd640 1 -- 172.21.15.67:0/3724743071 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4744 (secure 0 0 0) 0x7fc54c0d23d0 con 0x7fc55c1083a0 2024-09-21T06:34:05.472 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:34:05.472 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":6,"btime":"2024-09-21T06:34:04:141627+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-21T06:34:05.472 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 6 2024-09-21T06:34:05.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.472+0000 7fc563f70640 1 -- 172.21.15.67:0/3724743071 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc5480b6510 msgr2=0x7fc5480b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:05.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.472+0000 7fc563f70640 1 --2- 172.21.15.67:0/3724743071 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc5480b6510 0x7fc5480b8a20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc55c111800 tx=0x7fc55800a2d0 comp rx=0 tx=0).stop 2024-09-21T06:34:05.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.472+0000 7fc563f70640 1 -- 172.21.15.67:0/3724743071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc55c1083a0 msgr2=0x7fc55c1102f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:05.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.472+0000 7fc563f70640 1 --2- 172.21.15.67:0/3724743071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc55c1083a0 0x7fc55c1102f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc54c005940 tx=0x7fc54c0042f0 comp rx=0 tx=0).stop 2024-09-21T06:34:05.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.473+0000 7fc563f70640 1 -- 172.21.15.67:0/3724743071 shutdown_connections 2024-09-21T06:34:05.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.473+0000 7fc563f70640 1 --2- 172.21.15.67:0/3724743071 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc5480b6510 0x7fc5480b8a20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:05.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.473+0000 7fc563f70640 1 --2- 172.21.15.67:0/3724743071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc55c108cf0 0x7fc55c110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:05.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.473+0000 7fc563f70640 1 --2- 172.21.15.67:0/3724743071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc55c1083a0 0x7fc55c1102f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:05.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.473+0000 7fc563f70640 1 --2- 172.21.15.67:0/3724743071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc55c102a60 0x7fc55c10fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:05.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.473+0000 7fc563f70640 1 -- 172.21.15.67:0/3724743071 >> 172.21.15.67:0/3724743071 conn(0x7fc55c079730 msgr2=0x7fc55c104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:05.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.473+0000 7fc563f70640 1 -- 172.21.15.67:0/3724743071 shutdown_connections 2024-09-21T06:34:05.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:05.473+0000 7fc563f70640 1 -- 172.21.15.67:0/3724743071 wait complete. 2024-09-21T06:34:05.521 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 06:34:05 smithi203 systemd[1]: Starting Ceph mds.f for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:34:05.779 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:05 smithi203 ceph-mon[33735]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:05.779 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:05 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3724743071' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:34:05.779 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 06:34:05 smithi203 podman[61383]: 2024-09-21 06:34:05.547775146 +0000 UTC m=+0.015899575 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:34:05.779 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 06:34:05 smithi203 podman[61383]: 2024-09-21 06:34:05.6543055 +0000 UTC m=+0.122429929 container create 34dd868b4ddb350322e752d8e712b9d4e3b152af18f3047372eaf504725090ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-f, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-21T06:34:05.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:05 smithi067 ceph-mon[31232]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:05.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:05 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3724743071' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:34:05.797 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:05 smithi100 ceph-mon[33401]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:05.797 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:05 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3724743071' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:34:06.232 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 06:34:05 smithi203 podman[61383]: 2024-09-21 06:34:05.915751104 +0000 UTC m=+0.383875541 container init 34dd868b4ddb350322e752d8e712b9d4e3b152af18f3047372eaf504725090ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-f, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:34:06.232 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 06:34:05 smithi203 podman[61383]: 2024-09-21 06:34:05.918706158 +0000 UTC m=+0.386830595 container start 34dd868b4ddb350322e752d8e712b9d4e3b152af18f3047372eaf504725090ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-f, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-21T06:34:06.232 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 06:34:05 smithi203 ceph-mds[61402]: -- 172.21.15.203:0/2118223702 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c8ce325860 con 0x55c8d24cda80 2024-09-21T06:34:06.232 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 06:34:05 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-f[61398]: starting mds.f at 2024-09-21T06:34:06.232 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 06:34:06 smithi203 bash[61383]: 34dd868b4ddb350322e752d8e712b9d4e3b152af18f3047372eaf504725090ef 2024-09-21T06:34:06.232 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 06:34:06 smithi203 systemd[1]: Started Ceph mds.f for 2a791768-77e2-11ef-baf5-efdc52797490. 2024-09-21T06:34:07.444 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:07 smithi067 ceph-mon[31232]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:07.444 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:07 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:07.444 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:07 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:07.444 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:07 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:07.444 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:07 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:34:07.444 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:07 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:34:07.444 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:07 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:34:07.444 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:07 smithi067 ceph-mon[31232]: Deploying daemon mds.c on smithi203 2024-09-21T06:34:07.444 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:07 smithi067 ceph-mon[31232]: mds.? [v2:172.21.15.203:6834/2689490358,v1:172.21.15.203:6835/2689490358] up:boot 2024-09-21T06:34:07.444 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:07 smithi067 ceph-mon[31232]: fsmap 6 up:standby 2024-09-21T06:34:07.444 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:07 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T06:34:07.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:07 smithi203 ceph-mon[33735]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:07.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:07 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:07.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:07 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:07.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:07 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:07.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:07 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:34:07.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:07 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:34:07.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:07 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:34:07.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:07 smithi203 ceph-mon[33735]: Deploying daemon mds.c on smithi203 2024-09-21T06:34:07.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:07 smithi203 ceph-mon[33735]: mds.? [v2:172.21.15.203:6834/2689490358,v1:172.21.15.203:6835/2689490358] up:boot 2024-09-21T06:34:07.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:07 smithi203 ceph-mon[33735]: fsmap 6 up:standby 2024-09-21T06:34:07.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:07 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T06:34:07.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:07 smithi100 ceph-mon[33401]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:07.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:07 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:07.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:07 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:07.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:07 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:07.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:07 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:34:07.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:07 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:34:07.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:07 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:34:07.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:07 smithi100 ceph-mon[33401]: Deploying daemon mds.c on smithi203 2024-09-21T06:34:07.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:07 smithi100 ceph-mon[33401]: mds.? [v2:172.21.15.203:6834/2689490358,v1:172.21.15.203:6835/2689490358] up:boot 2024-09-21T06:34:07.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:07 smithi100 ceph-mon[33401]: fsmap 6 up:standby 2024-09-21T06:34:07.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:07 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T06:34:08.182 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 06:34:07 smithi203 systemd[1]: Starting Ceph mds.c for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:34:08.195 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:34:08.471 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:34:08.472 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 06:34:08 smithi203 podman[61654]: 2024-09-21 06:34:08.083262512 +0000 UTC m=+0.016127979 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:34:08.472 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 06:34:08 smithi203 podman[61654]: 2024-09-21 06:34:08.210243513 +0000 UTC m=+0.143108981 container create f580d722c33d324aebff6f345c0278e5ea62c8b6bfd60553cb71f6b350436443 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-21T06:34:08.732 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 06:34:08 smithi203 podman[61654]: 2024-09-21 06:34:08.471789849 +0000 UTC m=+0.404655322 container init f580d722c33d324aebff6f345c0278e5ea62c8b6bfd60553cb71f6b350436443 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-21T06:34:08.732 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 06:34:08 smithi203 podman[61654]: 2024-09-21 06:34:08.474774505 +0000 UTC m=+0.407639981 container start f580d722c33d324aebff6f345c0278e5ea62c8b6bfd60553cb71f6b350436443 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-c, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T06:34:08.732 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 06:34:08 smithi203 ceph-mds[61673]: -- 172.21.15.203:0/3211923124 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55afcce4f860 con 0x55afd0ff7180 2024-09-21T06:34:08.732 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 06:34:08 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-c[61669]: starting mds.c at 2024-09-21T06:34:08.732 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 06:34:08 smithi203 bash[61654]: f580d722c33d324aebff6f345c0278e5ea62c8b6bfd60553cb71f6b350436443 2024-09-21T06:34:08.732 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 06:34:08 smithi203 systemd[1]: Started Ceph mds.c for 2a791768-77e2-11ef-baf5-efdc52797490. 2024-09-21T06:34:08.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.963+0000 7f81fddce640 1 -- 172.21.15.67:0/1672966547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f810ddf0 msgr2=0x7f81f8110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:08.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.963+0000 7f81fddce640 1 --2- 172.21.15.67:0/1672966547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f810ddf0 0x7f81f8110300 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f81f0009ca0 tx=0x7f81f002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:34:08.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.963+0000 7f81fddce640 1 -- 172.21.15.67:0/1672966547 shutdown_connections 2024-09-21T06:34:08.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.963+0000 7f81fddce640 1 --2- 172.21.15.67:0/1672966547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81f8105270 0x7f81f810d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:08.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.963+0000 7f81fddce640 1 --2- 172.21.15.67:0/1672966547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f810ddf0 0x7f81f8110300 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:08.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.963+0000 7f81fddce640 1 --2- 172.21.15.67:0/1672966547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81f8104840 0x7f81f8104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:08.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.963+0000 7f81fddce640 1 -- 172.21.15.67:0/1672966547 >> 172.21.15.67:0/1672966547 conn(0x7f81f8079730 msgr2=0x7f81f8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:08.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.963+0000 7f81fddce640 1 -- 172.21.15.67:0/1672966547 shutdown_connections 2024-09-21T06:34:08.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.963+0000 7f81fddce640 1 -- 172.21.15.67:0/1672966547 wait complete. 2024-09-21T06:34:08.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.964+0000 7f81fddce640 1 Processor -- start 2024-09-21T06:34:08.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.965+0000 7f81fddce640 1 -- start start 2024-09-21T06:34:08.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.965+0000 7f81fddce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81f8104840 0x7f81f819c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:08.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.965+0000 7f81fddce640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f8105270 0x7f81f819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:08.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.965+0000 7f81fddce640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81f810ddf0 0x7f81f81a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:08.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.965+0000 7f81fddce640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f81f8113990 con 0x7f81f8105270 2024-09-21T06:34:08.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.965+0000 7f81fddce640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f81f8113810 con 0x7f81f8104840 2024-09-21T06:34:08.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.965+0000 7f81fddce640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f81f8113b10 con 0x7f81f810ddf0 2024-09-21T06:34:08.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.965+0000 7f81f6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f8105270 0x7f81f819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:08.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.965+0000 7f81f6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f8105270 0x7f81f819cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60674/0 (socket says 172.21.15.67:60674) 2024-09-21T06:34:08.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.965+0000 7f81f6ffd640 1 -- 172.21.15.67:0/3078132988 learned_addr learned my addr 172.21.15.67:0/3078132988 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:34:08.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.965+0000 7f81f77fe640 1 --2- 172.21.15.67:0/3078132988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81f8104840 0x7f81f819c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:08.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.965+0000 7f81f7fff640 1 --2- 172.21.15.67:0/3078132988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81f810ddf0 0x7f81f81a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:08.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.965+0000 7f81f6ffd640 1 -- 172.21.15.67:0/3078132988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81f810ddf0 msgr2=0x7f81f81a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:08.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.965+0000 7f81f6ffd640 1 --2- 172.21.15.67:0/3078132988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81f810ddf0 0x7f81f81a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:08.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.965+0000 7f81f6ffd640 1 -- 172.21.15.67:0/3078132988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81f8104840 msgr2=0x7f81f819c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:08.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.965+0000 7f81f6ffd640 1 --2- 172.21.15.67:0/3078132988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81f8104840 0x7f81f819c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:08.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.965+0000 7f81f6ffd640 1 -- 172.21.15.67:0/3078132988 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f81a24d0 con 0x7f81f8105270 2024-09-21T06:34:08.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.965+0000 7f81f7fff640 1 --2- 172.21.15.67:0/3078132988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81f810ddf0 0x7f81f81a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:34:08.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.966+0000 7f81f6ffd640 1 --2- 172.21.15.67:0/3078132988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f8105270 0x7f81f819cbc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f81e800c8c0 tx=0x7f81e800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:08.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.966+0000 7f81f4ff9640 1 -- 172.21.15.67:0/3078132988 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8007bb0 con 0x7f81f8105270 2024-09-21T06:34:08.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.966+0000 7f81fddce640 1 -- 172.21.15.67:0/3078132988 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81f81a27c0 con 0x7f81f8105270 2024-09-21T06:34:08.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.966+0000 7f81f4ff9640 1 -- 172.21.15.67:0/3078132988 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f81e8007d50 con 0x7f81f8105270 2024-09-21T06:34:08.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.966+0000 7f81f4ff9640 1 -- 172.21.15.67:0/3078132988 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8018660 con 0x7f81f8105270 2024-09-21T06:34:08.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.966+0000 7f81fddce640 1 -- 172.21.15.67:0/3078132988 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81f81a8fe0 con 0x7f81f8105270 2024-09-21T06:34:08.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.967+0000 7f81fddce640 1 -- 172.21.15.67:0/3078132988 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81f8104c50 con 0x7f81f8105270 2024-09-21T06:34:08.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.969+0000 7f81f4ff9640 1 -- 172.21.15.67:0/3078132988 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f81e800fa00 con 0x7f81f8105270 2024-09-21T06:34:08.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.970+0000 7f81f4ff9640 1 --2- 172.21.15.67:0/3078132988 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f81e40b6540 0x7f81e40b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:08.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.970+0000 7f81f77fe640 1 --2- 172.21.15.67:0/3078132988 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f81e40b6540 0x7f81e40b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:08.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.970+0000 7f81f4ff9640 1 -- 172.21.15.67:0/3078132988 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f81e80e70b0 con 0x7f81f8105270 2024-09-21T06:34:08.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.970+0000 7f81f77fe640 1 --2- 172.21.15.67:0/3078132988 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f81e40b6540 0x7f81e40b8a50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f81f806bc00 tx=0x7f81e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:08.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:08.977+0000 7f81f4ff9640 1 -- 172.21.15.67:0/3078132988 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f81e8014030 con 0x7f81f8105270 2024-09-21T06:34:09.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:09.208+0000 7f81fddce640 1 -- 172.21.15.67:0/3078132988 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81f819d750 con 0x7f81f8105270 2024-09-21T06:34:09.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:09.209+0000 7f81f4ff9640 1 -- 172.21.15.67:0/3078132988 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6400 (secure 0 0 0) 0x7f81e80ade70 con 0x7f81f8105270 2024-09-21T06:34:09.211 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:34:09.211 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":8,"btime":"2024-09-21T06:34:09:067980+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-21T06:34:09.211 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 8 2024-09-21T06:34:09.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:09.211+0000 7f81fddce640 1 -- 172.21.15.67:0/3078132988 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f81e40b6540 msgr2=0x7f81e40b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:09.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:09.211+0000 7f81fddce640 1 --2- 172.21.15.67:0/3078132988 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f81e40b6540 0x7f81e40b8a50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f81f806bc00 tx=0x7f81e0005f90 comp rx=0 tx=0).stop 2024-09-21T06:34:09.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:09.211+0000 7f81fddce640 1 -- 172.21.15.67:0/3078132988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f8105270 msgr2=0x7f81f819cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:09.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:09.211+0000 7f81fddce640 1 --2- 172.21.15.67:0/3078132988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f8105270 0x7f81f819cbc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f81e800c8c0 tx=0x7f81e800cd80 comp rx=0 tx=0).stop 2024-09-21T06:34:09.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:09.211+0000 7f81fddce640 1 -- 172.21.15.67:0/3078132988 shutdown_connections 2024-09-21T06:34:09.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:09.211+0000 7f81fddce640 1 --2- 172.21.15.67:0/3078132988 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f81e40b6540 0x7f81e40b8a50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:09.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:09.211+0000 7f81fddce640 1 --2- 172.21.15.67:0/3078132988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81f810ddf0 0x7f81f81a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:09.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:09.211+0000 7f81fddce640 1 --2- 172.21.15.67:0/3078132988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f8105270 0x7f81f819cbc0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:09.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:09.211+0000 7f81fddce640 1 --2- 172.21.15.67:0/3078132988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81f8104840 0x7f81f819c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:09.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:09.211+0000 7f81fddce640 1 -- 172.21.15.67:0/3078132988 >> 172.21.15.67:0/3078132988 conn(0x7f81f8079730 msgr2=0x7f81f8100580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:09.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:09.211+0000 7f81fddce640 1 -- 172.21.15.67:0/3078132988 shutdown_connections 2024-09-21T06:34:09.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:09.211+0000 7f81fddce640 1 -- 172.21.15.67:0/3078132988 wait complete. 2024-09-21T06:34:09.288 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:09 smithi100 ceph-mon[33401]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:09.288 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:09 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:09.288 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:09 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:09.289 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:09 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:09.289 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:09 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:34:09.289 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:09 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:34:09.289 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:09 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:34:09.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:09 smithi203 ceph-mon[33735]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:09.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:09 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:09.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:09 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:09.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:09 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:09.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:09 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:34:09.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:09 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:34:09.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:09 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:34:09.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:09 smithi067 ceph-mon[31232]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:09.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:09 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:09.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:09 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:09.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:09 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:09.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:09 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:34:09.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:09 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:34:09.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:09 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:34:10.291 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:10 smithi067 ceph-mon[31232]: Deploying daemon mds.h on smithi100 2024-09-21T06:34:10.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:10 smithi067 ceph-mon[31232]: mds.? [v2:172.21.15.203:6836/1426013144,v1:172.21.15.203:6837/1426013144] up:boot 2024-09-21T06:34:10.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:10 smithi067 ceph-mon[31232]: fsmap 7 up:standby 2024-09-21T06:34:10.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:10 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T06:34:10.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:10 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3078132988' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:34:10.403 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:10 smithi100 ceph-mon[33401]: Deploying daemon mds.h on smithi100 2024-09-21T06:34:10.403 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:10 smithi100 ceph-mon[33401]: mds.? [v2:172.21.15.203:6836/1426013144,v1:172.21.15.203:6837/1426013144] up:boot 2024-09-21T06:34:10.403 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:10 smithi100 ceph-mon[33401]: fsmap 7 up:standby 2024-09-21T06:34:10.403 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:10 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T06:34:10.403 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:10 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3078132988' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:34:10.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:10 smithi203 ceph-mon[33735]: Deploying daemon mds.h on smithi100 2024-09-21T06:34:10.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:10 smithi203 ceph-mon[33735]: mds.? [v2:172.21.15.203:6836/1426013144,v1:172.21.15.203:6837/1426013144] up:boot 2024-09-21T06:34:10.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:10 smithi203 ceph-mon[33735]: fsmap 7 up:standby 2024-09-21T06:34:10.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:10 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T06:34:10.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:10 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3078132988' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:34:10.791 INFO:journalctl@ceph.mds.h.smithi100.stdout:Sep 21 06:34:10 smithi100 systemd[1]: Starting Ceph mds.h for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:34:11.054 INFO:journalctl@ceph.mds.h.smithi100.stdout:Sep 21 06:34:10 smithi100 podman[60785]: 2024-09-21 06:34:10.693163675 +0000 UTC m=+0.015836753 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:34:11.054 INFO:journalctl@ceph.mds.h.smithi100.stdout:Sep 21 06:34:10 smithi100 podman[60785]: 2024-09-21 06:34:10.803925024 +0000 UTC m=+0.126598085 container create fcaaf2d7561b4d11ce076fbc61fda482b1ce84a67750d32d22c8e671b8d6d257 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-h, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-21T06:34:11.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:11 smithi203 ceph-mon[33735]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:11.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:11 smithi067 ceph-mon[31232]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:11.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:11 smithi100 ceph-mon[33401]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:11.553 INFO:journalctl@ceph.mds.h.smithi100.stdout:Sep 21 06:34:11 smithi100 podman[60785]: 2024-09-21 06:34:11.057227506 +0000 UTC m=+0.379900576 container init fcaaf2d7561b4d11ce076fbc61fda482b1ce84a67750d32d22c8e671b8d6d257 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-21T06:34:11.554 INFO:journalctl@ceph.mds.h.smithi100.stdout:Sep 21 06:34:11 smithi100 podman[60785]: 2024-09-21 06:34:11.060022076 +0000 UTC m=+0.382695144 container start fcaaf2d7561b4d11ce076fbc61fda482b1ce84a67750d32d22c8e671b8d6d257 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-21T06:34:11.554 INFO:journalctl@ceph.mds.h.smithi100.stdout:Sep 21 06:34:11 smithi100 ceph-mds[60804]: -- 172.21.15.100:0/1309041769 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fe88843c20 con 0x55fe8c9eb600 2024-09-21T06:34:11.554 INFO:journalctl@ceph.mds.h.smithi100.stdout:Sep 21 06:34:11 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-h[60800]: starting mds.h at 2024-09-21T06:34:11.554 INFO:journalctl@ceph.mds.h.smithi100.stdout:Sep 21 06:34:11 smithi100 bash[60785]: fcaaf2d7561b4d11ce076fbc61fda482b1ce84a67750d32d22c8e671b8d6d257 2024-09-21T06:34:11.554 INFO:journalctl@ceph.mds.h.smithi100.stdout:Sep 21 06:34:11 smithi100 systemd[1]: Started Ceph mds.h for 2a791768-77e2-11ef-baf5-efdc52797490. 2024-09-21T06:34:11.946 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:34:12.222 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:34:12.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:12 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:12.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:12 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:12.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:12 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:12.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:12 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:34:12.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:12 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:34:12.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:12 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:34:12.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:12 smithi067 ceph-mon[31232]: Deploying daemon mds.i on smithi203 2024-09-21T06:34:12.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:12 smithi067 ceph-mon[31232]: mds.? [v2:172.21.15.100:6836/259395872,v1:172.21.15.100:6837/259395872] up:boot 2024-09-21T06:34:12.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:12 smithi067 ceph-mon[31232]: fsmap 8 up:standby 2024-09-21T06:34:12.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:12 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T06:34:12.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:12 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:12.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:12 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:12.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:12 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:12.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:12 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:34:12.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:12 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:34:12.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:12 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:34:12.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:12 smithi203 ceph-mon[33735]: Deploying daemon mds.i on smithi203 2024-09-21T06:34:12.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:12 smithi203 ceph-mon[33735]: mds.? [v2:172.21.15.100:6836/259395872,v1:172.21.15.100:6837/259395872] up:boot 2024-09-21T06:34:12.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:12 smithi203 ceph-mon[33735]: fsmap 8 up:standby 2024-09-21T06:34:12.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:12 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T06:34:12.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:12 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:12.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:12 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:12.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:12 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:12.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:12 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:34:12.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:12 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:34:12.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:12 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:34:12.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:12 smithi100 ceph-mon[33401]: Deploying daemon mds.i on smithi203 2024-09-21T06:34:12.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:12 smithi100 ceph-mon[33401]: mds.? [v2:172.21.15.100:6836/259395872,v1:172.21.15.100:6837/259395872] up:boot 2024-09-21T06:34:12.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:12 smithi100 ceph-mon[33401]: fsmap 8 up:standby 2024-09-21T06:34:12.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:12 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T06:34:12.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.736+0000 7f46b5088640 1 -- 172.21.15.67:0/3602771332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46b01049b0 msgr2=0x7f46b0104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:12.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.736+0000 7f46b5088640 1 --2- 172.21.15.67:0/3602771332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46b01049b0 0x7f46b0104dc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f46a8009900 tx=0x7f46a802a410 comp rx=0 tx=0).stop 2024-09-21T06:34:12.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.736+0000 7f46b5088640 1 -- 172.21.15.67:0/3602771332 shutdown_connections 2024-09-21T06:34:12.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.736+0000 7f46b5088640 1 --2- 172.21.15.67:0/3602771332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46b010de10 0x7f46b0110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:12.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.736+0000 7f46b5088640 1 --2- 172.21.15.67:0/3602771332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46b0105300 0x7f46b010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:12.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.736+0000 7f46b5088640 1 --2- 172.21.15.67:0/3602771332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46b01049b0 0x7f46b0104dc0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:12.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.736+0000 7f46b5088640 1 -- 172.21.15.67:0/3602771332 >> 172.21.15.67:0/3602771332 conn(0x7f46b0079730 msgr2=0x7f46b0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:12.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.736+0000 7f46b5088640 1 -- 172.21.15.67:0/3602771332 shutdown_connections 2024-09-21T06:34:12.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.736+0000 7f46b5088640 1 -- 172.21.15.67:0/3602771332 wait complete. 2024-09-21T06:34:12.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.737+0000 7f46b5088640 1 Processor -- start 2024-09-21T06:34:12.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.737+0000 7f46b5088640 1 -- start start 2024-09-21T06:34:12.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.737+0000 7f46b5088640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46b01049b0 0x7f46b019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:12.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.737+0000 7f46b5088640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46b0105300 0x7f46b019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:12.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.737+0000 7f46b5088640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46b010de10 0x7f46b019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:12.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.737+0000 7f46b5088640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f46b0113920 con 0x7f46b010de10 2024-09-21T06:34:12.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.737+0000 7f46b5088640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f46b01137a0 con 0x7f46b0105300 2024-09-21T06:34:12.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.737+0000 7f46b5088640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f46b0113aa0 con 0x7f46b01049b0 2024-09-21T06:34:12.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.737+0000 7f46af59e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46b010de10 0x7f46b019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:12.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.738+0000 7f46af59e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46b010de10 0x7f46b019fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60692/0 (socket says 172.21.15.67:60692) 2024-09-21T06:34:12.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.738+0000 7f46af59e640 1 -- 172.21.15.67:0/2824506302 learned_addr learned my addr 172.21.15.67:0/2824506302 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:34:12.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.738+0000 7f46ae59c640 1 --2- 172.21.15.67:0/2824506302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46b0105300 0x7f46b019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:12.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.738+0000 7f46aed9d640 1 --2- 172.21.15.67:0/2824506302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46b01049b0 0x7f46b019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:12.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.738+0000 7f46af59e640 1 -- 172.21.15.67:0/2824506302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46b01049b0 msgr2=0x7f46b019a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:12.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.738+0000 7f46af59e640 1 --2- 172.21.15.67:0/2824506302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46b01049b0 0x7f46b019a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:12.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.738+0000 7f46af59e640 1 -- 172.21.15.67:0/2824506302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46b0105300 msgr2=0x7f46b019a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:12.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.738+0000 7f46af59e640 1 --2- 172.21.15.67:0/2824506302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46b0105300 0x7f46b019a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:12.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.738+0000 7f46af59e640 1 -- 172.21.15.67:0/2824506302 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b01a02a0 con 0x7f46b010de10 2024-09-21T06:34:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.738+0000 7f46ae59c640 1 --2- 172.21.15.67:0/2824506302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46b0105300 0x7f46b019a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.738+0000 7f46af59e640 1 --2- 172.21.15.67:0/2824506302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46b010de10 0x7f46b019fc30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f46a000eaa0 tx=0x7f46a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.738+0000 7f46a7fff640 1 -- 172.21.15.67:0/2824506302 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a000cdb0 con 0x7f46b010de10 2024-09-21T06:34:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.738+0000 7f46b5088640 1 -- 172.21.15.67:0/2824506302 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b01a0590 con 0x7f46b010de10 2024-09-21T06:34:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.738+0000 7f46a7fff640 1 -- 172.21.15.67:0/2824506302 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46a0005020 con 0x7f46b010de10 2024-09-21T06:34:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.738+0000 7f46a7fff640 1 -- 172.21.15.67:0/2824506302 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a00116d0 con 0x7f46b010de10 2024-09-21T06:34:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.738+0000 7f46b5088640 1 -- 172.21.15.67:0/2824506302 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b01a6db0 con 0x7f46b010de10 2024-09-21T06:34:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.740+0000 7f46b5088640 1 -- 172.21.15.67:0/2824506302 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46b0104dc0 con 0x7f46b010de10 2024-09-21T06:34:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.741+0000 7f46a7fff640 1 -- 172.21.15.67:0/2824506302 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f46a00051d0 con 0x7f46b010de10 2024-09-21T06:34:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.742+0000 7f46a7fff640 1 --2- 172.21.15.67:0/2824506302 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f469c0b6540 0x7f469c0b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.742+0000 7f46aed9d640 1 --2- 172.21.15.67:0/2824506302 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f469c0b6540 0x7f469c0b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.742+0000 7f46a7fff640 1 -- 172.21.15.67:0/2824506302 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f46a00e73f0 con 0x7f46b010de10 2024-09-21T06:34:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.742+0000 7f46aed9d640 1 --2- 172.21.15.67:0/2824506302 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f469c0b6540 0x7f469c0b8a50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f46a8006010 tx=0x7f46a8032c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:12.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.746+0000 7f46a7fff640 1 -- 172.21.15.67:0/2824506302 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f46a0016030 con 0x7f46b010de10 2024-09-21T06:34:12.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.971+0000 7f46b5088640 1 -- 172.21.15.67:0/2824506302 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46b0108f70 con 0x7f46b010de10 2024-09-21T06:34:12.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.972+0000 7f46a7fff640 1 -- 172.21.15.67:0/2824506302 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7225 (secure 0 0 0) 0x7f46a00ae120 con 0x7f46b010de10 2024-09-21T06:34:12.974 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:34:12.974 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":9,"btime":"2024-09-21T06:34:12:081097+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24641,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-21T06:34:12.974 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 9 2024-09-21T06:34:12.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.974+0000 7f46b5088640 1 -- 172.21.15.67:0/2824506302 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f469c0b6540 msgr2=0x7f469c0b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:12.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.974+0000 7f46b5088640 1 --2- 172.21.15.67:0/2824506302 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f469c0b6540 0x7f469c0b8a50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f46a8006010 tx=0x7f46a8032c80 comp rx=0 tx=0).stop 2024-09-21T06:34:12.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.974+0000 7f46b5088640 1 -- 172.21.15.67:0/2824506302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46b010de10 msgr2=0x7f46b019fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:12.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.974+0000 7f46b5088640 1 --2- 172.21.15.67:0/2824506302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46b010de10 0x7f46b019fc30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f46a000eaa0 tx=0x7f46a000ef60 comp rx=0 tx=0).stop 2024-09-21T06:34:12.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.974+0000 7f46b5088640 1 -- 172.21.15.67:0/2824506302 shutdown_connections 2024-09-21T06:34:12.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.974+0000 7f46b5088640 1 --2- 172.21.15.67:0/2824506302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46b0105300 0x7f46b019a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:12.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.974+0000 7f46b5088640 1 --2- 172.21.15.67:0/2824506302 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f469c0b6540 0x7f469c0b8a50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:12.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.974+0000 7f46b5088640 1 --2- 172.21.15.67:0/2824506302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46b010de10 0x7f46b019fc30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:12.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.974+0000 7f46b5088640 1 --2- 172.21.15.67:0/2824506302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46b01049b0 0x7f46b019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:12.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.974+0000 7f46b5088640 1 -- 172.21.15.67:0/2824506302 >> 172.21.15.67:0/2824506302 conn(0x7f46b0079730 msgr2=0x7f46b00ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:12.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.975+0000 7f46b5088640 1 -- 172.21.15.67:0/2824506302 shutdown_connections 2024-09-21T06:34:12.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:12.975+0000 7f46b5088640 1 -- 172.21.15.67:0/2824506302 wait complete. 2024-09-21T06:34:13.233 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 06:34:13 smithi203 systemd[1]: Starting Ceph mds.i for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:34:13.486 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:13 smithi203 ceph-mon[33735]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:13.487 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:13 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2824506302' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:34:13.487 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 06:34:13 smithi203 podman[61926]: 2024-09-21 06:34:13.256631822 +0000 UTC m=+0.016358663 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:34:13.487 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 06:34:13 smithi203 podman[61926]: 2024-09-21 06:34:13.373382311 +0000 UTC m=+0.133109141 container create 440a27cc8a72963ef3f84943f4fc4e9591d4a0d47a29b14e28ddde07ee9b93f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-i, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:34:13.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:13 smithi067 ceph-mon[31232]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:13.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:13 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2824506302' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:34:13.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:13 smithi100 ceph-mon[33401]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:13.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:13 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2824506302' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:34:13.982 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 06:34:13 smithi203 podman[61926]: 2024-09-21 06:34:13.626847789 +0000 UTC m=+0.386574627 container init 440a27cc8a72963ef3f84943f4fc4e9591d4a0d47a29b14e28ddde07ee9b93f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-21T06:34:13.982 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 06:34:13 smithi203 podman[61926]: 2024-09-21 06:34:13.629699862 +0000 UTC m=+0.389426698 container start 440a27cc8a72963ef3f84943f4fc4e9591d4a0d47a29b14e28ddde07ee9b93f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-i, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-21T06:34:13.982 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 06:34:13 smithi203 ceph-mds[61945]: -- 172.21.15.203:0/1411146661 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5602ad691860 con 0x5602b1839a80 2024-09-21T06:34:13.982 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 06:34:13 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-i[61941]: starting mds.i at 2024-09-21T06:34:13.982 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 06:34:13 smithi203 bash[61926]: 440a27cc8a72963ef3f84943f4fc4e9591d4a0d47a29b14e28ddde07ee9b93f3 2024-09-21T06:34:13.983 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 06:34:13 smithi203 systemd[1]: Started Ceph mds.i for 2a791768-77e2-11ef-baf5-efdc52797490. 2024-09-21T06:34:15.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:14 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:15.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:14 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:15.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:14 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:15.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:14 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:34:15.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:14 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:34:15.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:14 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:34:15.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:14 smithi067 ceph-mon[31232]: Deploying daemon mds.g on smithi067 2024-09-21T06:34:15.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:14 smithi067 ceph-mon[31232]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:15.043 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:14 smithi067 ceph-mon[31232]: mds.? [v2:172.21.15.203:6838/2443174511,v1:172.21.15.203:6839/2443174511] up:boot 2024-09-21T06:34:15.043 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:14 smithi067 ceph-mon[31232]: fsmap 9 up:standby 2024-09-21T06:34:15.043 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:14 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T06:34:15.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:14 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:15.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:14 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:15.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:14 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:15.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:14 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:34:15.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:14 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:34:15.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:14 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:34:15.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:14 smithi100 ceph-mon[33401]: Deploying daemon mds.g on smithi067 2024-09-21T06:34:15.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:14 smithi100 ceph-mon[33401]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:15.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:14 smithi100 ceph-mon[33401]: mds.? [v2:172.21.15.203:6838/2443174511,v1:172.21.15.203:6839/2443174511] up:boot 2024-09-21T06:34:15.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:14 smithi100 ceph-mon[33401]: fsmap 9 up:standby 2024-09-21T06:34:15.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:14 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T06:34:15.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:14 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:15.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:14 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:15.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:14 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:15.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:14 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:34:15.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:14 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:34:15.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:14 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:34:15.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:14 smithi203 ceph-mon[33735]: Deploying daemon mds.g on smithi067 2024-09-21T06:34:15.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:14 smithi203 ceph-mon[33735]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:15.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:14 smithi203 ceph-mon[33735]: mds.? [v2:172.21.15.203:6838/2443174511,v1:172.21.15.203:6839/2443174511] up:boot 2024-09-21T06:34:15.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:14 smithi203 ceph-mon[33735]: fsmap 9 up:standby 2024-09-21T06:34:15.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:14 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T06:34:15.720 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:34:16.363 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:34:16.486 INFO:journalctl@ceph.mds.g.smithi067.stdout:Sep 21 06:34:16 smithi067 systemd[1]: Started Ceph mds.g for 2a791768-77e2-11ef-baf5-efdc52797490. 2024-09-21T06:34:16.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.826+0000 7f0350733640 1 -- 172.21.15.67:0/3281796549 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03481083c0 msgr2=0x7f03481087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.826+0000 7f0350733640 1 --2- 172.21.15.67:0/3281796549 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03481083c0 0x7f03481087d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f033c009900 tx=0x7f033c02a410 comp rx=0 tx=0).stop 2024-09-21T06:34:16.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.826+0000 7f0350733640 1 -- 172.21.15.67:0/3281796549 shutdown_connections 2024-09-21T06:34:16.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.826+0000 7f0350733640 1 --2- 172.21.15.67:0/3281796549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0348108d10 0x7f034810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.826+0000 7f0350733640 1 --2- 172.21.15.67:0/3281796549 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03481083c0 0x7f03481087d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.826+0000 7f0350733640 1 --2- 172.21.15.67:0/3281796549 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0348102a80 0x7f0348102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.826+0000 7f0350733640 1 -- 172.21.15.67:0/3281796549 >> 172.21.15.67:0/3281796549 conn(0x7f0348079740 msgr2=0x7f0348079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:16.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.827+0000 7f0350733640 1 -- 172.21.15.67:0/3281796549 shutdown_connections 2024-09-21T06:34:16.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.827+0000 7f0350733640 1 -- 172.21.15.67:0/3281796549 wait complete. 2024-09-21T06:34:16.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.827+0000 7f0350733640 1 Processor -- start 2024-09-21T06:34:16.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.827+0000 7f0350733640 1 -- start start 2024-09-21T06:34:16.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.827+0000 7f0350733640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0348102a80 0x7f034819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.827+0000 7f0350733640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03481083c0 0x7f034819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.827+0000 7f0350733640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0348108d10 0x7f034819fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.827+0000 7f0350733640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0348113950 con 0x7f03481083c0 2024-09-21T06:34:16.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.827+0000 7f0350733640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f03481137d0 con 0x7f0348102a80 2024-09-21T06:34:16.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.827+0000 7f0350733640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0348113ad0 con 0x7f0348108d10 2024-09-21T06:34:16.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.828+0000 7f034dcce640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03481083c0 0x7f034819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.828+0000 7f034dcce640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03481083c0 0x7f034819a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60732/0 (socket says 172.21.15.67:60732) 2024-09-21T06:34:16.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.828+0000 7f034dcce640 1 -- 172.21.15.67:0/2688781004 learned_addr learned my addr 172.21.15.67:0/2688781004 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:34:16.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.828+0000 7f034e4cf640 1 --2- 172.21.15.67:0/2688781004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0348102a80 0x7f034819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.828+0000 7f034ecd0640 1 --2- 172.21.15.67:0/2688781004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0348108d10 0x7f034819fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.828+0000 7f034dcce640 1 -- 172.21.15.67:0/2688781004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0348108d10 msgr2=0x7f034819fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.828+0000 7f034dcce640 1 --2- 172.21.15.67:0/2688781004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0348108d10 0x7f034819fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.828+0000 7f034dcce640 1 -- 172.21.15.67:0/2688781004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0348102a80 msgr2=0x7f034819a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:16.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.828+0000 7f034dcce640 1 --2- 172.21.15.67:0/2688781004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0348102a80 0x7f034819a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:16.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.828+0000 7f034dcce640 1 -- 172.21.15.67:0/2688781004 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03481a0290 con 0x7f03481083c0 2024-09-21T06:34:16.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.828+0000 7f034e4cf640 1 --2- 172.21.15.67:0/2688781004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0348102a80 0x7f034819a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:16.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.828+0000 7f034ecd0640 1 --2- 172.21.15.67:0/2688781004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0348108d10 0x7f034819fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:16.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.828+0000 7f034dcce640 1 --2- 172.21.15.67:0/2688781004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03481083c0 0x7f034819a980 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f033c005940 tx=0x7f033c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:16.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.828+0000 7f033b7fe640 1 -- 172.21.15.67:0/2688781004 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c03f030 con 0x7f03481083c0 2024-09-21T06:34:16.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.828+0000 7f033b7fe640 1 -- 172.21.15.67:0/2688781004 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f033c003c90 con 0x7f03481083c0 2024-09-21T06:34:16.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.828+0000 7f0350733640 1 -- 172.21.15.67:0/2688781004 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03481a0520 con 0x7f03481083c0 2024-09-21T06:34:16.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.828+0000 7f033b7fe640 1 -- 172.21.15.67:0/2688781004 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c035070 con 0x7f03481083c0 2024-09-21T06:34:16.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.828+0000 7f0350733640 1 -- 172.21.15.67:0/2688781004 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03481a0a00 con 0x7f03481083c0 2024-09-21T06:34:16.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.829+0000 7f0350733640 1 -- 172.21.15.67:0/2688781004 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0348103d00 con 0x7f03481083c0 2024-09-21T06:34:16.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.831+0000 7f033b7fe640 1 -- 172.21.15.67:0/2688781004 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f033c042030 con 0x7f03481083c0 2024-09-21T06:34:16.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.832+0000 7f033b7fe640 1 --2- 172.21.15.67:0/2688781004 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f03340b6540 0x7f03340b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:16.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.832+0000 7f034e4cf640 1 --2- 172.21.15.67:0/2688781004 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f03340b6540 0x7f03340b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:16.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.832+0000 7f033b7fe640 1 -- 172.21.15.67:0/2688781004 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f033c1077e0 con 0x7f03481083c0 2024-09-21T06:34:16.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.832+0000 7f034e4cf640 1 --2- 172.21.15.67:0/2688781004 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f03340b6540 0x7f03340b8a50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0344005fd0 tx=0x7f0344009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:16.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:16.835+0000 7f033b7fe640 1 -- 172.21.15.67:0/2688781004 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f033c0ce510 con 0x7f03481083c0 2024-09-21T06:34:17.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:17.085+0000 7f0350733640 1 -- 172.21.15.67:0/2688781004 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f034819b510 con 0x7f03481083c0 2024-09-21T06:34:17.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:17.086+0000 7f033b7fe640 1 -- 172.21.15.67:0/2688781004 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8881 (secure 0 0 0) 0x7f034819b510 con 0x7f03481083c0 2024-09-21T06:34:17.088 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:34:17.088 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":11,"btime":"2024-09-21T06:34:16:371451+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24641,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-21T06:34:17.088 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 11 2024-09-21T06:34:17.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:17.088+0000 7f0350733640 1 -- 172.21.15.67:0/2688781004 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f03340b6540 msgr2=0x7f03340b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:17.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:17.088+0000 7f0350733640 1 --2- 172.21.15.67:0/2688781004 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f03340b6540 0x7f03340b8a50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0344005fd0 tx=0x7f0344009600 comp rx=0 tx=0).stop 2024-09-21T06:34:17.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:17.088+0000 7f0350733640 1 -- 172.21.15.67:0/2688781004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03481083c0 msgr2=0x7f034819a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:17.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:17.088+0000 7f0350733640 1 --2- 172.21.15.67:0/2688781004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03481083c0 0x7f034819a980 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f033c005940 tx=0x7f033c0042f0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:17.088+0000 7f0350733640 1 -- 172.21.15.67:0/2688781004 shutdown_connections 2024-09-21T06:34:17.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:17.088+0000 7f0350733640 1 --2- 172.21.15.67:0/2688781004 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f03340b6540 0x7f03340b8a50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:17.088+0000 7f0350733640 1 --2- 172.21.15.67:0/2688781004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0348108d10 0x7f034819fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:17.088+0000 7f0350733640 1 --2- 172.21.15.67:0/2688781004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03481083c0 0x7f034819a980 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:17.088+0000 7f0350733640 1 --2- 172.21.15.67:0/2688781004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0348102a80 0x7f034819a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:17.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:17.088+0000 7f0350733640 1 -- 172.21.15.67:0/2688781004 >> 172.21.15.67:0/2688781004 conn(0x7f0348079740 msgr2=0x7f03481044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:17.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:17.088+0000 7f0350733640 1 -- 172.21.15.67:0/2688781004 shutdown_connections 2024-09-21T06:34:17.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:17.088+0000 7f0350733640 1 -- 172.21.15.67:0/2688781004 wait complete. 2024-09-21T06:34:17.609 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:17 smithi067 ceph-mon[31232]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:17.610 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:17 smithi067 ceph-mon[31232]: mds.? [v2:172.21.15.67:6838/1168518926,v1:172.21.15.67:6839/1168518926] up:boot 2024-09-21T06:34:17.610 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:17 smithi067 ceph-mon[31232]: fsmap 10 up:standby 2024-09-21T06:34:17.610 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:17 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T06:34:17.610 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:17 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:17.610 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:17 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:17.610 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:17 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:17.611 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:17 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:34:17.611 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:17 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:34:17.611 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:17 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:34:17.611 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:17 smithi067 ceph-mon[31232]: Deploying daemon mds.a on smithi067 2024-09-21T06:34:17.611 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:17 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2688781004' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:34:17.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:17 smithi203 ceph-mon[33735]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:17.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:17 smithi203 ceph-mon[33735]: mds.? [v2:172.21.15.67:6838/1168518926,v1:172.21.15.67:6839/1168518926] up:boot 2024-09-21T06:34:17.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:17 smithi203 ceph-mon[33735]: fsmap 10 up:standby 2024-09-21T06:34:17.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:17 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T06:34:17.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:17 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:17.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:17 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:17.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:17 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:17.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:17 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:34:17.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:17 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:34:17.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:17 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:34:17.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:17 smithi203 ceph-mon[33735]: Deploying daemon mds.a on smithi067 2024-09-21T06:34:17.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:17 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2688781004' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:34:17.758 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:17 smithi100 ceph-mon[33401]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:17.758 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:17 smithi100 ceph-mon[33401]: mds.? [v2:172.21.15.67:6838/1168518926,v1:172.21.15.67:6839/1168518926] up:boot 2024-09-21T06:34:17.758 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:17 smithi100 ceph-mon[33401]: fsmap 10 up:standby 2024-09-21T06:34:17.758 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:17 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T06:34:17.758 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:17 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:17.758 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:17 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:17.758 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:17 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:17.758 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:17 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:34:17.758 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:17 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:34:17.758 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:17 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:34:17.758 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:17 smithi100 ceph-mon[33401]: Deploying daemon mds.a on smithi067 2024-09-21T06:34:17.758 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:17 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2688781004' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:34:18.823 INFO:journalctl@ceph.mds.a.smithi067.stdout:Sep 21 06:34:18 smithi067 systemd[1]: Starting Ceph mds.a for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:34:19.292 INFO:journalctl@ceph.mds.a.smithi067.stdout:Sep 21 06:34:19 smithi067 podman[72110]: 2024-09-21 06:34:18.938944933 +0000 UTC m=+0.016506556 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:34:19.292 INFO:journalctl@ceph.mds.a.smithi067.stdout:Sep 21 06:34:19 smithi067 podman[72110]: 2024-09-21 06:34:19.08364236 +0000 UTC m=+0.161203986 container create 3ecf1f8453a69c9ee050d25ff411b4406bf4410e5593f853196fab42396e7e2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:34:19.610 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:19 smithi067 ceph-mon[31232]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:19.610 INFO:journalctl@ceph.mds.a.smithi067.stdout:Sep 21 06:34:19 smithi067 podman[72110]: 2024-09-21 06:34:19.345184461 +0000 UTC m=+0.422746085 container init 3ecf1f8453a69c9ee050d25ff411b4406bf4410e5593f853196fab42396e7e2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-21T06:34:19.610 INFO:journalctl@ceph.mds.a.smithi067.stdout:Sep 21 06:34:19 smithi067 podman[72110]: 2024-09-21 06:34:19.348139311 +0000 UTC m=+0.425700935 container start 3ecf1f8453a69c9ee050d25ff411b4406bf4410e5593f853196fab42396e7e2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-a, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-21T06:34:19.610 INFO:journalctl@ceph.mds.a.smithi067.stdout:Sep 21 06:34:19 smithi067 ceph-mds[72129]: -- 172.21.15.67:0/2157417397 <== mon.0 v2:172.21.15.67:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d072e4b860 con 0x55d076ff3600 2024-09-21T06:34:19.610 INFO:journalctl@ceph.mds.a.smithi067.stdout:Sep 21 06:34:19 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-a[72125]: starting mds.a at 2024-09-21T06:34:19.610 INFO:journalctl@ceph.mds.a.smithi067.stdout:Sep 21 06:34:19 smithi067 bash[72110]: 3ecf1f8453a69c9ee050d25ff411b4406bf4410e5593f853196fab42396e7e2a 2024-09-21T06:34:19.610 INFO:journalctl@ceph.mds.a.smithi067.stdout:Sep 21 06:34:19 smithi067 systemd[1]: Started Ceph mds.a for 2a791768-77e2-11ef-baf5-efdc52797490. 2024-09-21T06:34:19.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:19 smithi203 ceph-mon[33735]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:19.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:19 smithi100 ceph-mon[33401]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:20.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:20 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:20.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:20 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:20.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:20 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:20.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:20 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:34:20.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:20 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:34:20.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:20 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:34:20.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:20 smithi067 ceph-mon[31232]: Deploying daemon mds.b on smithi100 2024-09-21T06:34:20.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:20 smithi067 ceph-mon[31232]: mds.? [v2:172.21.15.67:6840/2331956394,v1:172.21.15.67:6841/2331956394] up:boot 2024-09-21T06:34:20.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:20 smithi067 ceph-mon[31232]: fsmap 11 up:standby 2024-09-21T06:34:20.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:20 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T06:34:20.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:20 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:20.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:20 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:20.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:20 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:20.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:20 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:34:20.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:20 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:34:20.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:20 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:34:20.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:20 smithi100 ceph-mon[33401]: Deploying daemon mds.b on smithi100 2024-09-21T06:34:20.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:20 smithi100 ceph-mon[33401]: mds.? [v2:172.21.15.67:6840/2331956394,v1:172.21.15.67:6841/2331956394] up:boot 2024-09-21T06:34:20.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:20 smithi100 ceph-mon[33401]: fsmap 11 up:standby 2024-09-21T06:34:20.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:20 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T06:34:20.821 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:34:20.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:20 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:20.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:20 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:20.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:20 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:20.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:20 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:34:20.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:20 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:34:20.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:20 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:34:20.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:20 smithi203 ceph-mon[33735]: Deploying daemon mds.b on smithi100 2024-09-21T06:34:20.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:20 smithi203 ceph-mon[33735]: mds.? [v2:172.21.15.67:6840/2331956394,v1:172.21.15.67:6841/2331956394] up:boot 2024-09-21T06:34:20.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:20 smithi203 ceph-mon[33735]: fsmap 11 up:standby 2024-09-21T06:34:20.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:20 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T06:34:21.096 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:34:21.543 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:21 smithi100 ceph-mon[33401]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:21.543 INFO:journalctl@ceph.mds.b.smithi100.stdout:Sep 21 06:34:21 smithi100 systemd[1]: Starting Ceph mds.b for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:34:21.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.614+0000 7f9f103f5640 1 -- 172.21.15.67:0/1194114110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f08076080 msgr2=0x7f9f0810fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:21.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.614+0000 7f9f103f5640 1 --2- 172.21.15.67:0/1194114110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f08076080 0x7f9f0810fc20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9efc009900 tx=0x7f9efc02a410 comp rx=0 tx=0).stop 2024-09-21T06:34:21.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.614+0000 7f9f103f5640 1 -- 172.21.15.67:0/1194114110 shutdown_connections 2024-09-21T06:34:21.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.614+0000 7f9f103f5640 1 --2- 172.21.15.67:0/1194114110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f08110160 0x7f9f08112580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.614+0000 7f9f103f5640 1 --2- 172.21.15.67:0/1194114110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f08076080 0x7f9f0810fc20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.614+0000 7f9f103f5640 1 --2- 172.21.15.67:0/1194114110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f08075730 0x7f9f08075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.614+0000 7f9f103f5640 1 -- 172.21.15.67:0/1194114110 >> 172.21.15.67:0/1194114110 conn(0x7f9f080ff610 msgr2=0x7f9f08101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:21.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.614+0000 7f9f103f5640 1 -- 172.21.15.67:0/1194114110 shutdown_connections 2024-09-21T06:34:21.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.614+0000 7f9f103f5640 1 -- 172.21.15.67:0/1194114110 wait complete. 2024-09-21T06:34:21.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.615+0000 7f9f103f5640 1 Processor -- start 2024-09-21T06:34:21.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.615+0000 7f9f103f5640 1 -- start start 2024-09-21T06:34:21.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.615+0000 7f9f103f5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f08075730 0x7f9f0819c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.615+0000 7f9f103f5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f08076080 0x7f9f0819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.615+0000 7f9f103f5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f08110160 0x7f9f081a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.615+0000 7f9f103f5640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9f08115b20 con 0x7f9f08076080 2024-09-21T06:34:21.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.615+0000 7f9f103f5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9f081159a0 con 0x7f9f08110160 2024-09-21T06:34:21.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.615+0000 7f9f103f5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9f08115ca0 con 0x7f9f08075730 2024-09-21T06:34:21.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.615+0000 7f9f0d990640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f08076080 0x7f9f0819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.615+0000 7f9f0d990640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f08076080 0x7f9f0819cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59426/0 (socket says 172.21.15.67:59426) 2024-09-21T06:34:21.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.615+0000 7f9f0d990640 1 -- 172.21.15.67:0/2602575760 learned_addr learned my addr 172.21.15.67:0/2602575760 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:34:21.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.615+0000 7f9f0e992640 1 --2- 172.21.15.67:0/2602575760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f08110160 0x7f9f081a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.616+0000 7f9f0e191640 1 --2- 172.21.15.67:0/2602575760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f08075730 0x7f9f0819c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.616+0000 7f9f0d990640 1 -- 172.21.15.67:0/2602575760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f08075730 msgr2=0x7f9f0819c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:21.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.616+0000 7f9f0d990640 1 --2- 172.21.15.67:0/2602575760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f08075730 0x7f9f0819c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.616+0000 7f9f0d990640 1 -- 172.21.15.67:0/2602575760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f08110160 msgr2=0x7f9f081a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:21.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.616+0000 7f9f0d990640 1 --2- 172.21.15.67:0/2602575760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f08110160 0x7f9f081a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.616+0000 7f9f0d990640 1 -- 172.21.15.67:0/2602575760 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f081a2540 con 0x7f9f08076080 2024-09-21T06:34:21.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.616+0000 7f9f0e992640 1 --2- 172.21.15.67:0/2602575760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f08110160 0x7f9f081a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:34:21.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.616+0000 7f9f0d990640 1 --2- 172.21.15.67:0/2602575760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f08076080 0x7f9f0819cb70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9efc004770 tx=0x7f9efc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:21.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.616+0000 7f9f077fe640 1 -- 172.21.15.67:0/2602575760 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc03f030 con 0x7f9f08076080 2024-09-21T06:34:21.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.616+0000 7f9f103f5640 1 -- 172.21.15.67:0/2602575760 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f081a27d0 con 0x7f9f08076080 2024-09-21T06:34:21.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.616+0000 7f9f077fe640 1 -- 172.21.15.67:0/2602575760 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9efc004440 con 0x7f9f08076080 2024-09-21T06:34:21.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.616+0000 7f9f077fe640 1 -- 172.21.15.67:0/2602575760 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc035070 con 0x7f9f08076080 2024-09-21T06:34:21.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.616+0000 7f9f103f5640 1 -- 172.21.15.67:0/2602575760 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f081a8f90 con 0x7f9f08076080 2024-09-21T06:34:21.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.617+0000 7f9f103f5640 1 -- 172.21.15.67:0/2602575760 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f08076c40 con 0x7f9f08076080 2024-09-21T06:34:21.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.620+0000 7f9f077fe640 1 -- 172.21.15.67:0/2602575760 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f9efc042030 con 0x7f9f08076080 2024-09-21T06:34:21.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.621+0000 7f9f077fe640 1 --2- 172.21.15.67:0/2602575760 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9ef00b6510 0x7f9ef00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:21.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.621+0000 7f9f0e191640 1 --2- 172.21.15.67:0/2602575760 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9ef00b6510 0x7f9ef00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:21.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.621+0000 7f9f077fe640 1 -- 172.21.15.67:0/2602575760 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f9efc107540 con 0x7f9f08076080 2024-09-21T06:34:21.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.621+0000 7f9f0e191640 1 --2- 172.21.15.67:0/2602575760 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9ef00b6510 0x7f9ef00b8a20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9f00009bd0 tx=0x7f9f00005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:21.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.624+0000 7f9f077fe640 1 -- 172.21.15.67:0/2602575760 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9efc0d2650 con 0x7f9f08076080 2024-09-21T06:34:21.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:21 smithi067 ceph-mon[31232]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:21.803 INFO:journalctl@ceph.mds.b.smithi100.stdout:Sep 21 06:34:21 smithi100 podman[61064]: 2024-09-21 06:34:21.65753375 +0000 UTC m=+0.015887052 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:34:21.803 INFO:journalctl@ceph.mds.b.smithi100.stdout:Sep 21 06:34:21 smithi100 podman[61064]: 2024-09-21 06:34:21.777162958 +0000 UTC m=+0.135516322 container create 1e4cb69455a6927d27b9cb8659c75a4490dd314e090e8af084ddaaca655cb1b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-b, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-21T06:34:21.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.850+0000 7f9f103f5640 1 -- 172.21.15.67:0/2602575760 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f0810b310 con 0x7f9f08076080 2024-09-21T06:34:21.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.850+0000 7f9f077fe640 1 -- 172.21.15.67:0/2602575760 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9707 (secure 0 0 0) 0x7f9efc0d7590 con 0x7f9f08076080 2024-09-21T06:34:21.852 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:34:21.852 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":12,"btime":"2024-09-21T06:34:20:382190+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24641,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-21T06:34:21.852 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 12 2024-09-21T06:34:21.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.853+0000 7f9f103f5640 1 -- 172.21.15.67:0/2602575760 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9ef00b6510 msgr2=0x7f9ef00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:21.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.853+0000 7f9f103f5640 1 --2- 172.21.15.67:0/2602575760 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9ef00b6510 0x7f9ef00b8a20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9f00009bd0 tx=0x7f9f00005f90 comp rx=0 tx=0).stop 2024-09-21T06:34:21.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.853+0000 7f9f103f5640 1 -- 172.21.15.67:0/2602575760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f08076080 msgr2=0x7f9f0819cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:21.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.853+0000 7f9f103f5640 1 --2- 172.21.15.67:0/2602575760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f08076080 0x7f9f0819cb70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9efc004770 tx=0x7f9efc004040 comp rx=0 tx=0).stop 2024-09-21T06:34:21.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.853+0000 7f9f103f5640 1 -- 172.21.15.67:0/2602575760 shutdown_connections 2024-09-21T06:34:21.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.853+0000 7f9f103f5640 1 --2- 172.21.15.67:0/2602575760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f08110160 0x7f9f081a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.853+0000 7f9f103f5640 1 --2- 172.21.15.67:0/2602575760 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9ef00b6510 0x7f9ef00b8a20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.853+0000 7f9f103f5640 1 --2- 172.21.15.67:0/2602575760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f08076080 0x7f9f0819cb70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.853+0000 7f9f103f5640 1 --2- 172.21.15.67:0/2602575760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f08075730 0x7f9f0819c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:21.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.853+0000 7f9f103f5640 1 -- 172.21.15.67:0/2602575760 >> 172.21.15.67:0/2602575760 conn(0x7f9f080ff610 msgr2=0x7f9f080ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:21.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.853+0000 7f9f103f5640 1 -- 172.21.15.67:0/2602575760 shutdown_connections 2024-09-21T06:34:21.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:21.853+0000 7f9f103f5640 1 -- 172.21.15.67:0/2602575760 wait complete. 2024-09-21T06:34:21.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:21 smithi203 ceph-mon[33735]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:22.304 INFO:journalctl@ceph.mds.b.smithi100.stdout:Sep 21 06:34:22 smithi100 podman[61064]: 2024-09-21 06:34:22.030130627 +0000 UTC m=+0.388483915 container init 1e4cb69455a6927d27b9cb8659c75a4490dd314e090e8af084ddaaca655cb1b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-b, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:34:22.304 INFO:journalctl@ceph.mds.b.smithi100.stdout:Sep 21 06:34:22 smithi100 podman[61064]: 2024-09-21 06:34:22.032937397 +0000 UTC m=+0.391290688 container start 1e4cb69455a6927d27b9cb8659c75a4490dd314e090e8af084ddaaca655cb1b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-21T06:34:22.304 INFO:journalctl@ceph.mds.b.smithi100.stdout:Sep 21 06:34:22 smithi100 ceph-mds[61083]: -- 172.21.15.100:0/516965935 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bb64ead860 con 0x55bb69055180 2024-09-21T06:34:22.304 INFO:journalctl@ceph.mds.b.smithi100.stdout:Sep 21 06:34:22 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-b[61079]: starting mds.b at 2024-09-21T06:34:22.304 INFO:journalctl@ceph.mds.b.smithi100.stdout:Sep 21 06:34:22 smithi100 bash[61064]: 1e4cb69455a6927d27b9cb8659c75a4490dd314e090e8af084ddaaca655cb1b0 2024-09-21T06:34:22.304 INFO:journalctl@ceph.mds.b.smithi100.stdout:Sep 21 06:34:22 smithi100 systemd[1]: Started Ceph mds.b for 2a791768-77e2-11ef-baf5-efdc52797490. 2024-09-21T06:34:22.778 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:22 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2602575760' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:34:22.778 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:22 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:22.778 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:22 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:22.778 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:22 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:22.778 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:22 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:22.778 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:22 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:22.778 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:22 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:34:22.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:22 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2602575760' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:34:22.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:22 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:22.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:22 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:22.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:22 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:22.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:22 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:22.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:22 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:22.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:22 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:34:22.858 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:22 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2602575760' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:34:22.862 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:22 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:22.862 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:22 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:22.862 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:22 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:22.862 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:22 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:22.862 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:22 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:22.862 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:22 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:34:23.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:23 smithi067 ceph-mon[31232]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:23.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:23 smithi067 ceph-mon[31232]: mds.? [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] up:boot 2024-09-21T06:34:23.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:23 smithi067 ceph-mon[31232]: fsmap 12 up:standby 2024-09-21T06:34:23.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:23 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T06:34:23.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:23 smithi100 ceph-mon[33401]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:23.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:23 smithi100 ceph-mon[33401]: mds.? [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] up:boot 2024-09-21T06:34:23.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:23 smithi100 ceph-mon[33401]: fsmap 12 up:standby 2024-09-21T06:34:23.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:23 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T06:34:23.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:23 smithi203 ceph-mon[33735]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:23.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:23 smithi203 ceph-mon[33735]: mds.? [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] up:boot 2024-09-21T06:34:23.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:23 smithi203 ceph-mon[33735]: fsmap 12 up:standby 2024-09-21T06:34:23.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:23 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T06:34:24.583 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:34:24.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:24 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:24.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:24 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:24.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:24 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:24.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:24 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:24.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:24 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:24.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:24 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:24.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:24 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:34:24.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:24 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:34:24.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:24 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:24.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:24 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:34:24.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:24 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:24.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:24 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:24.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:24 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:24.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:24 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:24.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:24 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:24.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:24 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:24.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:24 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:24.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:24 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:34:24.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:24 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:34:24.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:24 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:24.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:24 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:34:24.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:24 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:24.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:24 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:24.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:24 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:24.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:24 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:24.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:24 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:24.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:24 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:24.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:24 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:24.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:24 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:34:24.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:24 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:34:24.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:24 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:24.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:24 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:34:24.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:24 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:24.948 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:34:25.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.458+0000 7f3a9ba8f640 1 -- 172.21.15.67:0/1634186025 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a94109df0 msgr2=0x7f3a94110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:25.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.458+0000 7f3a9ba8f640 1 --2- 172.21.15.67:0/1634186025 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a94109df0 0x7f3a94110620 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c009ca0 tx=0x7f3a8c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:34:25.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.458+0000 7f3a9ba8f640 1 -- 172.21.15.67:0/1634186025 shutdown_connections 2024-09-21T06:34:25.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.458+0000 7f3a9ba8f640 1 --2- 172.21.15.67:0/1634186025 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a94109df0 0x7f3a94110620 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:25.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.458+0000 7f3a9ba8f640 1 --2- 172.21.15.67:0/1634186025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a941094a0 0x7f3a941098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:25.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.458+0000 7f3a9ba8f640 1 --2- 172.21.15.67:0/1634186025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a94103b60 0x7f3a94103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:25.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.458+0000 7f3a9ba8f640 1 -- 172.21.15.67:0/1634186025 >> 172.21.15.67:0/1634186025 conn(0x7f3a9407b850 msgr2=0x7f3a9407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:25.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.458+0000 7f3a9ba8f640 1 -- 172.21.15.67:0/1634186025 shutdown_connections 2024-09-21T06:34:25.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.458+0000 7f3a9ba8f640 1 -- 172.21.15.67:0/1634186025 wait complete. 2024-09-21T06:34:25.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.458+0000 7f3a9ba8f640 1 Processor -- start 2024-09-21T06:34:25.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.459+0000 7f3a9ba8f640 1 -- start start 2024-09-21T06:34:25.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.459+0000 7f3a9ba8f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a94103b60 0x7f3a94075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:25.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.459+0000 7f3a9ba8f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a941094a0 0x7f3a94075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:25.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.459+0000 7f3a9ba8f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a94109df0 0x7f3a94076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:25.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.459+0000 7f3a9ba8f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3a941139b0 con 0x7f3a94109df0 2024-09-21T06:34:25.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.459+0000 7f3a9ba8f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3a94113830 con 0x7f3a941094a0 2024-09-21T06:34:25.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.459+0000 7f3a9ba8f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3a94113b30 con 0x7f3a94103b60 2024-09-21T06:34:25.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.459+0000 7f3a9a02c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a94109df0 0x7f3a94076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:25.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.459+0000 7f3a9a02c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a94109df0 0x7f3a94076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59440/0 (socket says 172.21.15.67:59440) 2024-09-21T06:34:25.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.459+0000 7f3a9a02c640 1 -- 172.21.15.67:0/935304253 learned_addr learned my addr 172.21.15.67:0/935304253 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:34:25.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.459+0000 7f3a9982b640 1 --2- 172.21.15.67:0/935304253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a94103b60 0x7f3a94075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:25.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.459+0000 7f3a9902a640 1 --2- 172.21.15.67:0/935304253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a941094a0 0x7f3a94075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:25.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.459+0000 7f3a9a02c640 1 -- 172.21.15.67:0/935304253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a94103b60 msgr2=0x7f3a94075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:25.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.459+0000 7f3a9a02c640 1 --2- 172.21.15.67:0/935304253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a94103b60 0x7f3a94075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:25.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.459+0000 7f3a9a02c640 1 -- 172.21.15.67:0/935304253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a941094a0 msgr2=0x7f3a94075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:25.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.459+0000 7f3a9a02c640 1 --2- 172.21.15.67:0/935304253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a941094a0 0x7f3a94075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:25.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.459+0000 7f3a9a02c640 1 -- 172.21.15.67:0/935304253 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a941a8780 con 0x7f3a94109df0 2024-09-21T06:34:25.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.460+0000 7f3a9a02c640 1 --2- 172.21.15.67:0/935304253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a94109df0 0x7f3a94076190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c005cc0 tx=0x7f3a8c003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:25.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.460+0000 7f3a86ffd640 1 -- 172.21.15.67:0/935304253 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c032c10 con 0x7f3a94109df0 2024-09-21T06:34:25.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.460+0000 7f3a9ba8f640 1 -- 172.21.15.67:0/935304253 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a941a8a10 con 0x7f3a94109df0 2024-09-21T06:34:25.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.460+0000 7f3a86ffd640 1 -- 172.21.15.67:0/935304253 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a8c032db0 con 0x7f3a94109df0 2024-09-21T06:34:25.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.460+0000 7f3a86ffd640 1 -- 172.21.15.67:0/935304253 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c030a60 con 0x7f3a94109df0 2024-09-21T06:34:25.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.460+0000 7f3a9ba8f640 1 -- 172.21.15.67:0/935304253 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a941a8f50 con 0x7f3a94109df0 2024-09-21T06:34:25.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.461+0000 7f3a9ba8f640 1 -- 172.21.15.67:0/935304253 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a94103f70 con 0x7f3a94109df0 2024-09-21T06:34:25.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.463+0000 7f3a86ffd640 1 -- 172.21.15.67:0/935304253 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f3a8c030c00 con 0x7f3a94109df0 2024-09-21T06:34:25.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.463+0000 7f3a86ffd640 1 --2- 172.21.15.67:0/935304253 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f3a800b6510 0x7f3a800b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:25.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.463+0000 7f3a9982b640 1 --2- 172.21.15.67:0/935304253 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f3a800b6510 0x7f3a800b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:25.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.464+0000 7f3a86ffd640 1 -- 172.21.15.67:0/935304253 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f3a8c108060 con 0x7f3a94109df0 2024-09-21T06:34:25.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.464+0000 7f3a9982b640 1 --2- 172.21.15.67:0/935304253 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f3a800b6510 0x7f3a800b8a20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3a94077160 tx=0x7f3a90005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:25.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.467+0000 7f3a86ffd640 1 -- 172.21.15.67:0/935304253 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3a8c0ced90 con 0x7f3a94109df0 2024-09-21T06:34:25.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.714+0000 7f3a9ba8f640 1 -- 172.21.15.67:0/935304253 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a94076a10 con 0x7f3a94109df0 2024-09-21T06:34:25.716 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:25 smithi067 ceph-mon[31232]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:25.716 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:25 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:25.716 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:25 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:25.716 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:25 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:25.716 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:25 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:25.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.714+0000 7f3a86ffd640 1 -- 172.21.15.67:0/935304253 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10533 (secure 0 0 0) 0x7f3a94076a10 con 0x7f3a94109df0 2024-09-21T06:34:25.717 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:34:25.718 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":13,"btime":"2024-09-21T06:34:22:516071+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24641,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24671,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-21T06:34:25.718 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 13 2024-09-21T06:34:25.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.718+0000 7f3a9ba8f640 1 -- 172.21.15.67:0/935304253 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f3a800b6510 msgr2=0x7f3a800b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:25.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.718+0000 7f3a9ba8f640 1 --2- 172.21.15.67:0/935304253 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f3a800b6510 0x7f3a800b8a20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3a94077160 tx=0x7f3a90005f90 comp rx=0 tx=0).stop 2024-09-21T06:34:25.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.718+0000 7f3a9ba8f640 1 -- 172.21.15.67:0/935304253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a94109df0 msgr2=0x7f3a94076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:25.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.718+0000 7f3a9ba8f640 1 --2- 172.21.15.67:0/935304253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a94109df0 0x7f3a94076190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c005cc0 tx=0x7f3a8c003f90 comp rx=0 tx=0).stop 2024-09-21T06:34:25.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.718+0000 7f3a9ba8f640 1 -- 172.21.15.67:0/935304253 shutdown_connections 2024-09-21T06:34:25.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.718+0000 7f3a9ba8f640 1 --2- 172.21.15.67:0/935304253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a941094a0 0x7f3a94075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:25.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.718+0000 7f3a9ba8f640 1 --2- 172.21.15.67:0/935304253 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f3a800b6510 0x7f3a800b8a20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:25.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.718+0000 7f3a9ba8f640 1 --2- 172.21.15.67:0/935304253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a94109df0 0x7f3a94076190 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:25.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.718+0000 7f3a9ba8f640 1 --2- 172.21.15.67:0/935304253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a94103b60 0x7f3a94075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:25.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.718+0000 7f3a9ba8f640 1 -- 172.21.15.67:0/935304253 >> 172.21.15.67:0/935304253 conn(0x7f3a9407b850 msgr2=0x7f3a94107690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:25.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.718+0000 7f3a9ba8f640 1 -- 172.21.15.67:0/935304253 shutdown_connections 2024-09-21T06:34:25.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:25.718+0000 7f3a9ba8f640 1 -- 172.21.15.67:0/935304253 wait complete. 2024-09-21T06:34:25.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:25 smithi203 ceph-mon[33735]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:25.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:25 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:25.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:25 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:25.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:25 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:25.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:25 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:26.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:25 smithi100 ceph-mon[33401]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:26.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:25 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:26.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:25 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:26.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:25 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:26.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:25 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:26.378 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph --format=json-pretty osd lspools 2024-09-21T06:34:26.642 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:34:26.763 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:26 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/935304253' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:34:26.763 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:26 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:26.763 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:26 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:26.763 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:26 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:34:26.763 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:26 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:34:26.763 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:26 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:26.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:26 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/935304253' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:34:26.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:26 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:26.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:26 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:26.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:26 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:34:26.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:26 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:34:26.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:26 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:27.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:26 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/935304253' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:34:27.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:26 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:27.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:26 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:27.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:26 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:34:27.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:26 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:34:27.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:26 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:34:27.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.159+0000 7f5137b49640 1 -- 172.21.15.67:0/773137832 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5130102ae0 msgr2=0x7f5130102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:27.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.159+0000 7f5137b49640 1 --2- 172.21.15.67:0/773137832 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5130102ae0 0x7f5130102ef0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f512c009900 tx=0x7f512c02a410 comp rx=0 tx=0).stop 2024-09-21T06:34:27.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.159+0000 7f5137b49640 1 -- 172.21.15.67:0/773137832 shutdown_connections 2024-09-21T06:34:27.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.159+0000 7f5137b49640 1 --2- 172.21.15.67:0/773137832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5130108d20 0x7f513010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:27.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.159+0000 7f5137b49640 1 --2- 172.21.15.67:0/773137832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51301083d0 0x7f51301087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:27.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.159+0000 7f5137b49640 1 --2- 172.21.15.67:0/773137832 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5130102ae0 0x7f5130102ef0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:27.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.159+0000 7f5137b49640 1 -- 172.21.15.67:0/773137832 >> 172.21.15.67:0/773137832 conn(0x7f5130079760 msgr2=0x7f5130079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:27.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.159+0000 7f5137b49640 1 -- 172.21.15.67:0/773137832 shutdown_connections 2024-09-21T06:34:27.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.159+0000 7f5137b49640 1 -- 172.21.15.67:0/773137832 wait complete. 2024-09-21T06:34:27.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.160+0000 7f5137b49640 1 Processor -- start 2024-09-21T06:34:27.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.160+0000 7f5137b49640 1 -- start start 2024-09-21T06:34:27.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.160+0000 7f5137b49640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5130102ae0 0x7f513019a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:27.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.160+0000 7f5137b49640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51301083d0 0x7f513019a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:27.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.160+0000 7f5137b49640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5130108d20 0x7f513019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:27.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.160+0000 7f5137b49640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5130113960 con 0x7f5130108d20 2024-09-21T06:34:27.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.160+0000 7f5137b49640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f51301137e0 con 0x7f51301083d0 2024-09-21T06:34:27.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.160+0000 7f5137b49640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5130113ae0 con 0x7f5130102ae0 2024-09-21T06:34:27.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.160+0000 7f51360e6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5130108d20 0x7f513019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:27.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.160+0000 7f51358e5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5130102ae0 0x7f513019a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:27.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.160+0000 7f51360e6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5130108d20 0x7f513019fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59462/0 (socket says 172.21.15.67:59462) 2024-09-21T06:34:27.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.160+0000 7f51360e6640 1 -- 172.21.15.67:0/297037426 learned_addr learned my addr 172.21.15.67:0/297037426 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:34:27.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.160+0000 7f51350e4640 1 --2- 172.21.15.67:0/297037426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51301083d0 0x7f513019a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:27.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.161+0000 7f51360e6640 1 -- 172.21.15.67:0/297037426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5130102ae0 msgr2=0x7f513019a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:27.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.161+0000 7f51360e6640 1 --2- 172.21.15.67:0/297037426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5130102ae0 0x7f513019a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:27.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.161+0000 7f51360e6640 1 -- 172.21.15.67:0/297037426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51301083d0 msgr2=0x7f513019a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:27.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.161+0000 7f51360e6640 1 --2- 172.21.15.67:0/297037426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51301083d0 0x7f513019a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:27.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.161+0000 7f51360e6640 1 -- 172.21.15.67:0/297037426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51301a0280 con 0x7f5130108d20 2024-09-21T06:34:27.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.161+0000 7f51360e6640 1 --2- 172.21.15.67:0/297037426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5130108d20 0x7f513019fc10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f512800cce0 tx=0x7f512800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:27.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.161+0000 7f5126ffd640 1 -- 172.21.15.67:0/297037426 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512800f450 con 0x7f5130108d20 2024-09-21T06:34:27.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.161+0000 7f5126ffd640 1 -- 172.21.15.67:0/297037426 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f512800fa80 con 0x7f5130108d20 2024-09-21T06:34:27.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.161+0000 7f5137b49640 1 -- 172.21.15.67:0/297037426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51301a0570 con 0x7f5130108d20 2024-09-21T06:34:27.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.161+0000 7f5126ffd640 1 -- 172.21.15.67:0/297037426 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5128018ad0 con 0x7f5130108d20 2024-09-21T06:34:27.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.161+0000 7f5137b49640 1 -- 172.21.15.67:0/297037426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51301a6d90 con 0x7f5130108d20 2024-09-21T06:34:27.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.162+0000 7f5137b49640 1 -- 172.21.15.67:0/297037426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5130103d10 con 0x7f5130108d20 2024-09-21T06:34:27.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.164+0000 7f5126ffd640 1 -- 172.21.15.67:0/297037426 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f512800f5f0 con 0x7f5130108d20 2024-09-21T06:34:27.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.164+0000 7f5126ffd640 1 --2- 172.21.15.67:0/297037426 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f511c0b6540 0x7f511c0b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:27.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.164+0000 7f51358e5640 1 --2- 172.21.15.67:0/297037426 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f511c0b6540 0x7f511c0b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:27.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.165+0000 7f5126ffd640 1 -- 172.21.15.67:0/297037426 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f51280e7600 con 0x7f5130108d20 2024-09-21T06:34:27.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.165+0000 7f51358e5640 1 --2- 172.21.15.67:0/297037426 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f511c0b6540 0x7f511c0b8a50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f512c0057c0 tx=0x7f512c0058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:27.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.167+0000 7f5126ffd640 1 -- 172.21.15.67:0/297037426 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5128014030 con 0x7f5130108d20 2024-09-21T06:34:27.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.359+0000 7f5137b49640 1 -- 172.21.15.67:0/297037426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5130102ef0 con 0x7f5130108d20 2024-09-21T06:34:27.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.360+0000 7f5126ffd640 1 -- 172.21.15.67:0/297037426 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v70) ==== 84+0+65 (secure 0 0 0) 0x7f5130102ef0 con 0x7f5130108d20 2024-09-21T06:34:27.362 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:34:27.363 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-09-21T06:34:27.363 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:34:27.363 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 1, 2024-09-21T06:34:27.363 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": ".mgr" 2024-09-21T06:34:27.363 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:34:27.363 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-09-21T06:34:27.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.363+0000 7f5137b49640 1 -- 172.21.15.67:0/297037426 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f511c0b6540 msgr2=0x7f511c0b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:27.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.363+0000 7f5137b49640 1 --2- 172.21.15.67:0/297037426 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f511c0b6540 0x7f511c0b8a50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f512c0057c0 tx=0x7f512c0058d0 comp rx=0 tx=0).stop 2024-09-21T06:34:27.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.363+0000 7f5137b49640 1 -- 172.21.15.67:0/297037426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5130108d20 msgr2=0x7f513019fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:27.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.363+0000 7f5137b49640 1 --2- 172.21.15.67:0/297037426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5130108d20 0x7f513019fc10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f512800cce0 tx=0x7f512800a620 comp rx=0 tx=0).stop 2024-09-21T06:34:27.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.363+0000 7f5137b49640 1 -- 172.21.15.67:0/297037426 shutdown_connections 2024-09-21T06:34:27.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.363+0000 7f5137b49640 1 --2- 172.21.15.67:0/297037426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51301083d0 0x7f513019a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:27.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.363+0000 7f5137b49640 1 --2- 172.21.15.67:0/297037426 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f511c0b6540 0x7f511c0b8a50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:27.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.363+0000 7f5137b49640 1 --2- 172.21.15.67:0/297037426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5130108d20 0x7f513019fc10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:27.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.363+0000 7f5137b49640 1 --2- 172.21.15.67:0/297037426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5130102ae0 0x7f513019a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:27.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.363+0000 7f5137b49640 1 -- 172.21.15.67:0/297037426 >> 172.21.15.67:0/297037426 conn(0x7f5130079760 msgr2=0x7f5130101610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:27.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.364+0000 7f5137b49640 1 -- 172.21.15.67:0/297037426 shutdown_connections 2024-09-21T06:34:27.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:27.364+0000 7f5137b49640 1 -- 172.21.15.67:0/297037426 wait complete. 2024-09-21T06:34:27.739 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:27 smithi067 ceph-mon[31232]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:27.739 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:27 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/297037426' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T06:34:27.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:27 smithi203 ceph-mon[33735]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:27.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:27 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/297037426' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T06:34:28.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:27 smithi100 ceph-mon[33401]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:28.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:27 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/297037426' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T06:34:28.109 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T06:34:28.110 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T06:34:28.369 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:34:28.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.879+0000 7f70e8d62640 1 -- 172.21.15.67:0/773170644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70e41094a0 msgr2=0x7f70e41098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:28.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.879+0000 7f70e8d62640 1 --2- 172.21.15.67:0/773170644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70e41094a0 0x7f70e41098b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f70d4009900 tx=0x7f70d402a410 comp rx=0 tx=0).stop 2024-09-21T06:34:28.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.879+0000 7f70e8d62640 1 -- 172.21.15.67:0/773170644 shutdown_connections 2024-09-21T06:34:28.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.879+0000 7f70e8d62640 1 --2- 172.21.15.67:0/773170644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70e4109df0 0x7f70e4110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:28.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.879+0000 7f70e8d62640 1 --2- 172.21.15.67:0/773170644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70e41094a0 0x7f70e41098b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:28.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.879+0000 7f70e8d62640 1 --2- 172.21.15.67:0/773170644 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70e4103b60 0x7f70e4103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:28.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.879+0000 7f70e8d62640 1 -- 172.21.15.67:0/773170644 >> 172.21.15.67:0/773170644 conn(0x7f70e407b850 msgr2=0x7f70e407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:28.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.879+0000 7f70e8d62640 1 -- 172.21.15.67:0/773170644 shutdown_connections 2024-09-21T06:34:28.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.879+0000 7f70e8d62640 1 -- 172.21.15.67:0/773170644 wait complete. 2024-09-21T06:34:28.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.880+0000 7f70e8d62640 1 Processor -- start 2024-09-21T06:34:28.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.880+0000 7f70e8d62640 1 -- start start 2024-09-21T06:34:28.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.881+0000 7f70e8d62640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70e4103b60 0x7f70e4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:28.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.881+0000 7f70e8d62640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70e41094a0 0x7f70e4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:28.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.881+0000 7f70e8d62640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70e4109df0 0x7f70e4076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:28.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.881+0000 7f70e8d62640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f70e41139b0 con 0x7f70e4109df0 2024-09-21T06:34:28.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.881+0000 7f70e8d62640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f70e4113830 con 0x7f70e41094a0 2024-09-21T06:34:28.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.881+0000 7f70e8d62640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f70e4113b30 con 0x7f70e4103b60 2024-09-21T06:34:28.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.881+0000 7f70e2d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70e4109df0 0x7f70e4076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:28.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.881+0000 7f70e2d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70e4109df0 0x7f70e4076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35336/0 (socket says 172.21.15.67:35336) 2024-09-21T06:34:28.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.881+0000 7f70e2d9d640 1 -- 172.21.15.67:0/4279691870 learned_addr learned my addr 172.21.15.67:0/4279691870 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:34:28.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.881+0000 7f70e259c640 1 --2- 172.21.15.67:0/4279691870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70e4103b60 0x7f70e4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:28.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.881+0000 7f70e1d9b640 1 --2- 172.21.15.67:0/4279691870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70e41094a0 0x7f70e4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:28.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.881+0000 7f70e2d9d640 1 -- 172.21.15.67:0/4279691870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70e4103b60 msgr2=0x7f70e4075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:28.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.881+0000 7f70e2d9d640 1 --2- 172.21.15.67:0/4279691870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70e4103b60 0x7f70e4075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:28.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.881+0000 7f70e2d9d640 1 -- 172.21.15.67:0/4279691870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70e41094a0 msgr2=0x7f70e4075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:28.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.881+0000 7f70e2d9d640 1 --2- 172.21.15.67:0/4279691870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70e41094a0 0x7f70e4075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:28.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.881+0000 7f70e2d9d640 1 -- 172.21.15.67:0/4279691870 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e41a8780 con 0x7f70e4109df0 2024-09-21T06:34:28.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.881+0000 7f70e2d9d640 1 --2- 172.21.15.67:0/4279691870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70e4109df0 0x7f70e4076190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f70dc009fd0 tx=0x7f70dc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:28.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.881+0000 7f70db7fe640 1 -- 172.21.15.67:0/4279691870 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc017070 con 0x7f70e4109df0 2024-09-21T06:34:28.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.881+0000 7f70db7fe640 1 -- 172.21.15.67:0/4279691870 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f70dc003e00 con 0x7f70e4109df0 2024-09-21T06:34:28.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.882+0000 7f70e8d62640 1 -- 172.21.15.67:0/4279691870 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70e41a8a70 con 0x7f70e4109df0 2024-09-21T06:34:28.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.882+0000 7f70db7fe640 1 -- 172.21.15.67:0/4279691870 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc010460 con 0x7f70e4109df0 2024-09-21T06:34:28.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.882+0000 7f70e8d62640 1 -- 172.21.15.67:0/4279691870 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70e41a8f30 con 0x7f70e4109df0 2024-09-21T06:34:28.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.883+0000 7f70e8d62640 1 -- 172.21.15.67:0/4279691870 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70e4103f70 con 0x7f70e4109df0 2024-09-21T06:34:28.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.884+0000 7f70db7fe640 1 -- 172.21.15.67:0/4279691870 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f70dc003970 con 0x7f70e4109df0 2024-09-21T06:34:28.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.885+0000 7f70db7fe640 1 --2- 172.21.15.67:0/4279691870 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f70d00b6510 0x7f70d00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:28.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.885+0000 7f70e259c640 1 --2- 172.21.15.67:0/4279691870 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f70d00b6510 0x7f70d00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:28.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.885+0000 7f70db7fe640 1 -- 172.21.15.67:0/4279691870 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f70dc0e6f10 con 0x7f70e4109df0 2024-09-21T06:34:28.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.885+0000 7f70e259c640 1 --2- 172.21.15.67:0/4279691870 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f70d00b6510 0x7f70d00b8a20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f70e4077160 tx=0x7f70cc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:28.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:28.888+0000 7f70db7fe640 1 -- 172.21.15.67:0/4279691870 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f70dc0adc40 con 0x7f70e4109df0 2024-09-21T06:34:29.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:29.077+0000 7f70e8d62640 1 -- 172.21.15.67:0/4279691870 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f70e402e640 con 0x7f70e4109df0 2024-09-21T06:34:29.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:29.605+0000 7f70db7fe640 1 -- 172.21.15.67:0/4279691870 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v71) ==== 138+0+0 (secure 0 0 0) 0x7f70dc0b2020 con 0x7f70e4109df0 2024-09-21T06:34:29.608 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' created 2024-09-21T06:34:29.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:29.608+0000 7f70e8d62640 1 -- 172.21.15.67:0/4279691870 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f70d00b6510 msgr2=0x7f70d00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:29.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:29.608+0000 7f70e8d62640 1 --2- 172.21.15.67:0/4279691870 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f70d00b6510 0x7f70d00b8a20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f70e4077160 tx=0x7f70cc00b390 comp rx=0 tx=0).stop 2024-09-21T06:34:29.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:29.608+0000 7f70e8d62640 1 -- 172.21.15.67:0/4279691870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70e4109df0 msgr2=0x7f70e4076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:29.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:29.608+0000 7f70e8d62640 1 --2- 172.21.15.67:0/4279691870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70e4109df0 0x7f70e4076190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f70dc009fd0 tx=0x7f70dc00b260 comp rx=0 tx=0).stop 2024-09-21T06:34:29.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:29.608+0000 7f70e8d62640 1 -- 172.21.15.67:0/4279691870 shutdown_connections 2024-09-21T06:34:29.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:29.608+0000 7f70e8d62640 1 --2- 172.21.15.67:0/4279691870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70e41094a0 0x7f70e4075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:29.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:29.608+0000 7f70e8d62640 1 --2- 172.21.15.67:0/4279691870 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f70d00b6510 0x7f70d00b8a20 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:29.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:29.608+0000 7f70e8d62640 1 --2- 172.21.15.67:0/4279691870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70e4109df0 0x7f70e4076190 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:29.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:29.608+0000 7f70e8d62640 1 --2- 172.21.15.67:0/4279691870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70e4103b60 0x7f70e4075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:29.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:29.608+0000 7f70e8d62640 1 -- 172.21.15.67:0/4279691870 >> 172.21.15.67:0/4279691870 conn(0x7f70e407b850 msgr2=0x7f70e4105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:29.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:29.609+0000 7f70e8d62640 1 -- 172.21.15.67:0/4279691870 shutdown_connections 2024-09-21T06:34:29.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:29.609+0000 7f70e8d62640 1 -- 172.21.15.67:0/4279691870 wait complete. 2024-09-21T06:34:29.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:29 smithi203 ceph-mon[33735]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:29.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:29 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/4279691870' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T06:34:29.990 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:29 smithi067 ceph-mon[31232]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:29.990 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:29 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/4279691870' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T06:34:30.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:29 smithi100 ceph-mon[33401]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:30.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:29 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/4279691870' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T06:34:30.343 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-21T06:34:30.608 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:34:30.869 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:30 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/4279691870' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T06:34:30.869 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:30 smithi067 ceph-mon[31232]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T06:34:30.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:30 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/4279691870' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T06:34:30.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:30 smithi203 ceph-mon[33735]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T06:34:31.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:30 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/4279691870' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T06:34:31.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:30 smithi100 ceph-mon[33401]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T06:34:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.092+0000 7f2309291640 1 -- 172.21.15.67:0/3627053123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2304103b60 msgr2=0x7f2304103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.092+0000 7f2309291640 1 --2- 172.21.15.67:0/3627053123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2304103b60 0x7f2304103f70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f22f4009900 tx=0x7f22f402a3f0 comp rx=0 tx=0).stop 2024-09-21T06:34:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.092+0000 7f2309291640 1 -- 172.21.15.67:0/3627053123 shutdown_connections 2024-09-21T06:34:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.092+0000 7f2309291640 1 --2- 172.21.15.67:0/3627053123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23041094a0 0x7f23041098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.092+0000 7f2309291640 1 --2- 172.21.15.67:0/3627053123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2304109df0 0x7f2304110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.092+0000 7f2309291640 1 --2- 172.21.15.67:0/3627053123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2304103b60 0x7f2304103f70 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.092+0000 7f2309291640 1 -- 172.21.15.67:0/3627053123 >> 172.21.15.67:0/3627053123 conn(0x7f230407b850 msgr2=0x7f230407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.092+0000 7f2309291640 1 -- 172.21.15.67:0/3627053123 shutdown_connections 2024-09-21T06:34:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.092+0000 7f2309291640 1 -- 172.21.15.67:0/3627053123 wait complete. 2024-09-21T06:34:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.093+0000 7f2309291640 1 Processor -- start 2024-09-21T06:34:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.093+0000 7f2309291640 1 -- start start 2024-09-21T06:34:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.093+0000 7f2309291640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2304103b60 0x7f230419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.093+0000 7f2309291640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23041094a0 0x7f230419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.094+0000 7f2309291640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2304109df0 0x7f230419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.094+0000 7f2309291640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2304113960 con 0x7f23041094a0 2024-09-21T06:34:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.094+0000 7f2309291640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f23041137e0 con 0x7f2304109df0 2024-09-21T06:34:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.094+0000 7f2309291640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2304113ae0 con 0x7f2304103b60 2024-09-21T06:34:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.094+0000 7f22fa7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23041094a0 0x7f230419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.094+0000 7f22fa7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23041094a0 0x7f230419a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35354/0 (socket says 172.21.15.67:35354) 2024-09-21T06:34:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.094+0000 7f22fa7fc640 1 -- 172.21.15.67:0/2767755637 learned_addr learned my addr 172.21.15.67:0/2767755637 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:34:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.094+0000 7f23037fe640 1 --2- 172.21.15.67:0/2767755637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2304109df0 0x7f230419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.094+0000 7f2302ffd640 1 --2- 172.21.15.67:0/2767755637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2304103b60 0x7f230419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.094+0000 7f22fa7fc640 1 -- 172.21.15.67:0/2767755637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2304103b60 msgr2=0x7f230419a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.094+0000 7f22fa7fc640 1 --2- 172.21.15.67:0/2767755637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2304103b60 0x7f230419a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:31.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.094+0000 7f22fa7fc640 1 -- 172.21.15.67:0/2767755637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2304109df0 msgr2=0x7f230419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:31.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.094+0000 7f22fa7fc640 1 --2- 172.21.15.67:0/2767755637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2304109df0 0x7f230419fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:31.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.094+0000 7f22fa7fc640 1 -- 172.21.15.67:0/2767755637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23041a02a0 con 0x7f23041094a0 2024-09-21T06:34:31.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.094+0000 7f2302ffd640 1 --2- 172.21.15.67:0/2767755637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2304103b60 0x7f230419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:31.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.094+0000 7f22fa7fc640 1 --2- 172.21.15.67:0/2767755637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23041094a0 0x7f230419a990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f22ec00d950 tx=0x7f22ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:31.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.094+0000 7f2300ff9640 1 -- 172.21.15.67:0/2767755637 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec017070 con 0x7f23041094a0 2024-09-21T06:34:31.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.095+0000 7f2309291640 1 -- 172.21.15.67:0/2767755637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23041a0590 con 0x7f23041094a0 2024-09-21T06:34:31.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.095+0000 7f2300ff9640 1 -- 172.21.15.67:0/2767755637 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22ec009c30 con 0x7f23041094a0 2024-09-21T06:34:31.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.095+0000 7f2300ff9640 1 -- 172.21.15.67:0/2767755637 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec0047a0 con 0x7f23041094a0 2024-09-21T06:34:31.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.095+0000 7f2309291640 1 -- 172.21.15.67:0/2767755637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23041a6db0 con 0x7f23041094a0 2024-09-21T06:34:31.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.096+0000 7f2309291640 1 -- 172.21.15.67:0/2767755637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2304103f70 con 0x7f23041094a0 2024-09-21T06:34:31.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.097+0000 7f2300ff9640 1 -- 172.21.15.67:0/2767755637 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f22ec004020 con 0x7f23041094a0 2024-09-21T06:34:31.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.098+0000 7f2300ff9640 1 --2- 172.21.15.67:0/2767755637 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f22f00b6540 0x7f22f00b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:31.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.098+0000 7f2302ffd640 1 --2- 172.21.15.67:0/2767755637 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f22f00b6540 0x7f22f00b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:31.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.098+0000 7f2300ff9640 1 -- 172.21.15.67:0/2767755637 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7497+0+0 (secure 0 0 0) 0x7f22ec0e7150 con 0x7f23041094a0 2024-09-21T06:34:31.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.099+0000 7f2302ffd640 1 --2- 172.21.15.67:0/2767755637 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f22f00b6540 0x7f22f00b8a50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f22f4000c00 tx=0x7f22f4005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.102+0000 7f2300ff9640 1 -- 172.21.15.67:0/2767755637 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f22ec020020 con 0x7f23041094a0 2024-09-21T06:34:31.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.295+0000 7f2309291640 1 -- 172.21.15.67:0/2767755637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f230419b520 con 0x7f23041094a0 2024-09-21T06:34:31.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.614+0000 7f2300ff9640 1 -- 172.21.15.67:0/2767755637 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v73) ==== 171+0+0 (secure 0 0 0) 0x7f22ec0add10 con 0x7f23041094a0 2024-09-21T06:34:31.617 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' created 2024-09-21T06:34:31.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.617+0000 7f2309291640 1 -- 172.21.15.67:0/2767755637 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f22f00b6540 msgr2=0x7f22f00b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:31.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.617+0000 7f2309291640 1 --2- 172.21.15.67:0/2767755637 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f22f00b6540 0x7f22f00b8a50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f22f4000c00 tx=0x7f22f4005a20 comp rx=0 tx=0).stop 2024-09-21T06:34:31.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.617+0000 7f2309291640 1 -- 172.21.15.67:0/2767755637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23041094a0 msgr2=0x7f230419a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:31.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.617+0000 7f2309291640 1 --2- 172.21.15.67:0/2767755637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23041094a0 0x7f230419a990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f22ec00d950 tx=0x7f22ec00de10 comp rx=0 tx=0).stop 2024-09-21T06:34:31.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.617+0000 7f2309291640 1 -- 172.21.15.67:0/2767755637 shutdown_connections 2024-09-21T06:34:31.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.617+0000 7f2309291640 1 --2- 172.21.15.67:0/2767755637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2304109df0 0x7f230419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:31.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.617+0000 7f2309291640 1 --2- 172.21.15.67:0/2767755637 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f22f00b6540 0x7f22f00b8a50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:31.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.617+0000 7f2309291640 1 --2- 172.21.15.67:0/2767755637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23041094a0 0x7f230419a990 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:31.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.617+0000 7f2309291640 1 --2- 172.21.15.67:0/2767755637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2304103b60 0x7f230419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:31.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.617+0000 7f2309291640 1 -- 172.21.15.67:0/2767755637 >> 172.21.15.67:0/2767755637 conn(0x7f230407b850 msgr2=0x7f23041075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:31.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.617+0000 7f2309291640 1 -- 172.21.15.67:0/2767755637 shutdown_connections 2024-09-21T06:34:31.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:31.617+0000 7f2309291640 1 -- 172.21.15.67:0/2767755637 wait complete. 2024-09-21T06:34:31.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:31 smithi203 ceph-mon[33735]: pgmap v284: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:31.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:31 smithi203 ceph-mon[33735]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T06:34:31.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:31 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2767755637' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T06:34:31.999 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:31 smithi067 ceph-mon[31232]: pgmap v284: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:31.999 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:31 smithi067 ceph-mon[31232]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T06:34:31.999 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:31 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2767755637' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T06:34:32.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:31 smithi100 ceph-mon[33401]: pgmap v284: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:32.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:31 smithi100 ceph-mon[33401]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T06:34:32.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:31 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2767755637' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T06:34:32.308 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T06:34:32.568 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:34:32.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:32 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2767755637' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-21T06:34:32.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:32 smithi067 ceph-mon[31232]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T06:34:32.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:32 smithi067 ceph-mon[31232]: pgmap v287: 97 pgs: 1 creating+activating, 14 creating+peering, 75 unknown, 7 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:32.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:32 smithi067 ceph-mon[31232]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T06:34:32.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:32 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2767755637' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-21T06:34:32.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:32 smithi203 ceph-mon[33735]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T06:34:32.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:32 smithi203 ceph-mon[33735]: pgmap v287: 97 pgs: 1 creating+activating, 14 creating+peering, 75 unknown, 7 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:32.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:32 smithi203 ceph-mon[33735]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T06:34:33.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:32 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2767755637' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-21T06:34:33.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:32 smithi100 ceph-mon[33401]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T06:34:33.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:32 smithi100 ceph-mon[33401]: pgmap v287: 97 pgs: 1 creating+activating, 14 creating+peering, 75 unknown, 7 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:33.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:32 smithi100 ceph-mon[33401]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T06:34:33.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.063+0000 7fc93ada1640 1 -- 172.21.15.67:0/2412539672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc934102870 msgr2=0x7fc934102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:33.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.063+0000 7fc93ada1640 1 --2- 172.21.15.67:0/2412539672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc934102870 0x7fc934102c80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc928009900 tx=0x7fc92802a410 comp rx=0 tx=0).stop 2024-09-21T06:34:33.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.063+0000 7fc93ada1640 1 -- 172.21.15.67:0/2412539672 shutdown_connections 2024-09-21T06:34:33.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.063+0000 7fc93ada1640 1 --2- 172.21.15.67:0/2412539672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc934108b00 0x7fc93410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:33.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.063+0000 7fc93ada1640 1 --2- 172.21.15.67:0/2412539672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9341081b0 0x7fc9341085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:33.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.063+0000 7fc93ada1640 1 --2- 172.21.15.67:0/2412539672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc934102870 0x7fc934102c80 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:33.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.063+0000 7fc93ada1640 1 -- 172.21.15.67:0/2412539672 >> 172.21.15.67:0/2412539672 conn(0x7fc9340fe2c0 msgr2=0x7fc934100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:33.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.063+0000 7fc93ada1640 1 -- 172.21.15.67:0/2412539672 shutdown_connections 2024-09-21T06:34:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.063+0000 7fc93ada1640 1 -- 172.21.15.67:0/2412539672 wait complete. 2024-09-21T06:34:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.064+0000 7fc93ada1640 1 Processor -- start 2024-09-21T06:34:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.064+0000 7fc93ada1640 1 -- start start 2024-09-21T06:34:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.064+0000 7fc93ada1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc934102870 0x7fc93419b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.064+0000 7fc93ada1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9341081b0 0x7fc93419b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.064+0000 7fc93ada1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc934108b00 0x7fc93419baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.064+0000 7fc93ada1640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc934112680 con 0x7fc934102870 2024-09-21T06:34:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.064+0000 7fc93ada1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc934112500 con 0x7fc934108b00 2024-09-21T06:34:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.064+0000 7fc93ada1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc934112800 con 0x7fc9341081b0 2024-09-21T06:34:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.064+0000 7fc939d9f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc934102870 0x7fc93419b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.064+0000 7fc939d9f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc934102870 0x7fc93419b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35372/0 (socket says 172.21.15.67:35372) 2024-09-21T06:34:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.064+0000 7fc939d9f640 1 -- 172.21.15.67:0/3990456786 learned_addr learned my addr 172.21.15.67:0/3990456786 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:34:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.065+0000 7fc93a5a0640 1 --2- 172.21.15.67:0/3990456786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc934108b00 0x7fc93419baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.065+0000 7fc93959e640 1 --2- 172.21.15.67:0/3990456786 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9341081b0 0x7fc93419b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:33.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.065+0000 7fc939d9f640 1 -- 172.21.15.67:0/3990456786 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9341081b0 msgr2=0x7fc93419b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:33.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.065+0000 7fc939d9f640 1 --2- 172.21.15.67:0/3990456786 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9341081b0 0x7fc93419b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:33.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.065+0000 7fc939d9f640 1 -- 172.21.15.67:0/3990456786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc934108b00 msgr2=0x7fc93419baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:33.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.065+0000 7fc939d9f640 1 --2- 172.21.15.67:0/3990456786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc934108b00 0x7fc93419baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:33.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.065+0000 7fc939d9f640 1 -- 172.21.15.67:0/3990456786 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9341a6270 con 0x7fc934102870 2024-09-21T06:34:33.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.065+0000 7fc93a5a0640 1 --2- 172.21.15.67:0/3990456786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc934108b00 0x7fc93419baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:33.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.065+0000 7fc93959e640 1 --2- 172.21.15.67:0/3990456786 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9341081b0 0x7fc93419b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:33.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.065+0000 7fc939d9f640 1 --2- 172.21.15.67:0/3990456786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc934102870 0x7fc93419b070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc928009230 tx=0x7fc928004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:33.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.065+0000 7fc926ffd640 1 -- 172.21.15.67:0/3990456786 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92803f030 con 0x7fc934102870 2024-09-21T06:34:33.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.065+0000 7fc926ffd640 1 -- 172.21.15.67:0/3990456786 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc928003a80 con 0x7fc934102870 2024-09-21T06:34:33.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.065+0000 7fc93ada1640 1 -- 172.21.15.67:0/3990456786 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9341a6500 con 0x7fc934102870 2024-09-21T06:34:33.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.065+0000 7fc926ffd640 1 -- 172.21.15.67:0/3990456786 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc928035070 con 0x7fc934102870 2024-09-21T06:34:33.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.065+0000 7fc93ada1640 1 -- 172.21.15.67:0/3990456786 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9341a69e0 con 0x7fc934102870 2024-09-21T06:34:33.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.066+0000 7fc93ada1640 1 -- 172.21.15.67:0/3990456786 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc934102c80 con 0x7fc934102870 2024-09-21T06:34:33.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.068+0000 7fc926ffd640 1 -- 172.21.15.67:0/3990456786 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fc928042030 con 0x7fc934102870 2024-09-21T06:34:33.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.069+0000 7fc926ffd640 1 --2- 172.21.15.67:0/3990456786 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc9200b6540 0x7fc9200b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:33.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.069+0000 7fc93959e640 1 --2- 172.21.15.67:0/3990456786 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc9200b6540 0x7fc9200b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:33.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.069+0000 7fc926ffd640 1 -- 172.21.15.67:0/3990456786 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7879+0+0 (secure 0 0 0) 0x7fc928107d00 con 0x7fc934102870 2024-09-21T06:34:33.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.070+0000 7fc93959e640 1 --2- 172.21.15.67:0/3990456786 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc9200b6540 0x7fc9200b8a50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc9300058c0 tx=0x7fc930005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:33.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.073+0000 7fc926ffd640 1 -- 172.21.15.67:0/3990456786 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc9280ce720 con 0x7fc934102870 2024-09-21T06:34:33.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.304+0000 7fc93ada1640 1 -- 172.21.15.67:0/3990456786 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc93402e640 con 0x7fc934102870 2024-09-21T06:34:33.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.645+0000 7fc926ffd640 1 -- 172.21.15.67:0/3990456786 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fc9280d2b00 con 0x7fc934102870 2024-09-21T06:34:33.648 INFO:teuthology.orchestra.run.smithi067.stderr:new fs with metadata pool 2 and data pool 3 2024-09-21T06:34:33.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.648+0000 7fc93ada1640 1 -- 172.21.15.67:0/3990456786 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc9200b6540 msgr2=0x7fc9200b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:33.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.648+0000 7fc93ada1640 1 --2- 172.21.15.67:0/3990456786 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc9200b6540 0x7fc9200b8a50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc9300058c0 tx=0x7fc930005850 comp rx=0 tx=0).stop 2024-09-21T06:34:33.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.648+0000 7fc93ada1640 1 -- 172.21.15.67:0/3990456786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc934102870 msgr2=0x7fc93419b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:33.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.648+0000 7fc93ada1640 1 --2- 172.21.15.67:0/3990456786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc934102870 0x7fc93419b070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc928009230 tx=0x7fc928004270 comp rx=0 tx=0).stop 2024-09-21T06:34:33.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.648+0000 7fc93ada1640 1 -- 172.21.15.67:0/3990456786 shutdown_connections 2024-09-21T06:34:33.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.648+0000 7fc93ada1640 1 --2- 172.21.15.67:0/3990456786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc934108b00 0x7fc93419baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:33.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.648+0000 7fc93ada1640 1 --2- 172.21.15.67:0/3990456786 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc9200b6540 0x7fc9200b8a50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:33.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.648+0000 7fc93ada1640 1 --2- 172.21.15.67:0/3990456786 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9341081b0 0x7fc93419b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:33.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.648+0000 7fc93ada1640 1 --2- 172.21.15.67:0/3990456786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc934102870 0x7fc93419b070 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:33.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.648+0000 7fc93ada1640 1 -- 172.21.15.67:0/3990456786 >> 172.21.15.67:0/3990456786 conn(0x7fc9340fe2c0 msgr2=0x7fc9341006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:33.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.648+0000 7fc93ada1640 1 -- 172.21.15.67:0/3990456786 shutdown_connections 2024-09-21T06:34:33.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:33.648+0000 7fc93ada1640 1 -- 172.21.15.67:0/3990456786 wait complete. 2024-09-21T06:34:33.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:33 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3990456786' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T06:34:33.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:33 smithi203 ceph-mon[33735]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T06:34:34.025 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:33 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3990456786' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T06:34:34.025 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:33 smithi067 ceph-mon[31232]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T06:34:34.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:33 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3990456786' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T06:34:34.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:33 smithi100 ceph-mon[33401]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T06:34:34.433 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-21T06:34:34.434 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-21T06:34:34.692 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:34:34.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:34 smithi067 ceph-mon[31232]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T06:34:34.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:34 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3990456786' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T06:34:34.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:34 smithi067 ceph-mon[31232]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T06:34:34.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:34 smithi067 ceph-mon[31232]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T06:34:34.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:34 smithi067 ceph-mon[31232]: pgmap v290: 97 pgs: 1 creating+activating, 27 creating+peering, 45 unknown, 24 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:34.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:34 smithi203 ceph-mon[33735]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T06:34:34.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:34 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3990456786' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T06:34:34.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:34 smithi203 ceph-mon[33735]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T06:34:34.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:34 smithi203 ceph-mon[33735]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T06:34:34.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:34 smithi203 ceph-mon[33735]: pgmap v290: 97 pgs: 1 creating+activating, 27 creating+peering, 45 unknown, 24 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:35.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:34 smithi100 ceph-mon[33401]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T06:34:35.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:34 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3990456786' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T06:34:35.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:34 smithi100 ceph-mon[33401]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T06:34:35.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:34 smithi100 ceph-mon[33401]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T06:34:35.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:34 smithi100 ceph-mon[33401]: pgmap v290: 97 pgs: 1 creating+activating, 27 creating+peering, 45 unknown, 24 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:35.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.225+0000 7f6eb3b7e640 1 -- 172.21.15.67:0/2080239870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6eac1049b0 msgr2=0x7f6eac104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.225+0000 7f6eb3b7e640 1 --2- 172.21.15.67:0/2080239870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6eac1049b0 0x7f6eac104dc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8009900 tx=0x7f6ea802a410 comp rx=0 tx=0).stop 2024-09-21T06:34:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.225+0000 7f6eb3b7e640 1 -- 172.21.15.67:0/2080239870 shutdown_connections 2024-09-21T06:34:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.225+0000 7f6eb3b7e640 1 --2- 172.21.15.67:0/2080239870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6eac105300 0x7f6eac10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.225+0000 7f6eb3b7e640 1 --2- 172.21.15.67:0/2080239870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6eac10de10 0x7f6eac110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.225+0000 7f6eb3b7e640 1 --2- 172.21.15.67:0/2080239870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6eac1049b0 0x7f6eac104dc0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.225+0000 7f6eb3b7e640 1 -- 172.21.15.67:0/2080239870 >> 172.21.15.67:0/2080239870 conn(0x7f6eac079730 msgr2=0x7f6eac079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.225+0000 7f6eb3b7e640 1 -- 172.21.15.67:0/2080239870 shutdown_connections 2024-09-21T06:34:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.225+0000 7f6eb3b7e640 1 -- 172.21.15.67:0/2080239870 wait complete. 2024-09-21T06:34:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.226+0000 7f6eb3b7e640 1 Processor -- start 2024-09-21T06:34:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.226+0000 7f6eb3b7e640 1 -- start start 2024-09-21T06:34:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.226+0000 7f6eb3b7e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6eac1049b0 0x7f6eac19a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.226+0000 7f6eb3b7e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6eac105300 0x7f6eac19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.226+0000 7f6eb3b7e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6eac10de10 0x7f6eac19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.226+0000 7f6eb3b7e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac113940 con 0x7f6eac1049b0 2024-09-21T06:34:35.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.226+0000 7f6eb3b7e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac1137c0 con 0x7f6eac105300 2024-09-21T06:34:35.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.226+0000 7f6eb3b7e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac113ac0 con 0x7f6eac10de10 2024-09-21T06:34:35.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.226+0000 7f6eb191a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6eac1049b0 0x7f6eac19a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:35.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.226+0000 7f6eb191a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6eac1049b0 0x7f6eac19a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35394/0 (socket says 172.21.15.67:35394) 2024-09-21T06:34:35.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.226+0000 7f6eb191a640 1 -- 172.21.15.67:0/3214269538 learned_addr learned my addr 172.21.15.67:0/3214269538 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:34:35.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.226+0000 7f6eb1119640 1 --2- 172.21.15.67:0/3214269538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6eac105300 0x7f6eac19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:35.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.226+0000 7f6eb211b640 1 --2- 172.21.15.67:0/3214269538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6eac10de10 0x7f6eac19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:35.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.227+0000 7f6eb191a640 1 -- 172.21.15.67:0/3214269538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6eac10de10 msgr2=0x7f6eac19fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:35.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.227+0000 7f6eb191a640 1 --2- 172.21.15.67:0/3214269538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6eac10de10 0x7f6eac19fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:35.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.227+0000 7f6eb191a640 1 -- 172.21.15.67:0/3214269538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6eac105300 msgr2=0x7f6eac19a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:35.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.227+0000 7f6eb191a640 1 --2- 172.21.15.67:0/3214269538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6eac105300 0x7f6eac19a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:35.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.227+0000 7f6eb191a640 1 -- 172.21.15.67:0/3214269538 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eac1a0280 con 0x7f6eac1049b0 2024-09-21T06:34:35.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.227+0000 7f6eb211b640 1 --2- 172.21.15.67:0/3214269538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6eac10de10 0x7f6eac19fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:35.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.227+0000 7f6eb191a640 1 --2- 172.21.15.67:0/3214269538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6eac1049b0 0x7f6eac19a3d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8005940 tx=0x7f6ea80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:35.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.227+0000 7f6ea2ffd640 1 -- 172.21.15.67:0/3214269538 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea803f030 con 0x7f6eac1049b0 2024-09-21T06:34:35.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.227+0000 7f6ea2ffd640 1 -- 172.21.15.67:0/3214269538 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ea8003c90 con 0x7f6eac1049b0 2024-09-21T06:34:35.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.227+0000 7f6eb3b7e640 1 -- 172.21.15.67:0/3214269538 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eac1a0510 con 0x7f6eac1049b0 2024-09-21T06:34:35.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.227+0000 7f6ea2ffd640 1 -- 172.21.15.67:0/3214269538 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea8035070 con 0x7f6eac1049b0 2024-09-21T06:34:35.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.227+0000 7f6eb3b7e640 1 -- 172.21.15.67:0/3214269538 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eac1a09f0 con 0x7f6eac1049b0 2024-09-21T06:34:35.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.228+0000 7f6eb3b7e640 1 -- 172.21.15.67:0/3214269538 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eac104dc0 con 0x7f6eac1049b0 2024-09-21T06:34:35.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.230+0000 7f6ea2ffd640 1 -- 172.21.15.67:0/3214269538 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f6ea8042030 con 0x7f6eac1049b0 2024-09-21T06:34:35.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.230+0000 7f6ea2ffd640 1 --2- 172.21.15.67:0/3214269538 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f6e980b6540 0x7f6e980b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:35.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.231+0000 7f6ea2ffd640 1 -- 172.21.15.67:0/3214269538 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f6ea8107770 con 0x7f6eac1049b0 2024-09-21T06:34:35.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.231+0000 7f6eb1119640 1 --2- 172.21.15.67:0/3214269538 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f6e980b6540 0x7f6e980b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:35.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.231+0000 7f6eb1119640 1 --2- 172.21.15.67:0/3214269538 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f6e980b6540 0x7f6e980b8a50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c005e10 tx=0x7f6e9c005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:35.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.235+0000 7f6ea2ffd640 1 -- 172.21.15.67:0/3214269538 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6ea80ce150 con 0x7f6eac1049b0 2024-09-21T06:34:35.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.430+0000 7f6eb3b7e640 1 -- 172.21.15.67:0/3214269538 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f6eac02e640 con 0x7f6eac1049b0 2024-09-21T06:34:35.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.657+0000 7f6ea2ffd640 1 -- 172.21.15.67:0/3214269538 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v76) ==== 157+0+0 (secure 0 0 0) 0x7f6ea80d2530 con 0x7f6eac1049b0 2024-09-21T06:34:35.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.659+0000 7f6eb3b7e640 1 -- 172.21.15.67:0/3214269538 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f6e980b6540 msgr2=0x7f6e980b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:35.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.659+0000 7f6eb3b7e640 1 --2- 172.21.15.67:0/3214269538 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f6e980b6540 0x7f6e980b8a50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c005e10 tx=0x7f6e9c005da0 comp rx=0 tx=0).stop 2024-09-21T06:34:35.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.659+0000 7f6eb3b7e640 1 -- 172.21.15.67:0/3214269538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6eac1049b0 msgr2=0x7f6eac19a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:35.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.659+0000 7f6eb3b7e640 1 --2- 172.21.15.67:0/3214269538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6eac1049b0 0x7f6eac19a3d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8005940 tx=0x7f6ea80042f0 comp rx=0 tx=0).stop 2024-09-21T06:34:35.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.660+0000 7f6eb3b7e640 1 -- 172.21.15.67:0/3214269538 shutdown_connections 2024-09-21T06:34:35.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.660+0000 7f6eb3b7e640 1 --2- 172.21.15.67:0/3214269538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6eac105300 0x7f6eac19a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:35.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.660+0000 7f6eb3b7e640 1 --2- 172.21.15.67:0/3214269538 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f6e980b6540 0x7f6e980b8a50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:35.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.660+0000 7f6eb3b7e640 1 --2- 172.21.15.67:0/3214269538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6eac10de10 0x7f6eac19fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:35.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.660+0000 7f6eb3b7e640 1 --2- 172.21.15.67:0/3214269538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6eac1049b0 0x7f6eac19a3d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:35.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.660+0000 7f6eb3b7e640 1 -- 172.21.15.67:0/3214269538 >> 172.21.15.67:0/3214269538 conn(0x7f6eac079730 msgr2=0x7f6eac0ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:35.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.660+0000 7f6eb3b7e640 1 -- 172.21.15.67:0/3214269538 shutdown_connections 2024-09-21T06:34:35.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:35.660+0000 7f6eb3b7e640 1 -- 172.21.15.67:0/3214269538 wait complete. 2024-09-21T06:34:35.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:35 smithi067 ceph-mon[31232]: mds.? [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] up:active 2024-09-21T06:34:35.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:35 smithi067 ceph-mon[31232]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:34:35.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:35 smithi067 ceph-mon[31232]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:34:35.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:35 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3214269538' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T06:34:35.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:35 smithi203 ceph-mon[33735]: mds.? [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] up:active 2024-09-21T06:34:35.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:35 smithi203 ceph-mon[33735]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:34:35.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:35 smithi203 ceph-mon[33735]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:34:35.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:35 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3214269538' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T06:34:36.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:35 smithi100 ceph-mon[33401]: mds.? [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] up:active 2024-09-21T06:34:36.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:35 smithi100 ceph-mon[33401]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:34:36.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:35 smithi100 ceph-mon[33401]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:34:36.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:35 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3214269538' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T06:34:36.330 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T06:34:36.594 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:34:36.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:36 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3214269538' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-21T06:34:36.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:36 smithi067 ceph-mon[31232]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T06:34:36.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:36 smithi067 ceph-mon[31232]: pgmap v292: 97 pgs: 1 creating+activating, 13 creating+peering, 25 unknown, 58 active+clean; 578 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 6 op/s 2024-09-21T06:34:36.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:36 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3214269538' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-21T06:34:36.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:36 smithi203 ceph-mon[33735]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T06:34:36.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:36 smithi203 ceph-mon[33735]: pgmap v292: 97 pgs: 1 creating+activating, 13 creating+peering, 25 unknown, 58 active+clean; 578 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 6 op/s 2024-09-21T06:34:37.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:36 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3214269538' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-21T06:34:37.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:36 smithi100 ceph-mon[33401]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T06:34:37.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:36 smithi100 ceph-mon[33401]: pgmap v292: 97 pgs: 1 creating+activating, 13 creating+peering, 25 unknown, 58 active+clean; 578 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 6 op/s 2024-09-21T06:34:37.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.148+0000 7f2670bec640 1 -- 172.21.15.67:0/3871893480 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f266c108d20 msgr2=0x7f266c10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:37.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.148+0000 7f2670bec640 1 --2- 172.21.15.67:0/3871893480 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f266c108d20 0x7f266c10f550 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2664009ca0 tx=0x7f266402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:34:37.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.148+0000 7f2670bec640 1 -- 172.21.15.67:0/3871893480 shutdown_connections 2024-09-21T06:34:37.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.148+0000 7f2670bec640 1 --2- 172.21.15.67:0/3871893480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f266c1083d0 0x7f266c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:37.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.148+0000 7f2670bec640 1 --2- 172.21.15.67:0/3871893480 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f266c108d20 0x7f266c10f550 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:37.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.148+0000 7f2670bec640 1 --2- 172.21.15.67:0/3871893480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f266c102a90 0x7f266c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:37.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.148+0000 7f2670bec640 1 -- 172.21.15.67:0/3871893480 >> 172.21.15.67:0/3871893480 conn(0x7f266c079840 msgr2=0x7f266c079c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:37.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.148+0000 7f2670bec640 1 -- 172.21.15.67:0/3871893480 shutdown_connections 2024-09-21T06:34:37.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.148+0000 7f2670bec640 1 -- 172.21.15.67:0/3871893480 wait complete. 2024-09-21T06:34:37.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.149+0000 7f2670bec640 1 Processor -- start 2024-09-21T06:34:37.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.149+0000 7f2670bec640 1 -- start start 2024-09-21T06:34:37.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.150+0000 7f2670bec640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f266c102a90 0x7f266c19a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:37.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.150+0000 7f2670bec640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f266c1083d0 0x7f266c19a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:37.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.150+0000 7f2670bec640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f266c108d20 0x7f266c19fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:37.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.150+0000 7f2670bec640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f266c1128e0 con 0x7f266c108d20 2024-09-21T06:34:37.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.150+0000 7f2670bec640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f266c112760 con 0x7f266c102a90 2024-09-21T06:34:37.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.150+0000 7f2670bec640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f266c112a60 con 0x7f266c1083d0 2024-09-21T06:34:37.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.150+0000 7f266ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f266c108d20 0x7f266c19fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:37.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.150+0000 7f266ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f266c108d20 0x7f266c19fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35398/0 (socket says 172.21.15.67:35398) 2024-09-21T06:34:37.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.150+0000 7f266ad9d640 1 -- 172.21.15.67:0/71619583 learned_addr learned my addr 172.21.15.67:0/71619583 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:34:37.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.150+0000 7f266a59c640 1 --2- 172.21.15.67:0/71619583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f266c102a90 0x7f266c19a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:37.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.150+0000 7f2669d9b640 1 --2- 172.21.15.67:0/71619583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f266c1083d0 0x7f266c19a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:37.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.150+0000 7f266ad9d640 1 -- 172.21.15.67:0/71619583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f266c1083d0 msgr2=0x7f266c19a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:37.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.150+0000 7f266ad9d640 1 --2- 172.21.15.67:0/71619583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f266c1083d0 0x7f266c19a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:37.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.150+0000 7f266ad9d640 1 -- 172.21.15.67:0/71619583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f266c102a90 msgr2=0x7f266c19a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:37.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.150+0000 7f266ad9d640 1 --2- 172.21.15.67:0/71619583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f266c102a90 0x7f266c19a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:37.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.150+0000 7f266ad9d640 1 -- 172.21.15.67:0/71619583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f266c1a0210 con 0x7f266c108d20 2024-09-21T06:34:37.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.150+0000 7f266ad9d640 1 --2- 172.21.15.67:0/71619583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f266c108d20 0x7f266c19fba0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2664005cc0 tx=0x7f2664032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:37.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.150+0000 7f265b7fe640 1 -- 172.21.15.67:0/71619583 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2664032e00 con 0x7f266c108d20 2024-09-21T06:34:37.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.151+0000 7f265b7fe640 1 -- 172.21.15.67:0/71619583 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26640044d0 con 0x7f266c108d20 2024-09-21T06:34:37.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.151+0000 7f2670bec640 1 -- 172.21.15.67:0/71619583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f266c1a04a0 con 0x7f266c108d20 2024-09-21T06:34:37.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.151+0000 7f265b7fe640 1 -- 172.21.15.67:0/71619583 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2664030b70 con 0x7f266c108d20 2024-09-21T06:34:37.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.151+0000 7f2670bec640 1 -- 172.21.15.67:0/71619583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f266c1a09e0 con 0x7f266c108d20 2024-09-21T06:34:37.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.152+0000 7f2670bec640 1 -- 172.21.15.67:0/71619583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f266c103d10 con 0x7f266c108d20 2024-09-21T06:34:37.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.153+0000 7f265b7fe640 1 -- 172.21.15.67:0/71619583 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f2664030d10 con 0x7f266c108d20 2024-09-21T06:34:37.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.154+0000 7f265b7fe640 1 --2- 172.21.15.67:0/71619583 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f26540b6540 0x7f26540b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:37.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.154+0000 7f266a59c640 1 --2- 172.21.15.67:0/71619583 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f26540b6540 0x7f26540b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:37.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.154+0000 7f265b7fe640 1 -- 172.21.15.67:0/71619583 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8277+0+0 (secure 0 0 0) 0x7f2664108650 con 0x7f266c108d20 2024-09-21T06:34:37.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.155+0000 7f266a59c640 1 --2- 172.21.15.67:0/71619583 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f26540b6540 0x7f26540b8a50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f265c005fd0 tx=0x7f265c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:37.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.157+0000 7f265b7fe640 1 -- 172.21.15.67:0/71619583 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f26640ceeb0 con 0x7f266c108d20 2024-09-21T06:34:37.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:37.350+0000 7f2670bec640 1 -- 172.21.15.67:0/71619583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f266c19b490 con 0x7f266c108d20 2024-09-21T06:34:37.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:37 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/71619583' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T06:34:37.804 INFO:journalctl@ceph.mds.b.smithi100.stdout:Sep 21 06:34:37 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-b[61079]: 2024-09-21T06:34:37.532+0000 7f508e3a5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T06:34:37.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:37 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/71619583' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T06:34:38.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:37 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/71619583' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T06:34:38.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:38.677+0000 7f265b7fe640 1 -- 172.21.15.67:0/71619583 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v78) ==== 228+0+0 (secure 0 0 0) 0x7f26640d3290 con 0x7f266c108d20 2024-09-21T06:34:38.680 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data_ec' created 2024-09-21T06:34:38.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:38.680+0000 7f2670bec640 1 -- 172.21.15.67:0/71619583 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f26540b6540 msgr2=0x7f26540b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:38.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:38.680+0000 7f2670bec640 1 --2- 172.21.15.67:0/71619583 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f26540b6540 0x7f26540b8a50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f265c005fd0 tx=0x7f265c00b390 comp rx=0 tx=0).stop 2024-09-21T06:34:38.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:38.680+0000 7f2670bec640 1 -- 172.21.15.67:0/71619583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f266c108d20 msgr2=0x7f266c19fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:38.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:38.680+0000 7f2670bec640 1 --2- 172.21.15.67:0/71619583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f266c108d20 0x7f266c19fba0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2664005cc0 tx=0x7f2664032a10 comp rx=0 tx=0).stop 2024-09-21T06:34:38.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:38.681+0000 7f2670bec640 1 -- 172.21.15.67:0/71619583 shutdown_connections 2024-09-21T06:34:38.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:38.681+0000 7f2670bec640 1 --2- 172.21.15.67:0/71619583 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f26540b6540 0x7f26540b8a50 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:38.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:38.681+0000 7f2670bec640 1 --2- 172.21.15.67:0/71619583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f266c108d20 0x7f266c19fba0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:38.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:38.681+0000 7f2670bec640 1 --2- 172.21.15.67:0/71619583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f266c1083d0 0x7f266c19a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:38.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:38.681+0000 7f2670bec640 1 --2- 172.21.15.67:0/71619583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f266c102a90 0x7f266c19a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:38.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:38.681+0000 7f2670bec640 1 -- 172.21.15.67:0/71619583 >> 172.21.15.67:0/71619583 conn(0x7f266c079840 msgr2=0x7f266c06bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:38.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:38.681+0000 7f2670bec640 1 -- 172.21.15.67:0/71619583 shutdown_connections 2024-09-21T06:34:38.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:38.681+0000 7f2670bec640 1 -- 172.21.15.67:0/71619583 wait complete. 2024-09-21T06:34:38.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:38 smithi203 ceph-mon[33735]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T06:34:38.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:38 smithi203 ceph-mon[33735]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2024-09-21T06:34:39.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:38 smithi067 ceph-mon[31232]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T06:34:39.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:38 smithi067 ceph-mon[31232]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2024-09-21T06:34:39.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:38 smithi100 ceph-mon[33401]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T06:34:39.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:38 smithi100 ceph-mon[33401]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2024-09-21T06:34:39.437 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-21T06:34:39.714 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:34:39.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:39 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/71619583' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T06:34:39.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:39 smithi067 ceph-mon[31232]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T06:34:39.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:39 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/71619583' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T06:34:39.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:39 smithi203 ceph-mon[33735]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T06:34:40.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:39 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/71619583' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T06:34:40.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:39 smithi100 ceph-mon[33401]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T06:34:40.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.204+0000 7f2972a27640 1 -- 172.21.15.67:0/1788730682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f296c108d10 msgr2=0x7f296c10f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:40.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.204+0000 7f2972a27640 1 --2- 172.21.15.67:0/1788730682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f296c108d10 0x7f296c10f540 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2964009ca0 tx=0x7f296402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:34:40.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.204+0000 7f2972a27640 1 -- 172.21.15.67:0/1788730682 shutdown_connections 2024-09-21T06:34:40.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.204+0000 7f2972a27640 1 --2- 172.21.15.67:0/1788730682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f296c1083c0 0x7f296c1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:40.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.204+0000 7f2972a27640 1 --2- 172.21.15.67:0/1788730682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f296c108d10 0x7f296c10f540 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:40.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.204+0000 7f2972a27640 1 --2- 172.21.15.67:0/1788730682 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f296c102a80 0x7f296c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:40.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.204+0000 7f2972a27640 1 -- 172.21.15.67:0/1788730682 >> 172.21.15.67:0/1788730682 conn(0x7f296c079740 msgr2=0x7f296c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:40.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.204+0000 7f2972a27640 1 -- 172.21.15.67:0/1788730682 shutdown_connections 2024-09-21T06:34:40.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.205+0000 7f2972a27640 1 -- 172.21.15.67:0/1788730682 wait complete. 2024-09-21T06:34:40.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.205+0000 7f2972a27640 1 Processor -- start 2024-09-21T06:34:40.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.205+0000 7f2972a27640 1 -- start start 2024-09-21T06:34:40.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.206+0000 7f2972a27640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f296c102a80 0x7f296c199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:40.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.206+0000 7f2972a27640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f296c1083c0 0x7f296c1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:40.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.206+0000 7f2972a27640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f296c108d10 0x7f296c19ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:40.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.206+0000 7f2972a27640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f296c112880 con 0x7f296c102a80 2024-09-21T06:34:40.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.206+0000 7f2972a27640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f296c112700 con 0x7f296c108d10 2024-09-21T06:34:40.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.206+0000 7f2972a27640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f296c112a00 con 0x7f296c1083c0 2024-09-21T06:34:40.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.206+0000 7f296bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f296c102a80 0x7f296c199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:40.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.206+0000 7f296bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f296c102a80 0x7f296c199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49016/0 (socket says 172.21.15.67:49016) 2024-09-21T06:34:40.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.206+0000 7f296bfff640 1 -- 172.21.15.67:0/3681381849 learned_addr learned my addr 172.21.15.67:0/3681381849 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:34:40.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.206+0000 7f2970fc4640 1 --2- 172.21.15.67:0/3681381849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f296c108d10 0x7f296c19ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:40.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.206+0000 7f296bfff640 1 -- 172.21.15.67:0/3681381849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f296c1083c0 msgr2=0x7f296c1997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:40.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.206+0000 7f296bfff640 1 --2- 172.21.15.67:0/3681381849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f296c1083c0 0x7f296c1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:40.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.206+0000 7f296bfff640 1 -- 172.21.15.67:0/3681381849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f296c108d10 msgr2=0x7f296c19ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:40.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.206+0000 7f296bfff640 1 --2- 172.21.15.67:0/3681381849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f296c108d10 0x7f296c19ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:40.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.206+0000 7f296bfff640 1 -- 172.21.15.67:0/3681381849 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f296c19f180 con 0x7f296c102a80 2024-09-21T06:34:40.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.207+0000 7f296bfff640 1 --2- 172.21.15.67:0/3681381849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f296c102a80 0x7f296c199210 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f295c00d770 tx=0x7f295c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:40.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.207+0000 7f29697fa640 1 -- 172.21.15.67:0/3681381849 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295c017070 con 0x7f296c102a80 2024-09-21T06:34:40.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.207+0000 7f2972a27640 1 -- 172.21.15.67:0/3681381849 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f296c19f470 con 0x7f296c102a80 2024-09-21T06:34:40.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.207+0000 7f29697fa640 1 -- 172.21.15.67:0/3681381849 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f295c0054b0 con 0x7f296c102a80 2024-09-21T06:34:40.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.207+0000 7f29697fa640 1 -- 172.21.15.67:0/3681381849 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295c004c60 con 0x7f296c102a80 2024-09-21T06:34:40.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.207+0000 7f2972a27640 1 -- 172.21.15.67:0/3681381849 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f296c1a5bd0 con 0x7f296c102a80 2024-09-21T06:34:40.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.208+0000 7f2972a27640 1 -- 172.21.15.67:0/3681381849 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f296c103d00 con 0x7f296c102a80 2024-09-21T06:34:40.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.210+0000 7f29697fa640 1 -- 172.21.15.67:0/3681381849 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f295c004e00 con 0x7f296c102a80 2024-09-21T06:34:40.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.211+0000 7f29697fa640 1 --2- 172.21.15.67:0/3681381849 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f29540b6510 0x7f29540b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:40.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.211+0000 7f296b7fe640 1 --2- 172.21.15.67:0/3681381849 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f29540b6510 0x7f29540b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:40.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.211+0000 7f29697fa640 1 -- 172.21.15.67:0/3681381849 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8770+0+0 (secure 0 0 0) 0x7f295c0e73f0 con 0x7f296c102a80 2024-09-21T06:34:40.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.211+0000 7f296b7fe640 1 --2- 172.21.15.67:0/3681381849 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f29540b6510 0x7f29540b8a20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2960005fd0 tx=0x7f2960005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:40.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.214+0000 7f29697fa640 1 -- 172.21.15.67:0/3681381849 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f295c0ada60 con 0x7f296c102a80 2024-09-21T06:34:40.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.405+0000 7f2972a27640 1 -- 172.21.15.67:0/3681381849 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f296c19a340 con 0x7f296c102a80 2024-09-21T06:34:40.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.693+0000 7f29697fa640 1 -- 172.21.15.67:0/3681381849 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v80) ==== 169+0+0 (secure 0 0 0) 0x7f295c0b1e40 con 0x7f296c102a80 2024-09-21T06:34:40.696 INFO:teuthology.orchestra.run.smithi067.stderr:set pool 4 allow_ec_overwrites to true 2024-09-21T06:34:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.697+0000 7f2972a27640 1 -- 172.21.15.67:0/3681381849 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f29540b6510 msgr2=0x7f29540b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.697+0000 7f2972a27640 1 --2- 172.21.15.67:0/3681381849 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f29540b6510 0x7f29540b8a20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2960005fd0 tx=0x7f2960005850 comp rx=0 tx=0).stop 2024-09-21T06:34:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.697+0000 7f2972a27640 1 -- 172.21.15.67:0/3681381849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f296c102a80 msgr2=0x7f296c199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.697+0000 7f2972a27640 1 --2- 172.21.15.67:0/3681381849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f296c102a80 0x7f296c199210 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f295c00d770 tx=0x7f295c00dc30 comp rx=0 tx=0).stop 2024-09-21T06:34:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.697+0000 7f2972a27640 1 -- 172.21.15.67:0/3681381849 shutdown_connections 2024-09-21T06:34:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.697+0000 7f2972a27640 1 --2- 172.21.15.67:0/3681381849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f296c108d10 0x7f296c19ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.697+0000 7f2972a27640 1 --2- 172.21.15.67:0/3681381849 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f29540b6510 0x7f29540b8a20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.697+0000 7f2972a27640 1 --2- 172.21.15.67:0/3681381849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f296c1083c0 0x7f296c1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.697+0000 7f2972a27640 1 --2- 172.21.15.67:0/3681381849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f296c102a80 0x7f296c199210 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.697+0000 7f2972a27640 1 -- 172.21.15.67:0/3681381849 >> 172.21.15.67:0/3681381849 conn(0x7f296c079740 msgr2=0x7f296c1043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.697+0000 7f2972a27640 1 -- 172.21.15.67:0/3681381849 shutdown_connections 2024-09-21T06:34:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:40.697+0000 7f2972a27640 1 -- 172.21.15.67:0/3681381849 wait complete. 2024-09-21T06:34:40.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:40 smithi203 ceph-mon[33735]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T06:34:40.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:40 smithi203 ceph-mon[33735]: pgmap v297: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 735 B/s wr, 3 op/s 2024-09-21T06:34:40.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:40 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3681381849' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T06:34:41.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:40 smithi067 ceph-mon[31232]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T06:34:41.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:40 smithi067 ceph-mon[31232]: pgmap v297: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 735 B/s wr, 3 op/s 2024-09-21T06:34:41.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:40 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3681381849' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T06:34:41.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:40 smithi100 ceph-mon[33401]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T06:34:41.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:40 smithi100 ceph-mon[33401]: pgmap v297: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 735 B/s wr, 3 op/s 2024-09-21T06:34:41.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:40 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3681381849' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T06:34:41.380 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-21T06:34:41.641 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:34:41.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:41 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3681381849' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-21T06:34:41.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:41 smithi203 ceph-mon[33735]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T06:34:42.028 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:41 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3681381849' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-21T06:34:42.028 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:41 smithi067 ceph-mon[31232]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T06:34:42.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:41 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3681381849' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-21T06:34:42.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:41 smithi100 ceph-mon[33401]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T06:34:42.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.122+0000 7f3fb7a35640 1 -- 172.21.15.67:0/4163553465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fb0103b60 msgr2=0x7f3fb0103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:42.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.122+0000 7f3fb7a35640 1 --2- 172.21.15.67:0/4163553465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fb0103b60 0x7f3fb0103f70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4009900 tx=0x7f3fa402a410 comp rx=0 tx=0).stop 2024-09-21T06:34:42.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.122+0000 7f3fb7a35640 1 -- 172.21.15.67:0/4163553465 shutdown_connections 2024-09-21T06:34:42.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.122+0000 7f3fb7a35640 1 --2- 172.21.15.67:0/4163553465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fb01094a0 0x7f3fb01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:42.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.122+0000 7f3fb7a35640 1 --2- 172.21.15.67:0/4163553465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fb0109df0 0x7f3fb0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:42.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.122+0000 7f3fb7a35640 1 --2- 172.21.15.67:0/4163553465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fb0103b60 0x7f3fb0103f70 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:42.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.122+0000 7f3fb7a35640 1 -- 172.21.15.67:0/4163553465 >> 172.21.15.67:0/4163553465 conn(0x7f3fb007b850 msgr2=0x7f3fb007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:42.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.122+0000 7f3fb7a35640 1 -- 172.21.15.67:0/4163553465 shutdown_connections 2024-09-21T06:34:42.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.122+0000 7f3fb7a35640 1 -- 172.21.15.67:0/4163553465 wait complete. 2024-09-21T06:34:42.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.123+0000 7f3fb7a35640 1 Processor -- start 2024-09-21T06:34:42.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.123+0000 7f3fb7a35640 1 -- start start 2024-09-21T06:34:42.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.123+0000 7f3fb7a35640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fb0103b60 0x7f3fb019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:42.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.123+0000 7f3fb7a35640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fb01094a0 0x7f3fb019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:42.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.123+0000 7f3fb7a35640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fb0109df0 0x7f3fb019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:42.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.123+0000 7f3fb7a35640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb01139c0 con 0x7f3fb0103b60 2024-09-21T06:34:42.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.123+0000 7f3fb7a35640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb0113840 con 0x7f3fb01094a0 2024-09-21T06:34:42.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.123+0000 7f3fb7a35640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb0113b40 con 0x7f3fb0109df0 2024-09-21T06:34:42.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.123+0000 7f3fb57d1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fb0103b60 0x7f3fb019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:42.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.123+0000 7f3fb57d1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fb0103b60 0x7f3fb019a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49044/0 (socket says 172.21.15.67:49044) 2024-09-21T06:34:42.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.123+0000 7f3fb57d1640 1 -- 172.21.15.67:0/2428280098 learned_addr learned my addr 172.21.15.67:0/2428280098 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:34:42.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.123+0000 7f3fb4fd0640 1 --2- 172.21.15.67:0/2428280098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fb01094a0 0x7f3fb019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:42.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.124+0000 7f3fb57d1640 1 -- 172.21.15.67:0/2428280098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fb0109df0 msgr2=0x7f3fb019fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:42.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.124+0000 7f3fb5fd2640 1 --2- 172.21.15.67:0/2428280098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fb0109df0 0x7f3fb019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.124+0000 7f3fb57d1640 1 --2- 172.21.15.67:0/2428280098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fb0109df0 0x7f3fb019fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.124+0000 7f3fb57d1640 1 -- 172.21.15.67:0/2428280098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fb01094a0 msgr2=0x7f3fb019a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.124+0000 7f3fb57d1640 1 --2- 172.21.15.67:0/2428280098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fb01094a0 0x7f3fb019a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.124+0000 7f3fb57d1640 1 -- 172.21.15.67:0/2428280098 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb01a0380 con 0x7f3fb0103b60 2024-09-21T06:34:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.124+0000 7f3fb4fd0640 1 --2- 172.21.15.67:0/2428280098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fb01094a0 0x7f3fb019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.124+0000 7f3fb5fd2640 1 --2- 172.21.15.67:0/2428280098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fb0109df0 0x7f3fb019fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:34:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.124+0000 7f3fb57d1640 1 --2- 172.21.15.67:0/2428280098 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fb0103b60 0x7f3fb019a410 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4009a20 tx=0x7f3fa4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.124+0000 7f3fa27fc640 1 -- 172.21.15.67:0/2428280098 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa403f030 con 0x7f3fb0103b60 2024-09-21T06:34:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.124+0000 7f3fa27fc640 1 -- 172.21.15.67:0/2428280098 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3fa40035f0 con 0x7f3fb0103b60 2024-09-21T06:34:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.124+0000 7f3fb7a35640 1 -- 172.21.15.67:0/2428280098 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fb01a0610 con 0x7f3fb0103b60 2024-09-21T06:34:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.124+0000 7f3fa27fc640 1 -- 172.21.15.67:0/2428280098 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4035070 con 0x7f3fb0103b60 2024-09-21T06:34:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.124+0000 7f3fb7a35640 1 -- 172.21.15.67:0/2428280098 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fb01a6dd0 con 0x7f3fb0103b60 2024-09-21T06:34:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.125+0000 7f3fb7a35640 1 -- 172.21.15.67:0/2428280098 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fb0103f70 con 0x7f3fb0103b60 2024-09-21T06:34:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.128+0000 7f3fa27fc640 1 -- 172.21.15.67:0/2428280098 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f3fa4003aa0 con 0x7f3fb0103b60 2024-09-21T06:34:42.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.128+0000 7f3fa27fc640 1 --2- 172.21.15.67:0/2428280098 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f3f9c0b6590 0x7f3f9c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:42.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.129+0000 7f3fb4fd0640 1 --2- 172.21.15.67:0/2428280098 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f3f9c0b6590 0x7f3f9c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:42.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.129+0000 7f3fa27fc640 1 -- 172.21.15.67:0/2428280098 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8770+0+0 (secure 0 0 0) 0x7f3fa41084b0 con 0x7f3fb0103b60 2024-09-21T06:34:42.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.129+0000 7f3fb4fd0640 1 --2- 172.21.15.67:0/2428280098 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f3f9c0b6590 0x7f3f9c0b8aa0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0106530 tx=0x7f3fac005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:42.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.132+0000 7f3fa27fc640 1 -- 172.21.15.67:0/2428280098 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3fa40ceb20 con 0x7f3fb0103b60 2024-09-21T06:34:42.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.364+0000 7f3fb7a35640 1 -- 172.21.15.67:0/2428280098 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f3fb019b540 con 0x7f3fb0103b60 2024-09-21T06:34:42.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.707+0000 7f3fa27fc640 1 -- 172.21.15.67:0/2428280098 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f3fb019b540 con 0x7f3fb0103b60 2024-09-21T06:34:42.709 INFO:teuthology.orchestra.run.smithi067.stderr:added data pool 4 to fsmap 2024-09-21T06:34:42.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.709+0000 7f3fb7a35640 1 -- 172.21.15.67:0/2428280098 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f3f9c0b6590 msgr2=0x7f3f9c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:42.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.709+0000 7f3fb7a35640 1 --2- 172.21.15.67:0/2428280098 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f3f9c0b6590 0x7f3f9c0b8aa0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0106530 tx=0x7f3fac005ec0 comp rx=0 tx=0).stop 2024-09-21T06:34:42.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.709+0000 7f3fb7a35640 1 -- 172.21.15.67:0/2428280098 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fb0103b60 msgr2=0x7f3fb019a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:42.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.709+0000 7f3fb7a35640 1 --2- 172.21.15.67:0/2428280098 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fb0103b60 0x7f3fb019a410 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4009a20 tx=0x7f3fa4004270 comp rx=0 tx=0).stop 2024-09-21T06:34:42.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.709+0000 7f3fb7a35640 1 -- 172.21.15.67:0/2428280098 shutdown_connections 2024-09-21T06:34:42.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.709+0000 7f3fb7a35640 1 --2- 172.21.15.67:0/2428280098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fb01094a0 0x7f3fb019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:42.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.709+0000 7f3fb7a35640 1 --2- 172.21.15.67:0/2428280098 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f3f9c0b6590 0x7f3f9c0b8aa0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:42.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.709+0000 7f3fb7a35640 1 --2- 172.21.15.67:0/2428280098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fb0109df0 0x7f3fb019fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:42.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.709+0000 7f3fb7a35640 1 --2- 172.21.15.67:0/2428280098 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fb0103b60 0x7f3fb019a410 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:42.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.709+0000 7f3fb7a35640 1 -- 172.21.15.67:0/2428280098 >> 172.21.15.67:0/2428280098 conn(0x7f3fb007b850 msgr2=0x7f3fb01075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:42.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.709+0000 7f3fb7a35640 1 -- 172.21.15.67:0/2428280098 shutdown_connections 2024-09-21T06:34:42.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:42.709+0000 7f3fb7a35640 1 -- 172.21.15.67:0/2428280098 wait complete. 2024-09-21T06:34:42.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:42 smithi203 ceph-mon[33735]: pgmap v299: 129 pgs: 8 unknown, 121 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-21T06:34:42.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:42 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2428280098' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T06:34:43.032 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:42 smithi067 ceph-mon[31232]: pgmap v299: 129 pgs: 8 unknown, 121 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-21T06:34:43.032 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:42 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2428280098' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T06:34:43.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:42 smithi100 ceph-mon[33401]: pgmap v299: 129 pgs: 8 unknown, 121 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-21T06:34:43.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:42 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2428280098' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T06:34:43.369 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:34:43.632 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:34:43.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:43 smithi203 ceph-mon[33735]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T06:34:43.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:43 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2428280098' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-21T06:34:43.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:43 smithi203 ceph-mon[33735]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:34:44.041 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:43 smithi067 ceph-mon[31232]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T06:34:44.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:43 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2428280098' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-21T06:34:44.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:43 smithi067 ceph-mon[31232]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:34:44.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:43 smithi100 ceph-mon[33401]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T06:34:44.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:43 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2428280098' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-21T06:34:44.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:43 smithi100 ceph-mon[33401]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:34:44.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.226+0000 7f652ce4e640 1 -- 172.21.15.67:0/1269859815 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6528106350 msgr2=0x7f6528106760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:44.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.226+0000 7f652ce4e640 1 --2- 172.21.15.67:0/1269859815 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6528106350 0x7f6528106760 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6518009900 tx=0x7f651802a3f0 comp rx=0 tx=0).stop 2024-09-21T06:34:44.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.226+0000 7f652ce4e640 1 -- 172.21.15.67:0/1269859815 shutdown_connections 2024-09-21T06:34:44.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.226+0000 7f652ce4e640 1 --2- 172.21.15.67:0/1269859815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6528106ca0 0x7f6528110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:44.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.226+0000 7f652ce4e640 1 --2- 172.21.15.67:0/1269859815 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6528106350 0x7f6528106760 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:44.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.226+0000 7f652ce4e640 1 --2- 172.21.15.67:0/1269859815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6528102100 0x7f6528102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:44.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.226+0000 7f652ce4e640 1 -- 172.21.15.67:0/1269859815 >> 172.21.15.67:0/1269859815 conn(0x7f6528079ec0 msgr2=0x7f652807a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:44.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.226+0000 7f652ce4e640 1 -- 172.21.15.67:0/1269859815 shutdown_connections 2024-09-21T06:34:44.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.226+0000 7f652ce4e640 1 -- 172.21.15.67:0/1269859815 wait complete. 2024-09-21T06:34:44.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.227+0000 7f652ce4e640 1 Processor -- start 2024-09-21T06:34:44.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.227+0000 7f652ce4e640 1 -- start start 2024-09-21T06:34:44.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.227+0000 7f652ce4e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6528102100 0x7f652819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:44.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.227+0000 7f652ce4e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6528106350 0x7f652819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:44.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.227+0000 7f652ce4e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6528106ca0 0x7f652819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:44.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.228+0000 7f652ce4e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6528113a40 con 0x7f6528106350 2024-09-21T06:34:44.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.228+0000 7f652ce4e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f65281138c0 con 0x7f6528106ca0 2024-09-21T06:34:44.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.228+0000 7f652ce4e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6528113bc0 con 0x7f6528102100 2024-09-21T06:34:44.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.228+0000 7f6525d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6528106350 0x7f652819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:44.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.228+0000 7f6525d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6528106350 0x7f652819aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49072/0 (socket says 172.21.15.67:49072) 2024-09-21T06:34:44.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.228+0000 7f6525d9b640 1 -- 172.21.15.67:0/1932724478 learned_addr learned my addr 172.21.15.67:0/1932724478 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:34:44.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.228+0000 7f652659c640 1 --2- 172.21.15.67:0/1932724478 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6528102100 0x7f652819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:44.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.228+0000 7f6526d9d640 1 --2- 172.21.15.67:0/1932724478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6528106ca0 0x7f652819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:44.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.228+0000 7f6525d9b640 1 -- 172.21.15.67:0/1932724478 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6528102100 msgr2=0x7f652819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:44.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.228+0000 7f6525d9b640 1 --2- 172.21.15.67:0/1932724478 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6528102100 0x7f652819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:44.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.228+0000 7f6525d9b640 1 -- 172.21.15.67:0/1932724478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6528106ca0 msgr2=0x7f652819fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:44.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.228+0000 7f6525d9b640 1 --2- 172.21.15.67:0/1932724478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6528106ca0 0x7f652819fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:44.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.228+0000 7f6525d9b640 1 -- 172.21.15.67:0/1932724478 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65281a0320 con 0x7f6528106350 2024-09-21T06:34:44.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.228+0000 7f652659c640 1 --2- 172.21.15.67:0/1932724478 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6528102100 0x7f652819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:44.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.228+0000 7f6525d9b640 1 --2- 172.21.15.67:0/1932724478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6528106350 0x7f652819aa10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6518005940 tx=0x7f65180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:44.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.228+0000 7f651f7fe640 1 -- 172.21.15.67:0/1932724478 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651803f030 con 0x7f6528106350 2024-09-21T06:34:44.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.228+0000 7f651f7fe640 1 -- 172.21.15.67:0/1932724478 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6518003c90 con 0x7f6528106350 2024-09-21T06:34:44.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.228+0000 7f651f7fe640 1 -- 172.21.15.67:0/1932724478 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6518035070 con 0x7f6528106350 2024-09-21T06:34:44.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.228+0000 7f652ce4e640 1 -- 172.21.15.67:0/1932724478 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65281a05b0 con 0x7f6528106350 2024-09-21T06:34:44.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.229+0000 7f652ce4e640 1 -- 172.21.15.67:0/1932724478 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65281a0ae0 con 0x7f6528106350 2024-09-21T06:34:44.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.230+0000 7f652ce4e640 1 -- 172.21.15.67:0/1932724478 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6528102510 con 0x7f6528106350 2024-09-21T06:34:44.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.231+0000 7f651f7fe640 1 -- 172.21.15.67:0/1932724478 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f6518042030 con 0x7f6528106350 2024-09-21T06:34:44.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.232+0000 7f651f7fe640 1 --2- 172.21.15.67:0/1932724478 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f65140b65c0 0x7f65140b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:44.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.232+0000 7f652659c640 1 --2- 172.21.15.67:0/1932724478 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f65140b65c0 0x7f65140b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:44.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.232+0000 7f651f7fe640 1 -- 172.21.15.67:0/1932724478 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7f6518107bd0 con 0x7f6528106350 2024-09-21T06:34:44.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.232+0000 7f652659c640 1 --2- 172.21.15.67:0/1932724478 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f65140b65c0 0x7f65140b8ad0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f652806bb30 tx=0x7f651000a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:44.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.236+0000 7f651f7fe640 1 -- 172.21.15.67:0/1932724478 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f65281a0ae0 con 0x7f6528106350 2024-09-21T06:34:44.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.460+0000 7f652ce4e640 1 -- 172.21.15.67:0/1932724478 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f652802e640 con 0x7f6528106350 2024-09-21T06:34:44.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.461+0000 7f651f7fe640 1 -- 172.21.15.67:0/1932724478 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11851 (secure 0 0 0) 0x7f65180ce220 con 0x7f6528106350 2024-09-21T06:34:44.463 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:34:44.463 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":17,"btime":"2024-09-21T06:34:42:697039+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24641,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:34:42.366174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:34:44.463 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 17 2024-09-21T06:34:44.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.463+0000 7f652ce4e640 1 -- 172.21.15.67:0/1932724478 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f65140b65c0 msgr2=0x7f65140b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:44.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.463+0000 7f652ce4e640 1 --2- 172.21.15.67:0/1932724478 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f65140b65c0 0x7f65140b8ad0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f652806bb30 tx=0x7f651000a5e0 comp rx=0 tx=0).stop 2024-09-21T06:34:44.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.463+0000 7f652ce4e640 1 -- 172.21.15.67:0/1932724478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6528106350 msgr2=0x7f652819aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:44.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.463+0000 7f652ce4e640 1 --2- 172.21.15.67:0/1932724478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6528106350 0x7f652819aa10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6518005940 tx=0x7f65180042f0 comp rx=0 tx=0).stop 2024-09-21T06:34:44.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.463+0000 7f652ce4e640 1 -- 172.21.15.67:0/1932724478 shutdown_connections 2024-09-21T06:34:44.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.463+0000 7f652ce4e640 1 --2- 172.21.15.67:0/1932724478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6528106ca0 0x7f652819fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:44.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.463+0000 7f652ce4e640 1 --2- 172.21.15.67:0/1932724478 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f65140b65c0 0x7f65140b8ad0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:44.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.463+0000 7f652ce4e640 1 --2- 172.21.15.67:0/1932724478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6528106350 0x7f652819aa10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:44.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.463+0000 7f652ce4e640 1 --2- 172.21.15.67:0/1932724478 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6528102100 0x7f652819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:44.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.463+0000 7f652ce4e640 1 -- 172.21.15.67:0/1932724478 >> 172.21.15.67:0/1932724478 conn(0x7f6528079ec0 msgr2=0x7f65281085f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:44.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.463+0000 7f652ce4e640 1 -- 172.21.15.67:0/1932724478 shutdown_connections 2024-09-21T06:34:44.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:44.463+0000 7f652ce4e640 1 -- 172.21.15.67:0/1932724478 wait complete. 2024-09-21T06:34:44.783 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:44 smithi067 ceph-mon[31232]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:44.783 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:44 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1932724478' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:34:44.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:44 smithi203 ceph-mon[33735]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:44.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:44 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1932724478' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:34:45.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:44 smithi100 ceph-mon[33401]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:45.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:44 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1932724478' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:34:45.112 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:34:45.370 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:34:45.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.869+0000 7f1f0c315640 1 -- 172.21.15.67:0/1430470735 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f04109e10 msgr2=0x7f1f04110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:45.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.869+0000 7f1f0c315640 1 --2- 172.21.15.67:0/1430470735 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f04109e10 0x7f1f04110640 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009ca0 tx=0x7f1efc02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:34:45.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.869+0000 7f1f0c315640 1 -- 172.21.15.67:0/1430470735 shutdown_connections 2024-09-21T06:34:45.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.869+0000 7f1f0c315640 1 --2- 172.21.15.67:0/1430470735 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f04109e10 0x7f1f04110640 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:45.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.869+0000 7f1f0c315640 1 --2- 172.21.15.67:0/1430470735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f041094c0 0x7f1f041098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:45.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.869+0000 7f1f0c315640 1 --2- 172.21.15.67:0/1430470735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f04103b80 0x7f1f04103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:45.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.869+0000 7f1f0c315640 1 -- 172.21.15.67:0/1430470735 >> 172.21.15.67:0/1430470735 conn(0x7f1f040ff5d0 msgr2=0x7f1f04101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:45.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.869+0000 7f1f0c315640 1 -- 172.21.15.67:0/1430470735 shutdown_connections 2024-09-21T06:34:45.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.869+0000 7f1f0c315640 1 -- 172.21.15.67:0/1430470735 wait complete. 2024-09-21T06:34:45.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.870+0000 7f1f0c315640 1 Processor -- start 2024-09-21T06:34:45.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.870+0000 7f1f0c315640 1 -- start start 2024-09-21T06:34:45.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.870+0000 7f1f0c315640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f04103b80 0x7f1f0419a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:45.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.870+0000 7f1f0c315640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f041094c0 0x7f1f0419a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:45.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.870+0000 7f1f0c315640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f04109e10 0x7f1f0419fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:45.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.870+0000 7f1f0c315640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1f041139d0 con 0x7f1f04103b80 2024-09-21T06:34:45.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.870+0000 7f1f0c315640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1f04113850 con 0x7f1f04109e10 2024-09-21T06:34:45.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.870+0000 7f1f0c315640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1f04113b50 con 0x7f1f041094c0 2024-09-21T06:34:45.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.870+0000 7f1f0a0b1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f04103b80 0x7f1f0419a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:45.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.870+0000 7f1f0a0b1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f04103b80 0x7f1f0419a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49086/0 (socket says 172.21.15.67:49086) 2024-09-21T06:34:45.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.870+0000 7f1f0a0b1640 1 -- 172.21.15.67:0/3477572168 learned_addr learned my addr 172.21.15.67:0/3477572168 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:34:45.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.870+0000 7f1f098b0640 1 --2- 172.21.15.67:0/3477572168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f041094c0 0x7f1f0419a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:45.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.870+0000 7f1f0a8b2640 1 --2- 172.21.15.67:0/3477572168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f04109e10 0x7f1f0419fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:45.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.870+0000 7f1f0a0b1640 1 -- 172.21.15.67:0/3477572168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f041094c0 msgr2=0x7f1f0419a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:45.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.870+0000 7f1f0a0b1640 1 --2- 172.21.15.67:0/3477572168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f041094c0 0x7f1f0419a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:45.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.870+0000 7f1f0a0b1640 1 -- 172.21.15.67:0/3477572168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f04109e10 msgr2=0x7f1f0419fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:45.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.870+0000 7f1f0a0b1640 1 --2- 172.21.15.67:0/3477572168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f04109e10 0x7f1f0419fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:45.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.870+0000 7f1f0a0b1640 1 -- 172.21.15.67:0/3477572168 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f041a02d0 con 0x7f1f04103b80 2024-09-21T06:34:45.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.871+0000 7f1f0a0b1640 1 --2- 172.21.15.67:0/3477572168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f04103b80 0x7f1f0419a360 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000d950 tx=0x7f1f0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:45.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.871+0000 7f1efb7fe640 1 -- 172.21.15.67:0/3477572168 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f00017070 con 0x7f1f04103b80 2024-09-21T06:34:45.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.871+0000 7f1efb7fe640 1 -- 172.21.15.67:0/3477572168 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f00009c30 con 0x7f1f04103b80 2024-09-21T06:34:45.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.871+0000 7f1f0c315640 1 -- 172.21.15.67:0/3477572168 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f041a05c0 con 0x7f1f04103b80 2024-09-21T06:34:45.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.871+0000 7f1efb7fe640 1 -- 172.21.15.67:0/3477572168 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0000f620 con 0x7f1f04103b80 2024-09-21T06:34:45.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.871+0000 7f1f0c315640 1 -- 172.21.15.67:0/3477572168 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f041a6d20 con 0x7f1f04103b80 2024-09-21T06:34:45.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.872+0000 7f1f0c315640 1 -- 172.21.15.67:0/3477572168 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f04103f90 con 0x7f1f04103b80 2024-09-21T06:34:45.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.874+0000 7f1efb7fe640 1 -- 172.21.15.67:0/3477572168 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f1f00005030 con 0x7f1f04103b80 2024-09-21T06:34:45.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.874+0000 7f1efb7fe640 1 --2- 172.21.15.67:0/3477572168 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f1ef00b6510 0x7f1ef00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:45.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.874+0000 7f1f098b0640 1 --2- 172.21.15.67:0/3477572168 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f1ef00b6510 0x7f1ef00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:45.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.875+0000 7f1efb7fe640 1 -- 172.21.15.67:0/3477572168 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7f1f000e7040 con 0x7f1f04103b80 2024-09-21T06:34:45.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.875+0000 7f1f098b0640 1 --2- 172.21.15.67:0/3477572168 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f1ef00b6510 0x7f1ef00b8a20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1f04108e50 tx=0x7f1ef400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:45.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:45.877+0000 7f1efb7fe640 1 -- 172.21.15.67:0/3477572168 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1f00009de0 con 0x7f1f04103b80 2024-09-21T06:34:46.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:46.069+0000 7f1f0c315640 1 -- 172.21.15.67:0/3477572168 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f0402e640 con 0x7f1f04103b80 2024-09-21T06:34:46.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:46.071+0000 7f1efb7fe640 1 -- 172.21.15.67:0/3477572168 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+24238 (secure 0 0 0) 0x7f1f000b1a70 con 0x7f1f04103b80 2024-09-21T06:34:46.073 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:34:46.073 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":81,"fsid":"2a791768-77e2-11ef-baf5-efdc52797490","created":"2024-09-21T06:26:15.222283+0000","modified":"2024-09-21T06:34:42.366026+0000","last_up_change":"2024-09-21T06:33:40.886310+0000","last_in_change":"2024-09-21T06:33:17.339486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:29:50.212825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:34:29.082881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:34:31.301094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:34:37.681356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"18778364-82ed-41c0-92da-ce44cdad7358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6803","nonce":2310759946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6805","nonce":2310759946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6809","nonce":2310759946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6807","nonce":2310759946}]},"public_addr":"172.21.15.67:6803/2310759946","cluster_addr":"172.21.15.67:6805/2310759946","heartbeat_back_addr":"172.21.15.67:6809/2310759946","heartbeat_front_addr":"172.21.15.67:6807/2310759946","state":["exists","up"]},{"osd":1,"uuid":"84e8d0c2-1c45-4673-ad49-f61df667d7f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6801","nonce":161807364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6803","nonce":161807364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6807","nonce":161807364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6805","nonce":161807364}]},"public_addr":"172.21.15.100:6801/161807364","cluster_addr":"172.21.15.100:6803/161807364","heartbeat_back_addr":"172.21.15.100:6807/161807364","heartbeat_front_addr":"172.21.15.100:6805/161807364","state":["exists","up"]},{"osd":2,"uuid":"e9a4d919-a779-4a1b-9b5b-d03a65f28ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6801","nonce":2754787169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6803","nonce":2754787169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6807","nonce":2754787169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6805","nonce":2754787169}]},"public_addr":"172.21.15.203:6801/2754787169","cluster_addr":"172.21.15.203:6803/2754787169","heartbeat_back_addr":"172.21.15.203:6807/2754787169","heartbeat_front_addr":"172.21.15.203:6805/2754787169","state":["exists","up"]},{"osd":3,"uuid":"3f7d9abd-587c-47ad-bf43-11f7dcdf71c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6811","nonce":4007517951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6813","nonce":4007517951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6817","nonce":4007517951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6815","nonce":4007517951}]},"public_addr":"172.21.15.67:6811/4007517951","cluster_addr":"172.21.15.67:6813/4007517951","heartbeat_back_addr":"172.21.15.67:6817/4007517951","heartbeat_front_addr":"172.21.15.67:6815/4007517951","state":["exists","up"]},{"osd":4,"uuid":"94f3a075-6d8a-4907-8418-a8513b35779a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6809","nonce":3773646501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6811","nonce":3773646501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6815","nonce":3773646501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6813","nonce":3773646501}]},"public_addr":"172.21.15.100:6809/3773646501","cluster_addr":"172.21.15.100:6811/3773646501","heartbeat_back_addr":"172.21.15.100:6815/3773646501","heartbeat_front_addr":"172.21.15.100:6813/3773646501","state":["exists","up"]},{"osd":5,"uuid":"5d842edf-06ad-49ae-ac71-863e5657270c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6809","nonce":2096171166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6811","nonce":2096171166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6815","nonce":2096171166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6813","nonce":2096171166}]},"public_addr":"172.21.15.203:6809/2096171166","cluster_addr":"172.21.15.203:6811/2096171166","heartbeat_back_addr":"172.21.15.203:6815/2096171166","heartbeat_front_addr":"172.21.15.203:6813/2096171166","state":["exists","up"]},{"osd":6,"uuid":"85b9beed-c93d-48ba-bc7a-c4cd816fbdff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6819","nonce":3229936280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6821","nonce":3229936280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6825","nonce":3229936280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6823","nonce":3229936280}]},"public_addr":"172.21.15.67:6819/3229936280","cluster_addr":"172.21.15.67:6821/3229936280","heartbeat_back_addr":"172.21.15.67:6825/3229936280","heartbeat_front_addr":"172.21.15.67:6823/3229936280","state":["exists","up"]},{"osd":7,"uuid":"86de30c2-b8ee-4c9d-9c17-8a56517934ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6817","nonce":1622917909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6819","nonce":1622917909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6823","nonce":1622917909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6821","nonce":1622917909}]},"public_addr":"172.21.15.100:6817/1622917909","cluster_addr":"172.21.15.100:6819/1622917909","heartbeat_back_addr":"172.21.15.100:6823/1622917909","heartbeat_front_addr":"172.21.15.100:6821/1622917909","state":["exists","up"]},{"osd":8,"uuid":"0e96cdad-a135-4115-9e2d-9a28306700e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6817","nonce":3452445943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6819","nonce":3452445943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6823","nonce":3452445943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6821","nonce":3452445943}]},"public_addr":"172.21.15.203:6817/3452445943","cluster_addr":"172.21.15.203:6819/3452445943","heartbeat_back_addr":"172.21.15.203:6823/3452445943","heartbeat_front_addr":"172.21.15.203:6821/3452445943","state":["exists","up"]},{"osd":9,"uuid":"95704ed6-6286-4094-9c6d-23919c7005ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6827","nonce":2998307533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6829","nonce":2998307533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6833","nonce":2998307533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6831","nonce":2998307533}]},"public_addr":"172.21.15.67:6827/2998307533","cluster_addr":"172.21.15.67:6829/2998307533","heartbeat_back_addr":"172.21.15.67:6833/2998307533","heartbeat_front_addr":"172.21.15.67:6831/2998307533","state":["exists","up"]},{"osd":10,"uuid":"bd2a7d33-b935-46c8-95c3-d76f28242ffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6825","nonce":2819057142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6827","nonce":2819057142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6831","nonce":2819057142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6829","nonce":2819057142}]},"public_addr":"172.21.15.100:6825/2819057142","cluster_addr":"172.21.15.100:6827/2819057142","heartbeat_back_addr":"172.21.15.100:6831/2819057142","heartbeat_front_addr":"172.21.15.100:6829/2819057142","state":["exists","up"]},{"osd":11,"uuid":"ea591507-10e2-4c08-8d29-d38ba9be96ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6825","nonce":566502600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6827","nonce":566502600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6831","nonce":566502600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6829","nonce":566502600}]},"public_addr":"172.21.15.203:6825/566502600","cluster_addr":"172.21.15.203:6827/566502600","heartbeat_back_addr":"172.21.15.203:6831/566502600","heartbeat_front_addr":"172.21.15.203:6829/566502600","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:28:58.945733+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:29:23.597118+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:29:48.303563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:30:14.139139+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:30:39.021508+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:04.981241+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:30.853841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:56.452967+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:32:22.239036+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:32:47.998803+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:33:13.850759+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:33:39.282150+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6800/1181476774":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/3100321138":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/2309364298":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/3458717076":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/4110206171":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6801/1181476774":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/2910038982":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6801/2622729103":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6800/2622729103":"2024-09-22T06:26:37.041377+0000","172.21.15.67:0/3346784528":"2024-09-22T06:26:37.041377+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:34:46.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:46.073+0000 7f1f0c315640 1 -- 172.21.15.67:0/3477572168 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f1ef00b6510 msgr2=0x7f1ef00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:46.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:46.073+0000 7f1f0c315640 1 --2- 172.21.15.67:0/3477572168 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f1ef00b6510 0x7f1ef00b8a20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1f04108e50 tx=0x7f1ef400b390 comp rx=0 tx=0).stop 2024-09-21T06:34:46.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:46.073+0000 7f1f0c315640 1 -- 172.21.15.67:0/3477572168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f04103b80 msgr2=0x7f1f0419a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:46.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:46.073+0000 7f1f0c315640 1 --2- 172.21.15.67:0/3477572168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f04103b80 0x7f1f0419a360 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000d950 tx=0x7f1f0000de10 comp rx=0 tx=0).stop 2024-09-21T06:34:46.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:46.073+0000 7f1f0c315640 1 -- 172.21.15.67:0/3477572168 shutdown_connections 2024-09-21T06:34:46.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:46.073+0000 7f1f0c315640 1 --2- 172.21.15.67:0/3477572168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f04109e10 0x7f1f0419fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:46.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:46.073+0000 7f1f0c315640 1 --2- 172.21.15.67:0/3477572168 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f1ef00b6510 0x7f1ef00b8a20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:46.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:46.073+0000 7f1f0c315640 1 --2- 172.21.15.67:0/3477572168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f041094c0 0x7f1f0419a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:46.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:46.073+0000 7f1f0c315640 1 --2- 172.21.15.67:0/3477572168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f04103b80 0x7f1f0419a360 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:46.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:46.073+0000 7f1f0c315640 1 -- 172.21.15.67:0/3477572168 >> 172.21.15.67:0/3477572168 conn(0x7f1f040ff5d0 msgr2=0x7f1f041019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:46.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:46.073+0000 7f1f0c315640 1 -- 172.21.15.67:0/3477572168 shutdown_connections 2024-09-21T06:34:46.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:46.073+0000 7f1f0c315640 1 -- 172.21.15.67:0/3477572168 wait complete. 2024-09-21T06:34:46.914 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:34:47.179 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:34:47.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:46 smithi203 ceph-mon[33735]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:47.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:46 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3477572168' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:34:47.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:46 smithi067 ceph-mon[31232]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:47.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:46 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3477572168' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:34:47.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:46 smithi100 ceph-mon[33401]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:47.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:46 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3477572168' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:34:47.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.687+0000 7f4839475640 1 -- 172.21.15.67:0/4113166738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4834108cf0 msgr2=0x7f483410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:47.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.687+0000 7f4839475640 1 --2- 172.21.15.67:0/4113166738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4834108cf0 0x7f483410f520 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f482c009900 tx=0x7f482c02a410 comp rx=0 tx=0).stop 2024-09-21T06:34:47.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.687+0000 7f4839475640 1 -- 172.21.15.67:0/4113166738 shutdown_connections 2024-09-21T06:34:47.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.687+0000 7f4839475640 1 --2- 172.21.15.67:0/4113166738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4834108cf0 0x7f483410f520 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:47.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.687+0000 7f4839475640 1 --2- 172.21.15.67:0/4113166738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48341083a0 0x7f48341087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:47.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.687+0000 7f4839475640 1 --2- 172.21.15.67:0/4113166738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4834102a60 0x7f4834102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:47.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.687+0000 7f4839475640 1 -- 172.21.15.67:0/4113166738 >> 172.21.15.67:0/4113166738 conn(0x7f4834079730 msgr2=0x7f4834079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:47.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.687+0000 7f4839475640 1 -- 172.21.15.67:0/4113166738 shutdown_connections 2024-09-21T06:34:47.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.687+0000 7f4839475640 1 -- 172.21.15.67:0/4113166738 wait complete. 2024-09-21T06:34:47.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.688+0000 7f4839475640 1 Processor -- start 2024-09-21T06:34:47.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.688+0000 7f4839475640 1 -- start start 2024-09-21T06:34:47.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.689+0000 7f4839475640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4834102a60 0x7f483419a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:47.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.689+0000 7f4839475640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48341083a0 0x7f483419a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:47.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.689+0000 7f4839475640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4834108cf0 0x7f483419fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:47.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.689+0000 7f4839475640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4834113960 con 0x7f4834108cf0 2024-09-21T06:34:47.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.689+0000 7f4839475640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f48341137e0 con 0x7f48341083a0 2024-09-21T06:34:47.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.689+0000 7f4839475640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4834113ae0 con 0x7f4834102a60 2024-09-21T06:34:47.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.689+0000 7f48337fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4834108cf0 0x7f483419fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:47.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.689+0000 7f48327fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48341083a0 0x7f483419a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:47.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.689+0000 7f48327fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48341083a0 0x7f483419a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.67:45534/0 (socket says 172.21.15.67:45534) 2024-09-21T06:34:47.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.689+0000 7f48327fc640 1 -- 172.21.15.67:0/2761583515 learned_addr learned my addr 172.21.15.67:0/2761583515 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:34:47.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.689+0000 7f48337fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4834108cf0 0x7f483419fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42438/0 (socket says 172.21.15.67:42438) 2024-09-21T06:34:47.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.689+0000 7f4832ffd640 1 --2- 172.21.15.67:0/2761583515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4834102a60 0x7f483419a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:47.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.689+0000 7f48327fc640 1 -- 172.21.15.67:0/2761583515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4834102a60 msgr2=0x7f483419a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:47.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.689+0000 7f48327fc640 1 --2- 172.21.15.67:0/2761583515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4834102a60 0x7f483419a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:47.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.689+0000 7f48327fc640 1 -- 172.21.15.67:0/2761583515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4834108cf0 msgr2=0x7f483419fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:47.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.689+0000 7f48327fc640 1 --2- 172.21.15.67:0/2761583515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4834108cf0 0x7f483419fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:47.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.689+0000 7f48327fc640 1 -- 172.21.15.67:0/2761583515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48341a0300 con 0x7f48341083a0 2024-09-21T06:34:47.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.689+0000 7f48327fc640 1 --2- 172.21.15.67:0/2761583515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48341083a0 0x7f483419a930 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f482400e9f0 tx=0x7f482400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:47.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.690+0000 7f482bfff640 1 -- 172.21.15.67:0/2761583515 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482400ccc0 con 0x7f48341083a0 2024-09-21T06:34:47.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.690+0000 7f482bfff640 1 -- 172.21.15.67:0/2761583515 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48240054b0 con 0x7f48341083a0 2024-09-21T06:34:47.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.690+0000 7f4839475640 1 -- 172.21.15.67:0/2761583515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48341a05f0 con 0x7f48341083a0 2024-09-21T06:34:47.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.690+0000 7f482bfff640 1 -- 172.21.15.67:0/2761583515 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824010430 con 0x7f48341083a0 2024-09-21T06:34:47.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.690+0000 7f4839475640 1 -- 172.21.15.67:0/2761583515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48341a6d50 con 0x7f48341083a0 2024-09-21T06:34:47.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.691+0000 7f4839475640 1 -- 172.21.15.67:0/2761583515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4834102e70 con 0x7f48341083a0 2024-09-21T06:34:47.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.692+0000 7f482bfff640 1 -- 172.21.15.67:0/2761583515 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f48240046c0 con 0x7f48341083a0 2024-09-21T06:34:47.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.693+0000 7f482bfff640 1 --2- 172.21.15.67:0/2761583515 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f48200b6590 0x7f48200b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:47.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.693+0000 7f4832ffd640 1 --2- 172.21.15.67:0/2761583515 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f48200b6590 0x7f48200b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:47.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.693+0000 7f482bfff640 1 -- 172.21.15.67:0/2761583515 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7f48240e7ab0 con 0x7f48341083a0 2024-09-21T06:34:47.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.694+0000 7f4832ffd640 1 --2- 172.21.15.67:0/2761583515 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f48200b6590 0x7f48200b8aa0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f481c005fd0 tx=0x7f481c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:47.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.696+0000 7f482bfff640 1 -- 172.21.15.67:0/2761583515 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f48240ae100 con 0x7f48341083a0 2024-09-21T06:34:47.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.890+0000 7f4839475640 1 -- 172.21.15.67:0/2761583515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f483419b4c0 con 0x7f48341083a0 2024-09-21T06:34:47.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.891+0000 7f482bfff640 1 -- 172.21.15.67:0/2761583515 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+24238 (secure 0 0 0) 0x7f483419b4c0 con 0x7f48341083a0 2024-09-21T06:34:47.894 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:34:47.894 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":81,"fsid":"2a791768-77e2-11ef-baf5-efdc52797490","created":"2024-09-21T06:26:15.222283+0000","modified":"2024-09-21T06:34:42.366026+0000","last_up_change":"2024-09-21T06:33:40.886310+0000","last_in_change":"2024-09-21T06:33:17.339486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:29:50.212825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:34:29.082881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:34:31.301094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:34:37.681356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"18778364-82ed-41c0-92da-ce44cdad7358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6803","nonce":2310759946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6805","nonce":2310759946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6809","nonce":2310759946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6807","nonce":2310759946}]},"public_addr":"172.21.15.67:6803/2310759946","cluster_addr":"172.21.15.67:6805/2310759946","heartbeat_back_addr":"172.21.15.67:6809/2310759946","heartbeat_front_addr":"172.21.15.67:6807/2310759946","state":["exists","up"]},{"osd":1,"uuid":"84e8d0c2-1c45-4673-ad49-f61df667d7f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6801","nonce":161807364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6803","nonce":161807364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6807","nonce":161807364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6805","nonce":161807364}]},"public_addr":"172.21.15.100:6801/161807364","cluster_addr":"172.21.15.100:6803/161807364","heartbeat_back_addr":"172.21.15.100:6807/161807364","heartbeat_front_addr":"172.21.15.100:6805/161807364","state":["exists","up"]},{"osd":2,"uuid":"e9a4d919-a779-4a1b-9b5b-d03a65f28ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6801","nonce":2754787169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6803","nonce":2754787169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6807","nonce":2754787169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6805","nonce":2754787169}]},"public_addr":"172.21.15.203:6801/2754787169","cluster_addr":"172.21.15.203:6803/2754787169","heartbeat_back_addr":"172.21.15.203:6807/2754787169","heartbeat_front_addr":"172.21.15.203:6805/2754787169","state":["exists","up"]},{"osd":3,"uuid":"3f7d9abd-587c-47ad-bf43-11f7dcdf71c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6811","nonce":4007517951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6813","nonce":4007517951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6817","nonce":4007517951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6815","nonce":4007517951}]},"public_addr":"172.21.15.67:6811/4007517951","cluster_addr":"172.21.15.67:6813/4007517951","heartbeat_back_addr":"172.21.15.67:6817/4007517951","heartbeat_front_addr":"172.21.15.67:6815/4007517951","state":["exists","up"]},{"osd":4,"uuid":"94f3a075-6d8a-4907-8418-a8513b35779a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6809","nonce":3773646501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6811","nonce":3773646501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6815","nonce":3773646501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6813","nonce":3773646501}]},"public_addr":"172.21.15.100:6809/3773646501","cluster_addr":"172.21.15.100:6811/3773646501","heartbeat_back_addr":"172.21.15.100:6815/3773646501","heartbeat_front_addr":"172.21.15.100:6813/3773646501","state":["exists","up"]},{"osd":5,"uuid":"5d842edf-06ad-49ae-ac71-863e5657270c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6809","nonce":2096171166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6811","nonce":2096171166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6815","nonce":2096171166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6813","nonce":2096171166}]},"public_addr":"172.21.15.203:6809/2096171166","cluster_addr":"172.21.15.203:6811/2096171166","heartbeat_back_addr":"172.21.15.203:6815/2096171166","heartbeat_front_addr":"172.21.15.203:6813/2096171166","state":["exists","up"]},{"osd":6,"uuid":"85b9beed-c93d-48ba-bc7a-c4cd816fbdff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6819","nonce":3229936280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6821","nonce":3229936280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6825","nonce":3229936280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6823","nonce":3229936280}]},"public_addr":"172.21.15.67:6819/3229936280","cluster_addr":"172.21.15.67:6821/3229936280","heartbeat_back_addr":"172.21.15.67:6825/3229936280","heartbeat_front_addr":"172.21.15.67:6823/3229936280","state":["exists","up"]},{"osd":7,"uuid":"86de30c2-b8ee-4c9d-9c17-8a56517934ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6817","nonce":1622917909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6819","nonce":1622917909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6823","nonce":1622917909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6821","nonce":1622917909}]},"public_addr":"172.21.15.100:6817/1622917909","cluster_addr":"172.21.15.100:6819/1622917909","heartbeat_back_addr":"172.21.15.100:6823/1622917909","heartbeat_front_addr":"172.21.15.100:6821/1622917909","state":["exists","up"]},{"osd":8,"uuid":"0e96cdad-a135-4115-9e2d-9a28306700e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6817","nonce":3452445943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6819","nonce":3452445943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6823","nonce":3452445943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6821","nonce":3452445943}]},"public_addr":"172.21.15.203:6817/3452445943","cluster_addr":"172.21.15.203:6819/3452445943","heartbeat_back_addr":"172.21.15.203:6823/3452445943","heartbeat_front_addr":"172.21.15.203:6821/3452445943","state":["exists","up"]},{"osd":9,"uuid":"95704ed6-6286-4094-9c6d-23919c7005ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6827","nonce":2998307533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6829","nonce":2998307533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6833","nonce":2998307533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6831","nonce":2998307533}]},"public_addr":"172.21.15.67:6827/2998307533","cluster_addr":"172.21.15.67:6829/2998307533","heartbeat_back_addr":"172.21.15.67:6833/2998307533","heartbeat_front_addr":"172.21.15.67:6831/2998307533","state":["exists","up"]},{"osd":10,"uuid":"bd2a7d33-b935-46c8-95c3-d76f28242ffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6825","nonce":2819057142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6827","nonce":2819057142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6831","nonce":2819057142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6829","nonce":2819057142}]},"public_addr":"172.21.15.100:6825/2819057142","cluster_addr":"172.21.15.100:6827/2819057142","heartbeat_back_addr":"172.21.15.100:6831/2819057142","heartbeat_front_addr":"172.21.15.100:6829/2819057142","state":["exists","up"]},{"osd":11,"uuid":"ea591507-10e2-4c08-8d29-d38ba9be96ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6825","nonce":566502600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6827","nonce":566502600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6831","nonce":566502600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6829","nonce":566502600}]},"public_addr":"172.21.15.203:6825/566502600","cluster_addr":"172.21.15.203:6827/566502600","heartbeat_back_addr":"172.21.15.203:6831/566502600","heartbeat_front_addr":"172.21.15.203:6829/566502600","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:28:58.945733+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:29:23.597118+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:29:48.303563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:30:14.139139+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:30:39.021508+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:04.981241+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:30.853841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:56.452967+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:32:22.239036+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:32:47.998803+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:33:13.850759+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:33:39.282150+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6800/1181476774":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/3100321138":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/2309364298":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/3458717076":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/4110206171":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6801/1181476774":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/2910038982":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6801/2622729103":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6800/2622729103":"2024-09-22T06:26:37.041377+0000","172.21.15.67:0/3346784528":"2024-09-22T06:26:37.041377+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:34:47.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.894+0000 7f4839475640 1 -- 172.21.15.67:0/2761583515 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f48200b6590 msgr2=0x7f48200b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:47.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.894+0000 7f4839475640 1 --2- 172.21.15.67:0/2761583515 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f48200b6590 0x7f48200b8aa0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f481c005fd0 tx=0x7f481c009500 comp rx=0 tx=0).stop 2024-09-21T06:34:47.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.895+0000 7f4839475640 1 -- 172.21.15.67:0/2761583515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48341083a0 msgr2=0x7f483419a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:47.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.895+0000 7f4839475640 1 --2- 172.21.15.67:0/2761583515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48341083a0 0x7f483419a930 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f482400e9f0 tx=0x7f482400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:34:47.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.895+0000 7f4839475640 1 -- 172.21.15.67:0/2761583515 shutdown_connections 2024-09-21T06:34:47.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.895+0000 7f4839475640 1 --2- 172.21.15.67:0/2761583515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48341083a0 0x7f483419a930 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:47.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.895+0000 7f4839475640 1 --2- 172.21.15.67:0/2761583515 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f48200b6590 0x7f48200b8aa0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:47.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.895+0000 7f4839475640 1 --2- 172.21.15.67:0/2761583515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4834108cf0 0x7f483419fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:47.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.895+0000 7f4839475640 1 --2- 172.21.15.67:0/2761583515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4834102a60 0x7f483419a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:47.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.895+0000 7f4839475640 1 -- 172.21.15.67:0/2761583515 >> 172.21.15.67:0/2761583515 conn(0x7f4834079730 msgr2=0x7f48341043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:47.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.895+0000 7f4839475640 1 -- 172.21.15.67:0/2761583515 shutdown_connections 2024-09-21T06:34:47.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:47.895+0000 7f4839475640 1 -- 172.21.15.67:0/2761583515 wait complete. 2024-09-21T06:34:48.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:47 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2761583515' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:34:48.267 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:47 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2761583515' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:34:48.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:47 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2761583515' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:34:48.780 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:34:49.042 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:34:49.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:48 smithi067 ceph-mon[31232]: pgmap v303: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:49.234 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:48 smithi203 ceph-mon[33735]: pgmap v303: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:49.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:48 smithi100 ceph-mon[33401]: pgmap v303: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:49.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.531+0000 7fe41c642640 1 -- 172.21.15.67:0/896141215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe414108d20 msgr2=0x7fe41410f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:49.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.531+0000 7fe41c642640 1 --2- 172.21.15.67:0/896141215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe414108d20 0x7fe41410f550 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe40c009900 tx=0x7fe40c02a410 comp rx=0 tx=0).stop 2024-09-21T06:34:49.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.531+0000 7fe41c642640 1 -- 172.21.15.67:0/896141215 shutdown_connections 2024-09-21T06:34:49.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.531+0000 7fe41c642640 1 --2- 172.21.15.67:0/896141215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe414108d20 0x7fe41410f550 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:49.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.531+0000 7fe41c642640 1 --2- 172.21.15.67:0/896141215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4141083d0 0x7fe4141087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:49.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.531+0000 7fe41c642640 1 --2- 172.21.15.67:0/896141215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe414102ae0 0x7fe414102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:49.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.531+0000 7fe41c642640 1 -- 172.21.15.67:0/896141215 >> 172.21.15.67:0/896141215 conn(0x7fe414079760 msgr2=0x7fe414079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:49.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.531+0000 7fe41c642640 1 -- 172.21.15.67:0/896141215 shutdown_connections 2024-09-21T06:34:49.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.531+0000 7fe41c642640 1 -- 172.21.15.67:0/896141215 wait complete. 2024-09-21T06:34:49.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.532+0000 7fe41c642640 1 Processor -- start 2024-09-21T06:34:49.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.532+0000 7fe41c642640 1 -- start start 2024-09-21T06:34:49.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.532+0000 7fe41c642640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe414102ae0 0x7fe41419b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:49.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.532+0000 7fe41c642640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4141083d0 0x7fe41419b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:49.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.532+0000 7fe41c642640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe414108d20 0x7fe41419bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:49.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.532+0000 7fe41c642640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe414112840 con 0x7fe4141083d0 2024-09-21T06:34:49.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.532+0000 7fe41c642640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe4141126c0 con 0x7fe414108d20 2024-09-21T06:34:49.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.532+0000 7fe41c642640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe4141129c0 con 0x7fe414102ae0 2024-09-21T06:34:49.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.532+0000 7fe419bdd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4141083d0 0x7fe41419b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:49.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.532+0000 7fe419bdd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4141083d0 0x7fe41419b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42468/0 (socket says 172.21.15.67:42468) 2024-09-21T06:34:49.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.532+0000 7fe419bdd640 1 -- 172.21.15.67:0/4254407568 learned_addr learned my addr 172.21.15.67:0/4254407568 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:34:49.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.532+0000 7fe41a3de640 1 --2- 172.21.15.67:0/4254407568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe414102ae0 0x7fe41419b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:49.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.532+0000 7fe41abdf640 1 --2- 172.21.15.67:0/4254407568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe414108d20 0x7fe41419bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:49.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.532+0000 7fe419bdd640 1 -- 172.21.15.67:0/4254407568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe414102ae0 msgr2=0x7fe41419b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:49.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.532+0000 7fe419bdd640 1 --2- 172.21.15.67:0/4254407568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe414102ae0 0x7fe41419b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:49.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.532+0000 7fe419bdd640 1 -- 172.21.15.67:0/4254407568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe414108d20 msgr2=0x7fe41419bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:49.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.532+0000 7fe419bdd640 1 --2- 172.21.15.67:0/4254407568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe414108d20 0x7fe41419bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:49.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.532+0000 7fe419bdd640 1 -- 172.21.15.67:0/4254407568 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4141a63f0 con 0x7fe4141083d0 2024-09-21T06:34:49.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.533+0000 7fe419bdd640 1 --2- 172.21.15.67:0/4254407568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4141083d0 0x7fe41419b670 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe40800d770 tx=0x7fe40800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:49.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.533+0000 7fe4077fe640 1 -- 172.21.15.67:0/4254407568 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe408017070 con 0x7fe4141083d0 2024-09-21T06:34:49.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.533+0000 7fe41c642640 1 -- 172.21.15.67:0/4254407568 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4141a66e0 con 0x7fe4141083d0 2024-09-21T06:34:49.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.533+0000 7fe4077fe640 1 -- 172.21.15.67:0/4254407568 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe408009c30 con 0x7fe4141083d0 2024-09-21T06:34:49.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.533+0000 7fe4077fe640 1 -- 172.21.15.67:0/4254407568 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40800f620 con 0x7fe4141083d0 2024-09-21T06:34:49.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.533+0000 7fe41c642640 1 -- 172.21.15.67:0/4254407568 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4141a6c70 con 0x7fe4141083d0 2024-09-21T06:34:49.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.534+0000 7fe41c642640 1 -- 172.21.15.67:0/4254407568 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe414103d10 con 0x7fe4141083d0 2024-09-21T06:34:49.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.536+0000 7fe4077fe640 1 -- 172.21.15.67:0/4254407568 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fe40801a070 con 0x7fe4141083d0 2024-09-21T06:34:49.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.536+0000 7fe4077fe640 1 --2- 172.21.15.67:0/4254407568 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe4000b6590 0x7fe4000b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:49.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.537+0000 7fe41a3de640 1 --2- 172.21.15.67:0/4254407568 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe4000b6590 0x7fe4000b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:49.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.537+0000 7fe4077fe640 1 -- 172.21.15.67:0/4254407568 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7fe4080e6f70 con 0x7fe4141083d0 2024-09-21T06:34:49.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.537+0000 7fe41a3de640 1 --2- 172.21.15.67:0/4254407568 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe4000b6590 0x7fe4000b8aa0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe41406c6c0 tx=0x7fe410009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:49.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.540+0000 7fe4077fe640 1 -- 172.21.15.67:0/4254407568 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe408009de0 con 0x7fe4141083d0 2024-09-21T06:34:49.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.765+0000 7fe41c642640 1 -- 172.21.15.67:0/4254407568 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe41406bf70 con 0x7fe4141083d0 2024-09-21T06:34:49.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.766+0000 7fe4077fe640 1 -- 172.21.15.67:0/4254407568 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11851 (secure 0 0 0) 0x7fe4080ad640 con 0x7fe4141083d0 2024-09-21T06:34:49.768 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:34:49.768 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":17,"btime":"2024-09-21T06:34:42:697039+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24641,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:34:42.366174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:34:49.768 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 17 2024-09-21T06:34:49.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.768+0000 7fe41c642640 1 -- 172.21.15.67:0/4254407568 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe4000b6590 msgr2=0x7fe4000b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:49.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.768+0000 7fe41c642640 1 --2- 172.21.15.67:0/4254407568 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe4000b6590 0x7fe4000b8aa0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe41406c6c0 tx=0x7fe410009680 comp rx=0 tx=0).stop 2024-09-21T06:34:49.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.768+0000 7fe41c642640 1 -- 172.21.15.67:0/4254407568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4141083d0 msgr2=0x7fe41419b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:49.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.768+0000 7fe41c642640 1 --2- 172.21.15.67:0/4254407568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4141083d0 0x7fe41419b670 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe40800d770 tx=0x7fe40800dc30 comp rx=0 tx=0).stop 2024-09-21T06:34:49.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.768+0000 7fe41c642640 1 -- 172.21.15.67:0/4254407568 shutdown_connections 2024-09-21T06:34:49.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.768+0000 7fe41c642640 1 --2- 172.21.15.67:0/4254407568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe414108d20 0x7fe41419bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:49.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.768+0000 7fe41c642640 1 --2- 172.21.15.67:0/4254407568 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe4000b6590 0x7fe4000b8aa0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:49.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.768+0000 7fe41c642640 1 --2- 172.21.15.67:0/4254407568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4141083d0 0x7fe41419b670 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:49.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.768+0000 7fe41c642640 1 --2- 172.21.15.67:0/4254407568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe414102ae0 0x7fe41419b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:49.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.768+0000 7fe41c642640 1 -- 172.21.15.67:0/4254407568 >> 172.21.15.67:0/4254407568 conn(0x7fe414079760 msgr2=0x7fe414104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:49.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.768+0000 7fe41c642640 1 -- 172.21.15.67:0/4254407568 shutdown_connections 2024-09-21T06:34:49.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:49.769+0000 7fe41c642640 1 -- 172.21.15.67:0/4254407568 wait complete. 2024-09-21T06:34:50.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:49 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/4254407568' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:34:50.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:49 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/4254407568' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:34:50.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:49 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/4254407568' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:34:50.518 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:34:50.780 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:34:51.033 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:50 smithi067 ceph-mon[31232]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:51.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:50 smithi203 ceph-mon[33735]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:51.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.283+0000 7fc7ed69d640 1 -- 172.21.15.67:0/468995584 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7e81021f0 msgr2=0x7fc7e8102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:51.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.283+0000 7fc7ed69d640 1 --2- 172.21.15.67:0/468995584 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7e81021f0 0x7fc7e8102600 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0009900 tx=0x7fc7e002a410 comp rx=0 tx=0).stop 2024-09-21T06:34:51.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.283+0000 7fc7ed69d640 1 -- 172.21.15.67:0/468995584 shutdown_connections 2024-09-21T06:34:51.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.283+0000 7fc7ed69d640 1 --2- 172.21.15.67:0/468995584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7e8108480 0x7fc7e810ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:51.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.283+0000 7fc7ed69d640 1 --2- 172.21.15.67:0/468995584 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7e8107b30 0x7fc7e8107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:51.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.283+0000 7fc7ed69d640 1 --2- 172.21.15.67:0/468995584 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7e81021f0 0x7fc7e8102600 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:51.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.283+0000 7fc7ed69d640 1 -- 172.21.15.67:0/468995584 >> 172.21.15.67:0/468995584 conn(0x7fc7e8077c60 msgr2=0x7fc7e80780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:51.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.283+0000 7fc7ed69d640 1 -- 172.21.15.67:0/468995584 shutdown_connections 2024-09-21T06:34:51.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.283+0000 7fc7ed69d640 1 -- 172.21.15.67:0/468995584 wait complete. 2024-09-21T06:34:51.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.283+0000 7fc7ed69d640 1 Processor -- start 2024-09-21T06:34:51.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.284+0000 7fc7ed69d640 1 -- start start 2024-09-21T06:34:51.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.284+0000 7fc7ed69d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7e81021f0 0x7fc7e80fede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:51.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.284+0000 7fc7ed69d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7e8107b30 0x7fc7e80ff320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:51.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.284+0000 7fc7ed69d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7e8108480 0x7fc7e80ff860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:51.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.284+0000 7fc7ed69d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e81128c0 con 0x7fc7e81021f0 2024-09-21T06:34:51.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.284+0000 7fc7ed69d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e8112740 con 0x7fc7e8108480 2024-09-21T06:34:51.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.284+0000 7fc7ed69d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e8112a40 con 0x7fc7e8107b30 2024-09-21T06:34:51.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.284+0000 7fc7e6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7e81021f0 0x7fc7e80fede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:51.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.284+0000 7fc7e6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7e81021f0 0x7fc7e80fede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42492/0 (socket says 172.21.15.67:42492) 2024-09-21T06:34:51.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.284+0000 7fc7e6ffd640 1 -- 172.21.15.67:0/1766811767 learned_addr learned my addr 172.21.15.67:0/1766811767 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:34:51.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.284+0000 7fc7e67fc640 1 --2- 172.21.15.67:0/1766811767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7e8107b30 0x7fc7e80ff320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:51.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.284+0000 7fc7e77fe640 1 --2- 172.21.15.67:0/1766811767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7e8108480 0x7fc7e80ff860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:51.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.284+0000 7fc7e6ffd640 1 -- 172.21.15.67:0/1766811767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7e8107b30 msgr2=0x7fc7e80ff320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:51.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.285+0000 7fc7e6ffd640 1 --2- 172.21.15.67:0/1766811767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7e8107b30 0x7fc7e80ff320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:51.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.285+0000 7fc7e6ffd640 1 -- 172.21.15.67:0/1766811767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7e8108480 msgr2=0x7fc7e80ff860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:51.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.285+0000 7fc7e6ffd640 1 --2- 172.21.15.67:0/1766811767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7e8108480 0x7fc7e80ff860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:51.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.285+0000 7fc7e6ffd640 1 -- 172.21.15.67:0/1766811767 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e81a86f0 con 0x7fc7e81021f0 2024-09-21T06:34:51.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.285+0000 7fc7e67fc640 1 --2- 172.21.15.67:0/1766811767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7e8107b30 0x7fc7e80ff320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:51.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.285+0000 7fc7e6ffd640 1 --2- 172.21.15.67:0/1766811767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7e81021f0 0x7fc7e80fede0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0009a20 tx=0x7fc7e0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:51.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.285+0000 7fc7dffff640 1 -- 172.21.15.67:0/1766811767 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e003f030 con 0x7fc7e81021f0 2024-09-21T06:34:51.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.285+0000 7fc7ed69d640 1 -- 172.21.15.67:0/1766811767 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e81a8980 con 0x7fc7e81021f0 2024-09-21T06:34:51.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.285+0000 7fc7ed69d640 1 -- 172.21.15.67:0/1766811767 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e81a8eb0 con 0x7fc7e81021f0 2024-09-21T06:34:51.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.285+0000 7fc7dffff640 1 -- 172.21.15.67:0/1766811767 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc7e00035f0 con 0x7fc7e81021f0 2024-09-21T06:34:51.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.285+0000 7fc7dffff640 1 -- 172.21.15.67:0/1766811767 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0035070 con 0x7fc7e81021f0 2024-09-21T06:34:51.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.286+0000 7fc7ed69d640 1 -- 172.21.15.67:0/1766811767 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7c8005290 con 0x7fc7e81021f0 2024-09-21T06:34:51.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.288+0000 7fc7dffff640 1 -- 172.21.15.67:0/1766811767 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fc7e0031c80 con 0x7fc7e81021f0 2024-09-21T06:34:51.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.289+0000 7fc7dffff640 1 --2- 172.21.15.67:0/1766811767 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc7d40b6510 0x7fc7d40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:51.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.289+0000 7fc7e67fc640 1 --2- 172.21.15.67:0/1766811767 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc7d40b6510 0x7fc7d40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:51.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.289+0000 7fc7dffff640 1 -- 172.21.15.67:0/1766811767 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7fc7e01081a0 con 0x7fc7e81021f0 2024-09-21T06:34:51.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.289+0000 7fc7e67fc640 1 --2- 172.21.15.67:0/1766811767 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc7d40b6510 0x7fc7d40b8a20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8104c30 tx=0x7fc7d0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:51.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.292+0000 7fc7dffff640 1 -- 172.21.15.67:0/1766811767 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc7e00ce7f0 con 0x7fc7e81021f0 2024-09-21T06:34:51.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:50 smithi100 ceph-mon[33401]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:51.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.480+0000 7fc7ed69d640 1 -- 172.21.15.67:0/1766811767 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc7c8005850 con 0x7fc7e81021f0 2024-09-21T06:34:51.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.481+0000 7fc7dffff640 1 -- 172.21.15.67:0/1766811767 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+24238 (secure 0 0 0) 0x7fc7e00d2bd0 con 0x7fc7e81021f0 2024-09-21T06:34:51.483 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:34:51.483 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":81,"fsid":"2a791768-77e2-11ef-baf5-efdc52797490","created":"2024-09-21T06:26:15.222283+0000","modified":"2024-09-21T06:34:42.366026+0000","last_up_change":"2024-09-21T06:33:40.886310+0000","last_in_change":"2024-09-21T06:33:17.339486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:29:50.212825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:34:29.082881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:34:31.301094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:34:37.681356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"18778364-82ed-41c0-92da-ce44cdad7358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6803","nonce":2310759946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6805","nonce":2310759946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6809","nonce":2310759946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6807","nonce":2310759946}]},"public_addr":"172.21.15.67:6803/2310759946","cluster_addr":"172.21.15.67:6805/2310759946","heartbeat_back_addr":"172.21.15.67:6809/2310759946","heartbeat_front_addr":"172.21.15.67:6807/2310759946","state":["exists","up"]},{"osd":1,"uuid":"84e8d0c2-1c45-4673-ad49-f61df667d7f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6801","nonce":161807364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6803","nonce":161807364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6807","nonce":161807364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6805","nonce":161807364}]},"public_addr":"172.21.15.100:6801/161807364","cluster_addr":"172.21.15.100:6803/161807364","heartbeat_back_addr":"172.21.15.100:6807/161807364","heartbeat_front_addr":"172.21.15.100:6805/161807364","state":["exists","up"]},{"osd":2,"uuid":"e9a4d919-a779-4a1b-9b5b-d03a65f28ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6801","nonce":2754787169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6803","nonce":2754787169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6807","nonce":2754787169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6805","nonce":2754787169}]},"public_addr":"172.21.15.203:6801/2754787169","cluster_addr":"172.21.15.203:6803/2754787169","heartbeat_back_addr":"172.21.15.203:6807/2754787169","heartbeat_front_addr":"172.21.15.203:6805/2754787169","state":["exists","up"]},{"osd":3,"uuid":"3f7d9abd-587c-47ad-bf43-11f7dcdf71c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6811","nonce":4007517951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6813","nonce":4007517951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6817","nonce":4007517951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6815","nonce":4007517951}]},"public_addr":"172.21.15.67:6811/4007517951","cluster_addr":"172.21.15.67:6813/4007517951","heartbeat_back_addr":"172.21.15.67:6817/4007517951","heartbeat_front_addr":"172.21.15.67:6815/4007517951","state":["exists","up"]},{"osd":4,"uuid":"94f3a075-6d8a-4907-8418-a8513b35779a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6809","nonce":3773646501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6811","nonce":3773646501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6815","nonce":3773646501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6813","nonce":3773646501}]},"public_addr":"172.21.15.100:6809/3773646501","cluster_addr":"172.21.15.100:6811/3773646501","heartbeat_back_addr":"172.21.15.100:6815/3773646501","heartbeat_front_addr":"172.21.15.100:6813/3773646501","state":["exists","up"]},{"osd":5,"uuid":"5d842edf-06ad-49ae-ac71-863e5657270c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6809","nonce":2096171166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6811","nonce":2096171166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6815","nonce":2096171166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6813","nonce":2096171166}]},"public_addr":"172.21.15.203:6809/2096171166","cluster_addr":"172.21.15.203:6811/2096171166","heartbeat_back_addr":"172.21.15.203:6815/2096171166","heartbeat_front_addr":"172.21.15.203:6813/2096171166","state":["exists","up"]},{"osd":6,"uuid":"85b9beed-c93d-48ba-bc7a-c4cd816fbdff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6819","nonce":3229936280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6821","nonce":3229936280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6825","nonce":3229936280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6823","nonce":3229936280}]},"public_addr":"172.21.15.67:6819/3229936280","cluster_addr":"172.21.15.67:6821/3229936280","heartbeat_back_addr":"172.21.15.67:6825/3229936280","heartbeat_front_addr":"172.21.15.67:6823/3229936280","state":["exists","up"]},{"osd":7,"uuid":"86de30c2-b8ee-4c9d-9c17-8a56517934ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6817","nonce":1622917909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6819","nonce":1622917909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6823","nonce":1622917909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6821","nonce":1622917909}]},"public_addr":"172.21.15.100:6817/1622917909","cluster_addr":"172.21.15.100:6819/1622917909","heartbeat_back_addr":"172.21.15.100:6823/1622917909","heartbeat_front_addr":"172.21.15.100:6821/1622917909","state":["exists","up"]},{"osd":8,"uuid":"0e96cdad-a135-4115-9e2d-9a28306700e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6817","nonce":3452445943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6819","nonce":3452445943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6823","nonce":3452445943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6821","nonce":3452445943}]},"public_addr":"172.21.15.203:6817/3452445943","cluster_addr":"172.21.15.203:6819/3452445943","heartbeat_back_addr":"172.21.15.203:6823/3452445943","heartbeat_front_addr":"172.21.15.203:6821/3452445943","state":["exists","up"]},{"osd":9,"uuid":"95704ed6-6286-4094-9c6d-23919c7005ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6827","nonce":2998307533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6829","nonce":2998307533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6833","nonce":2998307533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6831","nonce":2998307533}]},"public_addr":"172.21.15.67:6827/2998307533","cluster_addr":"172.21.15.67:6829/2998307533","heartbeat_back_addr":"172.21.15.67:6833/2998307533","heartbeat_front_addr":"172.21.15.67:6831/2998307533","state":["exists","up"]},{"osd":10,"uuid":"bd2a7d33-b935-46c8-95c3-d76f28242ffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6825","nonce":2819057142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6827","nonce":2819057142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6831","nonce":2819057142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6829","nonce":2819057142}]},"public_addr":"172.21.15.100:6825/2819057142","cluster_addr":"172.21.15.100:6827/2819057142","heartbeat_back_addr":"172.21.15.100:6831/2819057142","heartbeat_front_addr":"172.21.15.100:6829/2819057142","state":["exists","up"]},{"osd":11,"uuid":"ea591507-10e2-4c08-8d29-d38ba9be96ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6825","nonce":566502600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6827","nonce":566502600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6831","nonce":566502600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6829","nonce":566502600}]},"public_addr":"172.21.15.203:6825/566502600","cluster_addr":"172.21.15.203:6827/566502600","heartbeat_back_addr":"172.21.15.203:6831/566502600","heartbeat_front_addr":"172.21.15.203:6829/566502600","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:28:58.945733+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:29:23.597118+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:29:48.303563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:30:14.139139+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:30:39.021508+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:04.981241+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:30.853841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:56.452967+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:32:22.239036+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:32:47.998803+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:33:13.850759+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:33:39.282150+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6800/1181476774":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/3100321138":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/2309364298":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/3458717076":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/4110206171":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6801/1181476774":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/2910038982":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6801/2622729103":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6800/2622729103":"2024-09-22T06:26:37.041377+0000","172.21.15.67:0/3346784528":"2024-09-22T06:26:37.041377+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:34:51.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.484+0000 7fc7ed69d640 1 -- 172.21.15.67:0/1766811767 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc7d40b6510 msgr2=0x7fc7d40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:51.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.484+0000 7fc7ed69d640 1 --2- 172.21.15.67:0/1766811767 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc7d40b6510 0x7fc7d40b8a20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8104c30 tx=0x7fc7d0008040 comp rx=0 tx=0).stop 2024-09-21T06:34:51.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.484+0000 7fc7ed69d640 1 -- 172.21.15.67:0/1766811767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7e81021f0 msgr2=0x7fc7e80fede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:51.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.484+0000 7fc7ed69d640 1 --2- 172.21.15.67:0/1766811767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7e81021f0 0x7fc7e80fede0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0009a20 tx=0x7fc7e0004270 comp rx=0 tx=0).stop 2024-09-21T06:34:51.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.484+0000 7fc7ed69d640 1 -- 172.21.15.67:0/1766811767 shutdown_connections 2024-09-21T06:34:51.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.484+0000 7fc7ed69d640 1 --2- 172.21.15.67:0/1766811767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7e8108480 0x7fc7e80ff860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:51.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.484+0000 7fc7ed69d640 1 --2- 172.21.15.67:0/1766811767 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc7d40b6510 0x7fc7d40b8a20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:51.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.484+0000 7fc7ed69d640 1 --2- 172.21.15.67:0/1766811767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7e8107b30 0x7fc7e80ff320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:51.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.484+0000 7fc7ed69d640 1 --2- 172.21.15.67:0/1766811767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7e81021f0 0x7fc7e80fede0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:51.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.484+0000 7fc7ed69d640 1 -- 172.21.15.67:0/1766811767 >> 172.21.15.67:0/1766811767 conn(0x7fc7e8077c60 msgr2=0x7fc7e8101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:51.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.484+0000 7fc7ed69d640 1 -- 172.21.15.67:0/1766811767 shutdown_connections 2024-09-21T06:34:51.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:51.484+0000 7fc7ed69d640 1 -- 172.21.15.67:0/1766811767 wait complete. 2024-09-21T06:34:52.148 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:51 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1766811767' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:34:52.179 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T06:34:33.306959+0000', 'modified': '2024-09-21T06:34:42.366174+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24671}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24671': {'gid': 24671, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.100:6839/778060936', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6838', 'nonce': 778060936}, {'type': 'v1', 'addr': '172.21.15.100:6839', 'nonce': 778060936}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24671, 'qdb_cluster': [24671]} 2024-09-21T06:34:52.179 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T06:34:52.180 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T06:34:52.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:51 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1766811767' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:34:52.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:51 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1766811767' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:34:52.438 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:34:52.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.935+0000 7faa15021640 1 -- 172.21.15.67:0/1733583924 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa10102a60 msgr2=0x7faa10102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:52.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.935+0000 7faa15021640 1 --2- 172.21.15.67:0/1733583924 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa10102a60 0x7faa10102e70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7faa00009900 tx=0x7faa0002a410 comp rx=0 tx=0).stop 2024-09-21T06:34:52.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.935+0000 7faa15021640 1 -- 172.21.15.67:0/1733583924 shutdown_connections 2024-09-21T06:34:52.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.935+0000 7faa15021640 1 --2- 172.21.15.67:0/1733583924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa10108cf0 0x7faa1010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.935+0000 7faa15021640 1 --2- 172.21.15.67:0/1733583924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa101083a0 0x7faa101087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.935+0000 7faa15021640 1 --2- 172.21.15.67:0/1733583924 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa10102a60 0x7faa10102e70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.935+0000 7faa15021640 1 -- 172.21.15.67:0/1733583924 >> 172.21.15.67:0/1733583924 conn(0x7faa10079730 msgr2=0x7faa10079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:52.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.935+0000 7faa15021640 1 -- 172.21.15.67:0/1733583924 shutdown_connections 2024-09-21T06:34:52.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.935+0000 7faa15021640 1 -- 172.21.15.67:0/1733583924 wait complete. 2024-09-21T06:34:52.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.936+0000 7faa15021640 1 Processor -- start 2024-09-21T06:34:52.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.936+0000 7faa15021640 1 -- start start 2024-09-21T06:34:52.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.937+0000 7faa15021640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa10102a60 0x7faa1019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:52.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.937+0000 7faa15021640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa101083a0 0x7faa1019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:52.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.937+0000 7faa15021640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa10108cf0 0x7faa1019fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:52.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.937+0000 7faa15021640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7faa10112860 con 0x7faa10108cf0 2024-09-21T06:34:52.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.937+0000 7faa15021640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faa101126e0 con 0x7faa10102a60 2024-09-21T06:34:52.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.937+0000 7faa15021640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7faa101129e0 con 0x7faa101083a0 2024-09-21T06:34:52.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.937+0000 7faa0f59e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa10108cf0 0x7faa1019fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:52.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.937+0000 7faa0f59e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa10108cf0 0x7faa1019fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42510/0 (socket says 172.21.15.67:42510) 2024-09-21T06:34:52.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.937+0000 7faa0f59e640 1 -- 172.21.15.67:0/240668854 learned_addr learned my addr 172.21.15.67:0/240668854 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:34:52.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.937+0000 7faa0ed9d640 1 --2- 172.21.15.67:0/240668854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa10102a60 0x7faa1019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:52.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.937+0000 7faa0659c640 1 --2- 172.21.15.67:0/240668854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa101083a0 0x7faa1019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:52.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.937+0000 7faa0f59e640 1 -- 172.21.15.67:0/240668854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa101083a0 msgr2=0x7faa1019a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:52.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.937+0000 7faa0f59e640 1 --2- 172.21.15.67:0/240668854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa101083a0 0x7faa1019a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.937+0000 7faa0f59e640 1 -- 172.21.15.67:0/240668854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa10102a60 msgr2=0x7faa1019a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:52.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.937+0000 7faa0f59e640 1 --2- 172.21.15.67:0/240668854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa10102a60 0x7faa1019a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.937+0000 7faa0f59e640 1 -- 172.21.15.67:0/240668854 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa101a0310 con 0x7faa10108cf0 2024-09-21T06:34:52.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.937+0000 7faa0f59e640 1 --2- 172.21.15.67:0/240668854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa10108cf0 0x7faa1019fbe0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7faa0800eaa0 tx=0x7faa0800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:52.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.938+0000 7faa0cd99640 1 -- 172.21.15.67:0/240668854 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0800cdb0 con 0x7faa10108cf0 2024-09-21T06:34:52.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.938+0000 7faa0cd99640 1 -- 172.21.15.67:0/240668854 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa08005020 con 0x7faa10108cf0 2024-09-21T06:34:52.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.938+0000 7faa15021640 1 -- 172.21.15.67:0/240668854 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa101a0600 con 0x7faa10108cf0 2024-09-21T06:34:52.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.938+0000 7faa0cd99640 1 -- 172.21.15.67:0/240668854 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa080116d0 con 0x7faa10108cf0 2024-09-21T06:34:52.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.938+0000 7faa15021640 1 -- 172.21.15.67:0/240668854 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa10104360 con 0x7faa10108cf0 2024-09-21T06:34:52.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.940+0000 7faa1b595740 1 Processor -- start 2024-09-21T06:34:52.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.940+0000 7faa0cd99640 1 -- 172.21.15.67:0/240668854 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7faa080051d0 con 0x7faa10108cf0 2024-09-21T06:34:52.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.940+0000 7faa1b595740 1 -- start start 2024-09-21T06:34:52.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.941+0000 7faa1b595740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea44b8ff60 0x55ea44f35f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:52.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.941+0000 7faa1b595740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea44b670a0 0x55ea44f36440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:52.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.941+0000 7faa1b595740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea44b6f280 0x55ea44f36980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:52.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.941+0000 7faa1b595740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ea44eadc10 con 0x55ea44b8ff60 2024-09-21T06:34:52.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.941+0000 7faa1b595740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ea44e9d040 con 0x55ea44b6f280 2024-09-21T06:34:52.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.941+0000 7faa1b595740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ea44ea0e50 con 0x55ea44b670a0 2024-09-21T06:34:52.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.941+0000 7faa0659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea44b8ff60 0x55ea44f35f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:52.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.941+0000 7faa0659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea44b8ff60 0x55ea44f35f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42526/0 (socket says 172.21.15.67:42526) 2024-09-21T06:34:52.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.941+0000 7faa0659c640 1 -- 172.21.15.67:0/4131904373 learned_addr learned my addr 172.21.15.67:0/4131904373 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:34:52.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.941+0000 7faa0659c640 1 --2- 172.21.15.67:0/4131904373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea44b6f280 0x55ea44f36980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:52.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.941+0000 7faa0ed9d640 1 --2- 172.21.15.67:0/4131904373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea44b670a0 0x55ea44f36440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:52.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.941+0000 7faa0659c640 1 -- 172.21.15.67:0/4131904373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea44b670a0 msgr2=0x55ea44f36440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:52.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.941+0000 7faa0659c640 1 --2- 172.21.15.67:0/4131904373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea44b670a0 0x55ea44f36440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.941+0000 7faa0659c640 1 -- 172.21.15.67:0/4131904373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea44b6f280 msgr2=0x55ea44f36980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:52.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.941+0000 7faa0cd99640 1 --2- 172.21.15.67:0/240668854 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fa9fc0b6510 0x7fa9fc0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:52.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.941+0000 7faa0659c640 1 --2- 172.21.15.67:0/4131904373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea44b6f280 0x55ea44f36980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.941+0000 7faa0659c640 1 -- 172.21.15.67:0/4131904373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea44bfb540 con 0x55ea44b8ff60 2024-09-21T06:34:52.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.941+0000 7faa0f59e640 1 --2- 172.21.15.67:0/240668854 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fa9fc0b6510 0x7fa9fc0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:52.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.942+0000 7faa0659c640 1 --2- 172.21.15.67:0/4131904373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea44b8ff60 0x55ea44f35f00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8012570 tx=0x7fa9f800b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c92ae2d5188009d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:52.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.942+0000 7faa0cd99640 1 -- 172.21.15.67:0/240668854 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7faa080e75e0 con 0x7faa10108cf0 2024-09-21T06:34:52.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.942+0000 7faa0559a640 1 -- 172.21.15.67:0/4131904373 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8004020 con 0x55ea44b8ff60 2024-09-21T06:34:52.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.942+0000 7faa0559a640 1 -- 172.21.15.67:0/4131904373 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9f8004c30 con 0x55ea44b8ff60 2024-09-21T06:34:52.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.942+0000 7faa0559a640 1 -- 172.21.15.67:0/4131904373 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8005280 con 0x55ea44b8ff60 2024-09-21T06:34:52.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.943+0000 7faa1b595740 1 -- 172.21.15.67:0/4131904373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea44b8ff60 msgr2=0x55ea44f35f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:52.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.943+0000 7faa1b595740 1 --2- 172.21.15.67:0/4131904373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea44b8ff60 0x55ea44f35f00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8012570 tx=0x7fa9f800b650 comp rx=0 tx=0).stop 2024-09-21T06:34:52.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.943+0000 7faa0f59e640 1 --2- 172.21.15.67:0/240668854 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fa9fc0b6510 0x7fa9fc0b8a20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7faa080aa850 tx=0x7faa080ddfd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:52.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.943+0000 7faa1b595740 1 -- 172.21.15.67:0/4131904373 shutdown_connections 2024-09-21T06:34:52.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.943+0000 7faa1b595740 1 --2- 172.21.15.67:0/4131904373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea44b6f280 0x55ea44f36980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.943+0000 7faa1b595740 1 --2- 172.21.15.67:0/4131904373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea44b670a0 0x55ea44f36440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.943+0000 7faa1b595740 1 --2- 172.21.15.67:0/4131904373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea44b8ff60 0x55ea44f35f00 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.943+0000 7faa1b595740 1 -- 172.21.15.67:0/4131904373 >> 172.21.15.67:0/4131904373 conn(0x55ea44b84350 msgr2=0x55ea44f33df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:52.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.943+0000 7faa1b595740 1 -- 172.21.15.67:0/4131904373 shutdown_connections 2024-09-21T06:34:52.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.943+0000 7faa1b595740 1 -- 172.21.15.67:0/4131904373 wait complete. 2024-09-21T06:34:52.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.943+0000 7faa1b595740 1 -- start start 2024-09-21T06:34:52.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.943+0000 7faa1b595740 1 Processor -- start 2024-09-21T06:34:52.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.944+0000 7faa1b595740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea44b670a0 0x55ea44f419c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:52.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.944+0000 7faa1b595740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea44b6f280 0x55ea44f41f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:52.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.944+0000 7faa1b595740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea44b8ff60 0x55ea44f4d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:52.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.944+0000 7faa1b595740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ea44cfe5f0 con 0x55ea44b8ff60 2024-09-21T06:34:52.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.944+0000 7faa1b595740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ea44e38010 con 0x55ea44b6f280 2024-09-21T06:34:52.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.944+0000 7faa1b595740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ea44f446e0 con 0x55ea44b670a0 2024-09-21T06:34:52.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.944+0000 7faa1b595740 10 client.0 _ll_register_callbacks cb 0x55ea44e97340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:34:52.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.944+0000 7faa0659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea44b8ff60 0x55ea44f4d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:52.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.944+0000 7faa0659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea44b8ff60 0x55ea44f4d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42528/0 (socket says 172.21.15.67:42528) 2024-09-21T06:34:52.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.944+0000 7faa0659c640 1 -- 172.21.15.67:0/1138585972 learned_addr learned my addr 172.21.15.67:0/1138585972 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:34:52.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.944+0000 7faa0659c640 1 --2- 172.21.15.67:0/1138585972 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea44b670a0 0x55ea44f419c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:52.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.944+0000 7faa0ed9d640 1 --2- 172.21.15.67:0/1138585972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea44b6f280 0x55ea44f41f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:52.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.944+0000 7faa0659c640 1 -- 172.21.15.67:0/1138585972 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea44b670a0 msgr2=0x55ea44f419c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:52.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.944+0000 7faa0659c640 1 --2- 172.21.15.67:0/1138585972 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea44b670a0 0x55ea44f419c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.944+0000 7faa0659c640 1 -- 172.21.15.67:0/1138585972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea44b6f280 msgr2=0x55ea44f41f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:52.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.944+0000 7faa0659c640 1 --2- 172.21.15.67:0/1138585972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea44b6f280 0x55ea44f41f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.944+0000 7faa0659c640 1 -- 172.21.15.67:0/1138585972 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea44e65a00 con 0x55ea44b8ff60 2024-09-21T06:34:52.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.944+0000 7faa0659c640 1 --2- 172.21.15.67:0/1138585972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea44b8ff60 0x55ea44f4d9d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55ea44f44010 tx=0x7fa9f80044a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350416a86ac77fe3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:52.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.944+0000 7fa9f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:34:52.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.945+0000 7fa9f67fc640 1 -- 172.21.15.67:0/1138585972 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8026070 con 0x55ea44b8ff60 2024-09-21T06:34:52.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.945+0000 7fa9f67fc640 1 -- 172.21.15.67:0/1138585972 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7fa9f80048c0 con 0x55ea44b8ff60 2024-09-21T06:34:52.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.945+0000 7faa1b595740 1 -- 172.21.15.67:0/1138585972 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9f8004c30 con 0x55ea44b8ff60 2024-09-21T06:34:52.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.945+0000 7fa9f67fc640 1 -- 172.21.15.67:0/1138585972 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fa9f80052f0 con 0x55ea44b8ff60 2024-09-21T06:34:52.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.945+0000 7faa1b595740 10 client.14910 fetch_fsmap learned FSMap version 17 2024-09-21T06:34:52.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.945+0000 7faa1b595740 1 -- 172.21.15.67:0/1138585972 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55ea44f50600 con 0x55ea44b8ff60 2024-09-21T06:34:52.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.945+0000 7fa9f67fc640 1 -- 172.21.15.67:0/1138585972 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9f8004c30 con 0x55ea44b8ff60 2024-09-21T06:34:52.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.945+0000 7fa9f67fc640 1 -- 172.21.15.67:0/1138585972 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7fa9f8010790 con 0x55ea44b8ff60 2024-09-21T06:34:52.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.945+0000 7faa1b595740 10 client.14910 fetch_fsmap finished waiting for FSMap version 17 2024-09-21T06:34:52.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.945+0000 7faa1b595740 10 client.14910 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:34:52.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.945+0000 7faa1b595740 20 client.14910 populate_metadata read hostname 'smithi067' 2024-09-21T06:34:52.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.945+0000 7faa1b595740 1 --2- 172.21.15.67:0/1138585972 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55ea44b4e8e0 0x55ea44f53e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:52.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.945+0000 7faa1b595740 4 client.14910 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:34:52.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.945+0000 7faa1b595740 1 -- 172.21.15.67:0/1138585972 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea44eafb60 con 0x55ea44b4e8e0 2024-09-21T06:34:52.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.946+0000 7faa0ed9d640 1 --2- 172.21.15.67:0/1138585972 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55ea44b4e8e0 0x55ea44f53e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:52.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.946+0000 7fa9f67fc640 10 client.14910 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:34:52.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.946+0000 7faa0ed9d640 1 --2- 172.21.15.67:0/1138585972 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55ea44b4e8e0 0x55ea44f53e00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c8a3e157dbd5ed4 server_cookie=ac5eca1354d12223 in_seq=0 out_seq=0 2024-09-21T06:34:52.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.947+0000 7fa9f67fc640 1 -- 172.21.15.67:0/1138585972 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ea44eafb60 con 0x55ea44b4e8e0 2024-09-21T06:34:52.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.947+0000 7fa9f67fc640 10 client.14910 handle_command_reply: tid=0 2024-09-21T06:34:52.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.947+0000 7faa1b595740 1 client.14910 shutdown 2024-09-21T06:34:52.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.947+0000 7faa1b595740 1 -- 172.21.15.67:0/1138585972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea44b8ff60 msgr2=0x55ea44f4d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:52.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.947+0000 7faa1b595740 1 --2- 172.21.15.67:0/1138585972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea44b8ff60 0x55ea44f4d9d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55ea44f44010 tx=0x7fa9f80044a0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.947+0000 7faa1b595740 1 -- 172.21.15.67:0/1138585972 shutdown_connections 2024-09-21T06:34:52.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.947+0000 7faa1b595740 1 --2- 172.21.15.67:0/1138585972 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55ea44b4e8e0 0x55ea44f53e00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.947+0000 7faa1b595740 1 --2- 172.21.15.67:0/1138585972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea44b6f280 0x55ea44f41f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.947+0000 7faa1b595740 1 --2- 172.21.15.67:0/1138585972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea44b8ff60 0x55ea44f4d9d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.947+0000 7fa9f67fc640 0 client.14910 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:34:52.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.947+0000 7faa1b595740 1 --2- 172.21.15.67:0/1138585972 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea44b670a0 0x55ea44f419c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.947+0000 7faa1b595740 1 -- 172.21.15.67:0/1138585972 >> 172.21.15.67:0/1138585972 conn(0x55ea44b84350 msgr2=0x55ea44f390e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:52.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.947+0000 7faa1b595740 1 -- 172.21.15.67:0/1138585972 shutdown_connections 2024-09-21T06:34:52.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.947+0000 7faa1b595740 1 -- 172.21.15.67:0/1138585972 wait complete. 2024-09-21T06:34:52.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.947+0000 7faa1b595740 20 client.14910 trim_cache size 0 max 16384 2024-09-21T06:34:52.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.965+0000 7faa1b595740 1 Processor -- start 2024-09-21T06:34:52.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.965+0000 7faa1b595740 1 -- start start 2024-09-21T06:34:52.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.966+0000 7faa1b595740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea44b670a0 0x55ea44f3c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:52.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.966+0000 7faa1b595740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea44b6f280 0x55ea44f3cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:52.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.966+0000 7faa1b595740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea44b8ff60 0x55ea44f4d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:52.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.966+0000 7faa1b595740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ea44f52140 con 0x55ea44b8ff60 2024-09-21T06:34:52.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.966+0000 7faa1b595740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ea44f52810 con 0x55ea44b670a0 2024-09-21T06:34:52.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.966+0000 7faa1b595740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ea44f52ee0 con 0x55ea44b6f280 2024-09-21T06:34:52.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.966+0000 7faa0659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea44b8ff60 0x55ea44f4d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:52.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.966+0000 7faa0659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea44b8ff60 0x55ea44f4d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42544/0 (socket says 172.21.15.67:42544) 2024-09-21T06:34:52.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.966+0000 7faa0659c640 1 -- 172.21.15.67:0/948712044 learned_addr learned my addr 172.21.15.67:0/948712044 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:34:52.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.966+0000 7faa0ed9d640 1 --2- 172.21.15.67:0/948712044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea44b6f280 0x55ea44f3cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:52.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.966+0000 7faa0659c640 1 --2- 172.21.15.67:0/948712044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea44b670a0 0x55ea44f3c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:52.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.966+0000 7faa0659c640 1 -- 172.21.15.67:0/948712044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea44b6f280 msgr2=0x55ea44f3cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:52.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.966+0000 7faa0659c640 1 --2- 172.21.15.67:0/948712044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea44b6f280 0x55ea44f3cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.967+0000 7faa0659c640 1 -- 172.21.15.67:0/948712044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea44b670a0 msgr2=0x55ea44f3c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:52.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.967+0000 7faa0659c640 1 --2- 172.21.15.67:0/948712044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea44b670a0 0x55ea44f3c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.967+0000 7faa0659c640 1 -- 172.21.15.67:0/948712044 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea44f381f0 con 0x55ea44b8ff60 2024-09-21T06:34:52.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.967+0000 7faa0659c640 1 --2- 172.21.15.67:0/948712044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea44b8ff60 0x55ea44f4d940 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800bd90 tx=0x7fa9f8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7823056925e65079 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:52.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.971+0000 7faa04d99640 1 -- 172.21.15.67:0/948712044 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8026040 con 0x55ea44b8ff60 2024-09-21T06:34:52.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.971+0000 7faa04d99640 1 -- 172.21.15.67:0/948712044 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9f8022070 con 0x55ea44b8ff60 2024-09-21T06:34:52.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.972+0000 7faa04d99640 1 -- 172.21.15.67:0/948712044 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f800e6d0 con 0x55ea44b8ff60 2024-09-21T06:34:52.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.972+0000 7faa1b595740 1 -- 172.21.15.67:0/948712044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea44b8ff60 msgr2=0x55ea44f4d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:52.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.972+0000 7faa1b595740 1 --2- 172.21.15.67:0/948712044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea44b8ff60 0x55ea44f4d940 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800bd90 tx=0x7fa9f8000bc0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.973+0000 7faa1b595740 1 -- 172.21.15.67:0/948712044 shutdown_connections 2024-09-21T06:34:52.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.973+0000 7faa1b595740 1 --2- 172.21.15.67:0/948712044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea44b8ff60 0x55ea44f4d940 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.973+0000 7faa1b595740 1 --2- 172.21.15.67:0/948712044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea44b6f280 0x55ea44f3cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.973+0000 7faa1b595740 1 --2- 172.21.15.67:0/948712044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea44b670a0 0x55ea44f3c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.973+0000 7faa1b595740 1 -- 172.21.15.67:0/948712044 >> 172.21.15.67:0/948712044 conn(0x55ea44b84350 msgr2=0x55ea44f419a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:52.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.973+0000 7faa1b595740 1 -- 172.21.15.67:0/948712044 shutdown_connections 2024-09-21T06:34:52.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.973+0000 7faa1b595740 1 -- 172.21.15.67:0/948712044 wait complete. 2024-09-21T06:34:52.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.973+0000 7faa1b595740 1 -- start start 2024-09-21T06:34:52.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.973+0000 7faa1b595740 1 Processor -- start 2024-09-21T06:34:52.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.974+0000 7faa1b595740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea44b670a0 0x55ea44f55810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:52.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.974+0000 7faa1b595740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea44b6f280 0x55ea44f55dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:52.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.974+0000 7faa1b595740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea44b8ff60 0x55ea44f5ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:52.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.974+0000 7faa1b595740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ea44f53c80 con 0x55ea44b6f280 2024-09-21T06:34:52.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.974+0000 7faa1b595740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ea44f535b0 con 0x55ea44b8ff60 2024-09-21T06:34:52.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.974+0000 7faa1b595740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ea44f54350 con 0x55ea44b670a0 2024-09-21T06:34:52.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.974+0000 7faa1b595740 10 client.0 _ll_register_callbacks cb 0x7fa9f80123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:34:52.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.974+0000 7faa0659c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea44b670a0 0x55ea44f55810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:52.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.974+0000 7faa0ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea44b6f280 0x55ea44f55dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:52.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.974+0000 7faa0659c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea44b670a0 0x55ea44f55810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.67:59870/0 (socket says 172.21.15.67:59870) 2024-09-21T06:34:52.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.974+0000 7faa0ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea44b6f280 0x55ea44f55dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42556/0 (socket says 172.21.15.67:42556) 2024-09-21T06:34:52.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.974+0000 7faa0659c640 1 -- 172.21.15.67:0/4077860847 learned_addr learned my addr 172.21.15.67:0/4077860847 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:34:52.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.974+0000 7faa0659c640 1 --2- 172.21.15.67:0/4077860847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea44b8ff60 0x55ea44f5ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:52.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.974+0000 7faa0ed9d640 1 -- 172.21.15.67:0/4077860847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea44b670a0 msgr2=0x55ea44f55810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:52.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.974+0000 7faa0ed9d640 1 --2- 172.21.15.67:0/4077860847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea44b670a0 0x55ea44f55810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.974+0000 7faa0ed9d640 1 -- 172.21.15.67:0/4077860847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea44b8ff60 msgr2=0x55ea44f5ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:34:52.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.974+0000 7faa0ed9d640 1 --2- 172.21.15.67:0/4077860847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea44b8ff60 0x55ea44f5ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.974+0000 7faa0ed9d640 1 -- 172.21.15.67:0/4077860847 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea44f5b480 con 0x55ea44b6f280 2024-09-21T06:34:52.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.974+0000 7faa0ed9d640 1 --2- 172.21.15.67:0/4077860847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea44b6f280 0x55ea44f55dd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7faa0002a930 tx=0x7faa00009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75738eb5156fa00a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:52.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.974+0000 7fa9f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:34:52.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.975+0000 7fa9f67fc640 1 -- 172.21.15.67:0/4077860847 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa00035040 con 0x55ea44b6f280 2024-09-21T06:34:52.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.975+0000 7fa9f67fc640 1 -- 172.21.15.67:0/4077860847 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7faa00005510 con 0x55ea44b6f280 2024-09-21T06:34:52.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.975+0000 7faa1b595740 1 -- 172.21.15.67:0/4077860847 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9f8022070 con 0x55ea44b6f280 2024-09-21T06:34:52.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.975+0000 7fa9f67fc640 1 -- 172.21.15.67:0/4077860847 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7faa000046c0 con 0x55ea44b6f280 2024-09-21T06:34:52.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.975+0000 7faa1b595740 10 client.14916 fetch_fsmap learned FSMap version 17 2024-09-21T06:34:52.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.975+0000 7faa1b595740 1 -- 172.21.15.67:0/4077860847 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55ea44f5c8a0 con 0x55ea44b6f280 2024-09-21T06:34:52.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.975+0000 7fa9f67fc640 1 -- 172.21.15.67:0/4077860847 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9f8022070 con 0x55ea44b6f280 2024-09-21T06:34:52.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.975+0000 7fa9f67fc640 1 -- 172.21.15.67:0/4077860847 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7faa0004a020 con 0x55ea44b6f280 2024-09-21T06:34:52.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.975+0000 7faa1b595740 10 client.14916 fetch_fsmap finished waiting for FSMap version 17 2024-09-21T06:34:52.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.975+0000 7faa1b595740 10 client.14916 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:34:52.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.975+0000 7faa1b595740 20 client.14916 populate_metadata read hostname 'smithi067' 2024-09-21T06:34:52.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.975+0000 7faa1b595740 1 --2- 172.21.15.67:0/4077860847 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55ea44b23c90 0x55ea44fc2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:52.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.975+0000 7faa1b595740 4 client.14916 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:34:52.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.975+0000 7faa1b595740 1 -- 172.21.15.67:0/4077860847 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "status"}) -- 0x55ea44eab610 con 0x55ea44b23c90 2024-09-21T06:34:52.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.976+0000 7faa0659c640 1 --2- 172.21.15.67:0/4077860847 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55ea44b23c90 0x55ea44fc2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:52.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.976+0000 7faa0659c640 1 --2- 172.21.15.67:0/4077860847 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55ea44b23c90 0x55ea44fc2440 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e4905bc1f0d8be5 server_cookie=4ba4fa5ac9ce01d9 in_seq=0 out_seq=0 2024-09-21T06:34:52.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.976+0000 7fa9f67fc640 10 client.14916 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:34:52.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.976+0000 7fa9f67fc640 1 -- 172.21.15.67:0/4077860847 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ea44eab610 con 0x55ea44b23c90 2024-09-21T06:34:52.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.976+0000 7fa9f67fc640 10 client.14916 handle_command_reply: tid=0 2024-09-21T06:34:52.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.976+0000 7faa1b595740 1 client.14916 shutdown 2024-09-21T06:34:52.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.976+0000 7faa1b595740 1 -- 172.21.15.67:0/4077860847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea44b6f280 msgr2=0x55ea44f55dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:52.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.976+0000 7faa1b595740 1 --2- 172.21.15.67:0/4077860847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea44b6f280 0x55ea44f55dd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7faa0002a930 tx=0x7faa00009680 comp rx=0 tx=0).stop 2024-09-21T06:34:52.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.977+0000 7faa1b595740 1 -- 172.21.15.67:0/4077860847 shutdown_connections 2024-09-21T06:34:52.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.977+0000 7faa1b595740 1 --2- 172.21.15.67:0/4077860847 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55ea44b23c90 0x55ea44fc2440 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.977+0000 7faa1b595740 1 --2- 172.21.15.67:0/4077860847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea44b8ff60 0x55ea44f5ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.977+0000 7fa9f67fc640 0 client.14916 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:34:52.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.977+0000 7faa1b595740 1 --2- 172.21.15.67:0/4077860847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea44b6f280 0x55ea44f55dd0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.977+0000 7faa1b595740 1 --2- 172.21.15.67:0/4077860847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea44b670a0 0x55ea44f55810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.977+0000 7faa1b595740 1 -- 172.21.15.67:0/4077860847 >> 172.21.15.67:0/4077860847 conn(0x55ea44b84350 msgr2=0x55ea44f3c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:52.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.977+0000 7faa1b595740 1 -- 172.21.15.67:0/4077860847 shutdown_connections 2024-09-21T06:34:52.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.977+0000 7faa1b595740 1 -- 172.21.15.67:0/4077860847 wait complete. 2024-09-21T06:34:52.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.977+0000 7faa1b595740 20 client.14916 trim_cache size 0 max 16384 2024-09-21T06:34:52.982 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-21T06:34:52.982 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "2a791768-77e2-11ef-baf5-efdc52797490", 2024-09-21T06:34:52.982 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-09-21T06:34:52.982 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24671, 2024-09-21T06:34:52.982 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-09-21T06:34:52.982 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-09-21T06:34:52.982 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-09-21T06:34:52.982 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 19.328194421999999, 2024-09-21T06:34:52.982 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 17, 2024-09-21T06:34:52.983 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 81, 2024-09-21T06:34:52.983 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 81, 2024-09-21T06:34:52.983 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 30.908102208999999 2024-09-21T06:34:52.983 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-21T06:34:52.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.977+0000 7faa15021640 1 -- 172.21.15.67:0/240668854 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fa9fc0b6510 msgr2=0x7fa9fc0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:52.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.977+0000 7faa15021640 1 --2- 172.21.15.67:0/240668854 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fa9fc0b6510 0x7fa9fc0b8a20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7faa080aa850 tx=0x7faa080ddfd0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.977+0000 7faa15021640 1 -- 172.21.15.67:0/240668854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa10108cf0 msgr2=0x7faa1019fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:52.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.977+0000 7faa15021640 1 --2- 172.21.15.67:0/240668854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa10108cf0 0x7faa1019fbe0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7faa0800eaa0 tx=0x7faa0800ef60 comp rx=0 tx=0).stop 2024-09-21T06:34:52.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.977+0000 7faa15021640 1 -- 172.21.15.67:0/240668854 shutdown_connections 2024-09-21T06:34:52.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.977+0000 7faa15021640 1 --2- 172.21.15.67:0/240668854 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fa9fc0b6510 0x7fa9fc0b8a20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.977+0000 7faa15021640 1 --2- 172.21.15.67:0/240668854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa10108cf0 0x7faa1019fbe0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.977+0000 7faa15021640 1 --2- 172.21.15.67:0/240668854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa101083a0 0x7faa1019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.977+0000 7faa15021640 1 --2- 172.21.15.67:0/240668854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa10102a60 0x7faa1019a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:52.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.977+0000 7faa15021640 1 -- 172.21.15.67:0/240668854 >> 172.21.15.67:0/240668854 conn(0x7faa10079730 msgr2=0x7faa101067f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:52.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.978+0000 7faa15021640 1 -- 172.21.15.67:0/240668854 shutdown_connections 2024-09-21T06:34:52.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:52.978+0000 7faa15021640 1 -- 172.21.15.67:0/240668854 wait complete. 2024-09-21T06:34:53.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:52 smithi203 ceph-mon[33735]: pgmap v305: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:53.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:52 smithi067 ceph-mon[31232]: pgmap v305: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:53.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:52 smithi100 ceph-mon[33401]: pgmap v305: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:53.747 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T06:34:53.748 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-21T06:34:53.748 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs ls 2024-09-21T06:34:54.007 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:34:54.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.537+0000 7fcd4276c640 1 -- 172.21.15.67:0/1190669457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd3c076080 msgr2=0x7fcd3c10fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:54.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.537+0000 7fcd4276c640 1 --2- 172.21.15.67:0/1190669457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd3c076080 0x7fcd3c10fc00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009900 tx=0x7fcd3002a410 comp rx=0 tx=0).stop 2024-09-21T06:34:54.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.538+0000 7fcd4276c640 1 -- 172.21.15.67:0/1190669457 shutdown_connections 2024-09-21T06:34:54.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.538+0000 7fcd4276c640 1 --2- 172.21.15.67:0/1190669457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd3c110140 0x7fcd3c112560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:54.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.538+0000 7fcd4276c640 1 --2- 172.21.15.67:0/1190669457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd3c076080 0x7fcd3c10fc00 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:54.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.538+0000 7fcd4276c640 1 --2- 172.21.15.67:0/1190669457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd3c075730 0x7fcd3c075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:54.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.538+0000 7fcd4276c640 1 -- 172.21.15.67:0/1190669457 >> 172.21.15.67:0/1190669457 conn(0x7fcd3c07b850 msgr2=0x7fcd3c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:54.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.538+0000 7fcd4276c640 1 -- 172.21.15.67:0/1190669457 shutdown_connections 2024-09-21T06:34:54.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.538+0000 7fcd4276c640 1 -- 172.21.15.67:0/1190669457 wait complete. 2024-09-21T06:34:54.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.538+0000 7fcd4276c640 1 Processor -- start 2024-09-21T06:34:54.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.538+0000 7fcd4276c640 1 -- start start 2024-09-21T06:34:54.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.539+0000 7fcd4276c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd3c075730 0x7fcd3c19c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:54.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.539+0000 7fcd4276c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd3c076080 0x7fcd3c19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:54.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.539+0000 7fcd4276c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd3c110140 0x7fcd3c1a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:54.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.539+0000 7fcd4276c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c115b00 con 0x7fcd3c110140 2024-09-21T06:34:54.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.539+0000 7fcd4276c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c115980 con 0x7fcd3c075730 2024-09-21T06:34:54.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.539+0000 7fcd4276c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c115c80 con 0x7fcd3c076080 2024-09-21T06:34:54.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.539+0000 7fcd40d09640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd3c110140 0x7fcd3c1a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:54.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.539+0000 7fcd40d09640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd3c110140 0x7fcd3c1a1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42586/0 (socket says 172.21.15.67:42586) 2024-09-21T06:34:54.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.539+0000 7fcd40d09640 1 -- 172.21.15.67:0/848326929 learned_addr learned my addr 172.21.15.67:0/848326929 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:34:54.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.539+0000 7fcd3bfff640 1 --2- 172.21.15.67:0/848326929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd3c075730 0x7fcd3c19c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:54.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.539+0000 7fcd3b7fe640 1 --2- 172.21.15.67:0/848326929 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd3c076080 0x7fcd3c19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:54.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.539+0000 7fcd40d09640 1 -- 172.21.15.67:0/848326929 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd3c076080 msgr2=0x7fcd3c19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:54.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.539+0000 7fcd40d09640 1 --2- 172.21.15.67:0/848326929 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd3c076080 0x7fcd3c19cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:54.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.539+0000 7fcd40d09640 1 -- 172.21.15.67:0/848326929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd3c075730 msgr2=0x7fcd3c19c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:54.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.539+0000 7fcd40d09640 1 --2- 172.21.15.67:0/848326929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd3c075730 0x7fcd3c19c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:54.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.539+0000 7fcd40d09640 1 -- 172.21.15.67:0/848326929 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd3c1a25e0 con 0x7fcd3c110140 2024-09-21T06:34:54.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.539+0000 7fcd3b7fe640 1 --2- 172.21.15.67:0/848326929 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd3c076080 0x7fcd3c19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:54.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.540+0000 7fcd40d09640 1 --2- 172.21.15.67:0/848326929 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd3c110140 0x7fcd3c1a1eb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400eaa0 tx=0x7fcd3400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:54.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.540+0000 7fcd397fa640 1 -- 172.21.15.67:0/848326929 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3400cdb0 con 0x7fcd3c110140 2024-09-21T06:34:54.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.540+0000 7fcd397fa640 1 -- 172.21.15.67:0/848326929 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd34005020 con 0x7fcd3c110140 2024-09-21T06:34:54.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.540+0000 7fcd4276c640 1 -- 172.21.15.67:0/848326929 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd3c1a28d0 con 0x7fcd3c110140 2024-09-21T06:34:54.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.540+0000 7fcd397fa640 1 -- 172.21.15.67:0/848326929 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd340116d0 con 0x7fcd3c110140 2024-09-21T06:34:54.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.540+0000 7fcd4276c640 1 -- 172.21.15.67:0/848326929 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd3c1a9030 con 0x7fcd3c110140 2024-09-21T06:34:54.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.541+0000 7fcd4276c640 1 -- 172.21.15.67:0/848326929 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd3c075b40 con 0x7fcd3c110140 2024-09-21T06:34:54.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.543+0000 7fcd397fa640 1 -- 172.21.15.67:0/848326929 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fcd340051d0 con 0x7fcd3c110140 2024-09-21T06:34:54.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.543+0000 7fcd397fa640 1 --2- 172.21.15.67:0/848326929 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fcd240b6590 0x7fcd240b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:54.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.543+0000 7fcd3bfff640 1 --2- 172.21.15.67:0/848326929 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fcd240b6590 0x7fcd240b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:54.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.543+0000 7fcd397fa640 1 -- 172.21.15.67:0/848326929 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7fcd340e79b0 con 0x7fcd3c110140 2024-09-21T06:34:54.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.544+0000 7fcd3bfff640 1 --2- 172.21.15.67:0/848326929 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fcd240b6590 0x7fcd240b8aa0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c009bd0 tx=0x7fcd2c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:54.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.546+0000 7fcd397fa640 1 -- 172.21.15.67:0/848326929 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcd34016030 con 0x7fcd3c110140 2024-09-21T06:34:54.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.775+0000 7fcd4276c640 1 -- 172.21.15.67:0/848326929 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcd3c19d7a0 con 0x7fcd3c110140 2024-09-21T06:34:54.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.776+0000 7fcd397fa640 1 -- 172.21.15.67:0/848326929 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7fcd3c19d7a0 con 0x7fcd3c110140 2024-09-21T06:34:54.777 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T06:34:54.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.778+0000 7fcd4276c640 1 -- 172.21.15.67:0/848326929 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fcd240b6590 msgr2=0x7fcd240b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:54.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.778+0000 7fcd4276c640 1 --2- 172.21.15.67:0/848326929 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fcd240b6590 0x7fcd240b8aa0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c009bd0 tx=0x7fcd2c005f90 comp rx=0 tx=0).stop 2024-09-21T06:34:54.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.778+0000 7fcd4276c640 1 -- 172.21.15.67:0/848326929 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd3c110140 msgr2=0x7fcd3c1a1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:54.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.778+0000 7fcd4276c640 1 --2- 172.21.15.67:0/848326929 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd3c110140 0x7fcd3c1a1eb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400eaa0 tx=0x7fcd3400ef60 comp rx=0 tx=0).stop 2024-09-21T06:34:54.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.778+0000 7fcd4276c640 1 -- 172.21.15.67:0/848326929 shutdown_connections 2024-09-21T06:34:54.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.778+0000 7fcd4276c640 1 --2- 172.21.15.67:0/848326929 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fcd240b6590 0x7fcd240b8aa0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:54.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.778+0000 7fcd4276c640 1 --2- 172.21.15.67:0/848326929 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd3c110140 0x7fcd3c1a1eb0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:54.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.778+0000 7fcd4276c640 1 --2- 172.21.15.67:0/848326929 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd3c076080 0x7fcd3c19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:54.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.778+0000 7fcd4276c640 1 --2- 172.21.15.67:0/848326929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd3c075730 0x7fcd3c19c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:54.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.778+0000 7fcd4276c640 1 -- 172.21.15.67:0/848326929 >> 172.21.15.67:0/848326929 conn(0x7fcd3c07b850 msgr2=0x7fcd3c101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:54.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.778+0000 7fcd4276c640 1 -- 172.21.15.67:0/848326929 shutdown_connections 2024-09-21T06:34:54.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:54.778+0000 7fcd4276c640 1 -- 172.21.15.67:0/848326929 wait complete. 2024-09-21T06:34:55.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:54 smithi067 ceph-mon[31232]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:55.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:54 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/848326929' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T06:34:55.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:54 smithi203 ceph-mon[33735]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:55.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:54 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/848326929' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T06:34:55.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:54 smithi100 ceph-mon[33401]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:55.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:54 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/848326929' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T06:34:55.451 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T06:34:55.451 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T06:34:55.451 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2024-09-21T06:34:55.451 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-21T06:34:55.451 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-21T06:34:55.452 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T06:34:55.452 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T06:34:55.452 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T06:34:55.452 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-21T06:34:55.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:34:55.452 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2024-09-21T06:34:55.470 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T06:34:55.470 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T06:34:55.470 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T06:34:55.470 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:34:55.470 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2024-09-21T06:34:55.470 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:34:55.470 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:34:55.470 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2024-09-21T06:34:55.470 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f0 2024-09-21T06:34:55.470 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:34:55.470 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2024-09-21T06:34:55.471 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f1 2024-09-21T06:34:55.471 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:34:55.471 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2024-09-21T06:34:55.471 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T06:34:55.471 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2024-09-21T06:34:55.471 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T06:34:55.471 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 42236sec preferred_lft 42236sec 2024-09-21T06:34:55.471 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link noprefixroute 2024-09-21T06:34:55.471 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:34:55.472 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T06:34:55.472 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:34:55.472 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-09-21T06:34:55.472 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add name ceph-brx type bridge 2024-09-21T06:34:55.472 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr flush dev ceph-brx 2024-09-21T06:34:55.472 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set ceph-brx up 2024-09-21T06:34:55.472 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T06:34:55.472 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-09-21T06:34:55.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:34:55.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:34:55.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:34:55.677 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T06:34:55.703 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-09-21T06:34:55.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:34:55.705 DEBUG:teuthology.orchestra.run.smithi067:> route 2024-09-21T06:34:55.777 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2024-09-21T06:34:55.778 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T06:34:55.778 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T06:34:55.778 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T06:34:55.778 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T06:34:55.778 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:34:55.778 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-09-21T06:34:55.778 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T06:34:55.779 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T06:34:55.779 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T06:34:55.779 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-09-21T06:34:55.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:34:56.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:34:56.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:34:56.074 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-09-21T06:34:56.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:34:56.091 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list-id 2024-09-21T06:34:56.146 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:34:56.147 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-09-21T06:34:56.147 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T06:34:56.147 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-21T06:34:56.147 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-09-21T06:34:56.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:34:56.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:34:56.265 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-21T06:34:56.266 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:34:56.266 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-09-21T06:34:56.266 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-21T06:34:56.266 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T06:34:56.266 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-21T06:34:56.266 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-21T06:34:56.266 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-21T06:34:56.266 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-09-21T06:34:56.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:34:56.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:34:56.397 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:34:56.398 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-09-21T06:34:56.398 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2024-09-21T06:34:56.398 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T06:34:56.398 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-09-21T06:34:56.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:34:56.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:34:56.477 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-21T06:34:56.477 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:34:56.477 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-21T06:34:56.492 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-21T06:34:56.492 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:34:56.492 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-21T06:34:56.548 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-09-21T06:34:56.615 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T06:34:56.671 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-09-21T06:34:56.671 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-09-21T06:34:56.671 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-09-21T06:34:56.672 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-09-21T06:34:56.672 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-09-21T06:34:56.672 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-09-21T06:34:56.672 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-09-21T06:34:56.672 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-09-21T06:34:56.672 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-09-21T06:34:56.672 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-09-21T06:34:56.672 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-09-21T06:34:56.672 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-09-21T06:34:56.672 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T06:34:56.672 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-09-21T06:34:56.672 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-09-21T06:34:56.672 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-09-21T06:34:56.672 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-09-21T06:34:56.673 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-09-21T06:34:56.673 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-09-21T06:34:56.673 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T06:34:56.673 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T06:34:56.673 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T06:34:56.673 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T06:34:56.673 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-09-21T06:34:56.673 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-09-21T06:34:56.673 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay 2024-09-21T06:34:56.673 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/29fcd3741f2cc055bd00d5d0fa0c56546c5e775a4a1f53b1e931f43fdf5ff1e0/merged 2024-09-21T06:34:56.673 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/32acf6b7fe983d1dd15da3c2e23dc552a0df99470cc8cb96b4f745c41be94cde/merged 2024-09-21T06:34:56.673 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/0 2024-09-21T06:34:56.673 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T06:34:56.673 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/bafc1e5188868475e941c6d45eee229f72ba6ee7586332805d28b4774bba0dc2/merged 2024-09-21T06:34:56.673 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/39c1ee7a488e1cd808010f93a252b973411dc42fdab768269153bca71d230d48/merged 2024-09-21T06:34:56.674 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/477ce1b7f70d362fd35b8467af3f66a9ab9ddfebe22f2ca05e951c9c7e60264a/merged 2024-09-21T06:34:56.674 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/8363bb5cac1f856dec433b9c1ed2f566cd6bb9affd699fe771e5341a35835dd1/merged 2024-09-21T06:34:56.674 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/b8c3e6584fa2ea8ae7847e9343eb1ddf695ef1ff17a5598fb95d5cf5c8a54492/merged 2024-09-21T06:34:56.674 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/0cf20f5642d1e6f9125be88aaa36a3c5267b9e1ded4255030dff6a66c6178016/merged 2024-09-21T06:34:56.674 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/3d9fd1163c75a40ffd0f5d7f6a53413e9606d02c0f9e9a960e4c0b2f08910a6f/merged 2024-09-21T06:34:56.674 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/4525d5ac138e0d41fde6a3521150af1932abef421e5475cd735b2a2670135f97/merged 2024-09-21T06:34:56.674 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-09-21T06:34:56.674 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T06:34:56.674 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T06:34:56.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:34:56.675 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-09-21T06:34:56.731 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T06:34:56.731 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-21T06:34:56.773 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-09-21T06:34:56.799 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T06:34:56.855 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-09-21T06:34:56.856 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-09-21T06:34:56.856 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-09-21T06:34:56.856 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-09-21T06:34:56.856 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-09-21T06:34:56.856 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-09-21T06:34:56.856 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-09-21T06:34:56.856 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-09-21T06:34:56.856 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-09-21T06:34:56.856 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-09-21T06:34:56.856 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-09-21T06:34:56.856 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-09-21T06:34:56.856 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T06:34:56.856 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-09-21T06:34:56.856 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-09-21T06:34:56.857 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-09-21T06:34:56.857 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-09-21T06:34:56.857 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-09-21T06:34:56.857 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-09-21T06:34:56.857 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T06:34:56.857 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T06:34:56.857 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T06:34:56.857 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T06:34:56.857 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-09-21T06:34:56.857 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-09-21T06:34:56.857 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay 2024-09-21T06:34:56.857 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/29fcd3741f2cc055bd00d5d0fa0c56546c5e775a4a1f53b1e931f43fdf5ff1e0/merged 2024-09-21T06:34:56.857 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/32acf6b7fe983d1dd15da3c2e23dc552a0df99470cc8cb96b4f745c41be94cde/merged 2024-09-21T06:34:56.857 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/0 2024-09-21T06:34:56.857 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T06:34:56.858 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/bafc1e5188868475e941c6d45eee229f72ba6ee7586332805d28b4774bba0dc2/merged 2024-09-21T06:34:56.858 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/39c1ee7a488e1cd808010f93a252b973411dc42fdab768269153bca71d230d48/merged 2024-09-21T06:34:56.858 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/477ce1b7f70d362fd35b8467af3f66a9ab9ddfebe22f2ca05e951c9c7e60264a/merged 2024-09-21T06:34:56.858 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/8363bb5cac1f856dec433b9c1ed2f566cd6bb9affd699fe771e5341a35835dd1/merged 2024-09-21T06:34:56.858 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/b8c3e6584fa2ea8ae7847e9343eb1ddf695ef1ff17a5598fb95d5cf5c8a54492/merged 2024-09-21T06:34:56.858 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/0cf20f5642d1e6f9125be88aaa36a3c5267b9e1ded4255030dff6a66c6178016/merged 2024-09-21T06:34:56.858 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/3d9fd1163c75a40ffd0f5d7f6a53413e9606d02c0f9e9a960e4c0b2f08910a6f/merged 2024-09-21T06:34:56.858 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/4525d5ac138e0d41fde6a3521150af1932abef421e5475cd735b2a2670135f97/merged 2024-09-21T06:34:56.858 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-09-21T06:34:56.858 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T06:34:56.858 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T06:34:56.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:34:56.859 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-09-21T06:34:56.896 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi067.stderr:2024-09-21T06:34:56.895+0000 7f6f7f2cf480 -1 init, newargv = 0x7f6f60004720 newargc=15 2024-09-21T06:34:56.896 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi067.stderr:ceph-fuse[75179]: starting ceph client 2024-09-21T06:34:56.903 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi067.stderr:ceph-fuse[75179]: starting fuse 2024-09-21T06:34:56.913 INFO:teuthology.orchestra.run.smithi067.stdout:120 2024-09-21T06:34:56.913 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-21T06:34:56.914 DEBUG:teuthology.orchestra.run.smithi067:> sudo stdin-killer -- python3 -c ' 2024-09-21T06:34:56.914 DEBUG:teuthology.orchestra.run.smithi067:> import glob 2024-09-21T06:34:56.914 DEBUG:teuthology.orchestra.run.smithi067:> import re 2024-09-21T06:34:56.914 DEBUG:teuthology.orchestra.run.smithi067:> import os 2024-09-21T06:34:56.914 DEBUG:teuthology.orchestra.run.smithi067:> import subprocess 2024-09-21T06:34:56.914 DEBUG:teuthology.orchestra.run.smithi067:> 2024-09-21T06:34:56.914 DEBUG:teuthology.orchestra.run.smithi067:> def _find_admin_socket(client_name): 2024-09-21T06:34:56.914 DEBUG:teuthology.orchestra.run.smithi067:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-21T06:34:56.914 DEBUG:teuthology.orchestra.run.smithi067:> files = glob.glob(asok_path) 2024-09-21T06:34:56.914 DEBUG:teuthology.orchestra.run.smithi067:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-21T06:34:56.914 DEBUG:teuthology.orchestra.run.smithi067:> 2024-09-21T06:34:56.914 DEBUG:teuthology.orchestra.run.smithi067:> # Given a non-glob path, it better be there 2024-09-21T06:34:56.914 DEBUG:teuthology.orchestra.run.smithi067:> if "*" not in asok_path: 2024-09-21T06:34:56.915 DEBUG:teuthology.orchestra.run.smithi067:> assert(len(files) == 1) 2024-09-21T06:34:56.915 DEBUG:teuthology.orchestra.run.smithi067:> return files[0] 2024-09-21T06:34:56.915 DEBUG:teuthology.orchestra.run.smithi067:> 2024-09-21T06:34:56.915 DEBUG:teuthology.orchestra.run.smithi067:> for f in files: 2024-09-21T06:34:56.915 DEBUG:teuthology.orchestra.run.smithi067:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T06:34:56.915 DEBUG:teuthology.orchestra.run.smithi067:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T06:34:56.915 DEBUG:teuthology.orchestra.run.smithi067:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T06:34:56.915 DEBUG:teuthology.orchestra.run.smithi067:> contents = proc_f.read() 2024-09-21T06:34:56.915 DEBUG:teuthology.orchestra.run.smithi067:> if mountpoint in contents: 2024-09-21T06:34:56.915 DEBUG:teuthology.orchestra.run.smithi067:> return f 2024-09-21T06:34:56.915 DEBUG:teuthology.orchestra.run.smithi067:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T06:34:56.915 DEBUG:teuthology.orchestra.run.smithi067:> 2024-09-21T06:34:56.915 DEBUG:teuthology.orchestra.run.smithi067:> print(_find_admin_socket("client.admin")) 2024-09-21T06:34:56.915 DEBUG:teuthology.orchestra.run.smithi067:> ' 2024-09-21T06:34:57.035 INFO:teuthology.orchestra.run.smithi067.stdout:/var/run/ceph/ceph-client.admin.75179.asok 2024-09-21T06:34:57.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:34:57.046 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.75179.asok 2024-09-21T06:34:57.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:34:57.046 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.75179.asok status 2024-09-21T06:34:57.193 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-21T06:34:57.193 INFO:teuthology.orchestra.run.smithi067.stdout: "metadata": { 2024-09-21T06:34:57.193 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T06:34:57.193 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T06:34:57.193 INFO:teuthology.orchestra.run.smithi067.stdout: "entity_id": "admin", 2024-09-21T06:34:57.193 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": "smithi067", 2024-09-21T06:34:57.193 INFO:teuthology.orchestra.run.smithi067.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-21T06:34:57.193 INFO:teuthology.orchestra.run.smithi067.stdout: "pid": "75179", 2024-09-21T06:34:57.193 INFO:teuthology.orchestra.run.smithi067.stdout: "root": "/" 2024-09-21T06:34:57.193 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:34:57.193 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_count": 0, 2024-09-21T06:34:57.194 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_pinned_count": 0, 2024-09-21T06:34:57.194 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 14928, 2024-09-21T06:34:57.194 INFO:teuthology.orchestra.run.smithi067.stdout: "inst": { 2024-09-21T06:34:57.194 INFO:teuthology.orchestra.run.smithi067.stdout: "name": { 2024-09-21T06:34:57.194 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "client", 2024-09-21T06:34:57.194 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 14928 2024-09-21T06:34:57.194 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:34:57.194 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-09-21T06:34:57.194 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-09-21T06:34:57.194 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2024-09-21T06:34:57.194 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 477055163 2024-09-21T06:34:57.194 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:34:57.194 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:34:57.194 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-09-21T06:34:57.194 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-09-21T06:34:57.195 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2024-09-21T06:34:57.195 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 477055163 2024-09-21T06:34:57.195 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:34:57.195 INFO:teuthology.orchestra.run.smithi067.stdout: "inst_str": "client.14928 192.168.0.1:0/477055163", 2024-09-21T06:34:57.195 INFO:teuthology.orchestra.run.smithi067.stdout: "addr_str": "192.168.0.1:0/477055163", 2024-09-21T06:34:57.196 INFO:teuthology.orchestra.run.smithi067.stdout: "inode_count": 1, 2024-09-21T06:34:57.196 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_epoch": 17, 2024-09-21T06:34:57.196 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch": 81, 2024-09-21T06:34:57.196 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch_barrier": 0, 2024-09-21T06:34:57.196 INFO:teuthology.orchestra.run.smithi067.stdout: "blocklisted": false, 2024-09-21T06:34:57.196 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs" 2024-09-21T06:34:57.196 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-21T06:34:57.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:34:57.202 DEBUG:teuthology.orchestra.run.smithi067:> stat --file-system '--printf=%T 2024-09-21T06:34:57.202 DEBUG:teuthology.orchestra.run.smithi067:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-21T06:34:57.219 INFO:teuthology.orchestra.run.smithi067.stdout:fuseblk 2024-09-21T06:34:57.219 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-21T06:34:57.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:34:57.219 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-21T06:34:57.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:56 smithi203 ceph-mon[33735]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:57.281 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:56 smithi067 ceph-mon[31232]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:57.286 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-21T06:34:57.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:56 smithi100 ceph-mon[33401]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:57.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:34:57.392 INFO:teuthology.orchestra.run.smithi067.stdout:# file: . 2024-09-21T06:34:57.392 INFO:teuthology.orchestra.run.smithi067.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-21T06:34:57.392 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:34:57.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:34:57.398 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-21T06:34:57.398 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:34:57.416 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T06:34:57.715 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:34:58.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.360+0000 7fd59fba4640 1 -- 172.21.15.67:0/2989711841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd598102ae0 msgr2=0x7fd598102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:58.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.360+0000 7fd59fba4640 1 --2- 172.21.15.67:0/2989711841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd598102ae0 0x7fd598102ef0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd594009900 tx=0x7fd59402a410 comp rx=0 tx=0).stop 2024-09-21T06:34:58.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.361+0000 7fd59fba4640 1 -- 172.21.15.67:0/2989711841 shutdown_connections 2024-09-21T06:34:58.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.361+0000 7fd59fba4640 1 --2- 172.21.15.67:0/2989711841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd598108d20 0x7fd59810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:58.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.361+0000 7fd59fba4640 1 --2- 172.21.15.67:0/2989711841 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5981083d0 0x7fd5981087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:58.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.361+0000 7fd59fba4640 1 --2- 172.21.15.67:0/2989711841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd598102ae0 0x7fd598102ef0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:58.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.361+0000 7fd59fba4640 1 -- 172.21.15.67:0/2989711841 >> 172.21.15.67:0/2989711841 conn(0x7fd598079760 msgr2=0x7fd598079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:58.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.361+0000 7fd59fba4640 1 -- 172.21.15.67:0/2989711841 shutdown_connections 2024-09-21T06:34:58.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.361+0000 7fd59fba4640 1 -- 172.21.15.67:0/2989711841 wait complete. 2024-09-21T06:34:58.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.361+0000 7fd59fba4640 1 Processor -- start 2024-09-21T06:34:58.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.361+0000 7fd59fba4640 1 -- start start 2024-09-21T06:34:58.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.362+0000 7fd59fba4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd598102ae0 0x7fd59819b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:58.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.362+0000 7fd59fba4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5981083d0 0x7fd59819b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:58.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.362+0000 7fd59fba4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd598108d20 0x7fd59819bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:58.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.362+0000 7fd59fba4640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd598112840 con 0x7fd598108d20 2024-09-21T06:34:58.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.362+0000 7fd59fba4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd5981126c0 con 0x7fd5981083d0 2024-09-21T06:34:58.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.362+0000 7fd59fba4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd5981129c0 con 0x7fd598102ae0 2024-09-21T06:34:58.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.362+0000 7fd59e141640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd598108d20 0x7fd59819bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:58.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.362+0000 7fd59e141640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd598108d20 0x7fd59819bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56122/0 (socket says 172.21.15.67:56122) 2024-09-21T06:34:58.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.362+0000 7fd59e141640 1 -- 172.21.15.67:0/715236035 learned_addr learned my addr 172.21.15.67:0/715236035 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:34:58.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.362+0000 7fd59d940640 1 --2- 172.21.15.67:0/715236035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd598102ae0 0x7fd59819b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:58.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.362+0000 7fd59d13f640 1 --2- 172.21.15.67:0/715236035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5981083d0 0x7fd59819b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:58.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.362+0000 7fd59e141640 1 -- 172.21.15.67:0/715236035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd598102ae0 msgr2=0x7fd59819b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:58.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.362+0000 7fd59e141640 1 --2- 172.21.15.67:0/715236035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd598102ae0 0x7fd59819b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:58.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.362+0000 7fd59e141640 1 -- 172.21.15.67:0/715236035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5981083d0 msgr2=0x7fd59819b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:58.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.362+0000 7fd59e141640 1 --2- 172.21.15.67:0/715236035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5981083d0 0x7fd59819b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:58.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.362+0000 7fd59e141640 1 -- 172.21.15.67:0/715236035 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5981a6580 con 0x7fd598108d20 2024-09-21T06:34:58.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.362+0000 7fd59d940640 1 --2- 172.21.15.67:0/715236035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd598102ae0 0x7fd59819b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:34:58.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.362+0000 7fd59e141640 1 --2- 172.21.15.67:0/715236035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd598108d20 0x7fd59819bcf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd590009fd0 tx=0x7fd59000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:58.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.362+0000 7fd58effd640 1 -- 172.21.15.67:0/715236035 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd590017070 con 0x7fd598108d20 2024-09-21T06:34:58.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.362+0000 7fd59fba4640 1 -- 172.21.15.67:0/715236035 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5981a6870 con 0x7fd598108d20 2024-09-21T06:34:58.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.362+0000 7fd58effd640 1 -- 172.21.15.67:0/715236035 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd590003e00 con 0x7fd598108d20 2024-09-21T06:34:58.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.362+0000 7fd58effd640 1 -- 172.21.15.67:0/715236035 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd590010460 con 0x7fd598108d20 2024-09-21T06:34:58.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.363+0000 7fd59fba4640 1 -- 172.21.15.67:0/715236035 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5981a6d60 con 0x7fd598108d20 2024-09-21T06:34:58.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.364+0000 7fd59fba4640 1 -- 172.21.15.67:0/715236035 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd598103d10 con 0x7fd598108d20 2024-09-21T06:34:58.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.365+0000 7fd58effd640 1 -- 172.21.15.67:0/715236035 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fd590003970 con 0x7fd598108d20 2024-09-21T06:34:58.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.366+0000 7fd58effd640 1 --2- 172.21.15.67:0/715236035 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd5840b6510 0x7fd5840b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:34:58.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.366+0000 7fd59d940640 1 --2- 172.21.15.67:0/715236035 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd5840b6510 0x7fd5840b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:34:58.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.367+0000 7fd58effd640 1 -- 172.21.15.67:0/715236035 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7fd5900e7710 con 0x7fd598108d20 2024-09-21T06:34:58.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.367+0000 7fd59d940640 1 --2- 172.21.15.67:0/715236035 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd5840b6510 0x7fd5840b8a20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd594006010 tx=0x7fd594005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:34:58.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.370+0000 7fd58effd640 1 -- 172.21.15.67:0/715236035 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd5900add60 con 0x7fd598108d20 2024-09-21T06:34:58.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.569+0000 7fd59fba4640 1 -- 172.21.15.67:0/715236035 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd598102ef0 con 0x7fd598108d20 2024-09-21T06:34:58.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.570+0000 7fd58effd640 1 -- 172.21.15.67:0/715236035 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v81) ==== 81+0+592 (secure 0 0 0) 0x7fd5900b2140 con 0x7fd598108d20 2024-09-21T06:34:58.571 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/1181476774 2024-09-22T06:27:03.740475+0000 2024-09-21T06:34:58.571 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3100321138 2024-09-22T06:27:03.740475+0000 2024-09-21T06:34:58.571 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/2309364298 2024-09-22T06:27:03.740475+0000 2024-09-21T06:34:58.571 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3458717076 2024-09-22T06:27:03.740475+0000 2024-09-21T06:34:58.571 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/4110206171 2024-09-22T06:26:37.041377+0000 2024-09-21T06:34:58.571 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/1181476774 2024-09-22T06:27:03.740475+0000 2024-09-21T06:34:58.572 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/2910038982 2024-09-22T06:26:37.041377+0000 2024-09-21T06:34:58.572 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/2622729103 2024-09-22T06:26:37.041377+0000 2024-09-21T06:34:58.572 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/2622729103 2024-09-22T06:26:37.041377+0000 2024-09-21T06:34:58.572 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3346784528 2024-09-22T06:26:37.041377+0000 2024-09-21T06:34:58.572 INFO:teuthology.orchestra.run.smithi067.stderr:listed 10 entries 2024-09-21T06:34:58.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.572+0000 7fd59fba4640 1 -- 172.21.15.67:0/715236035 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd5840b6510 msgr2=0x7fd5840b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:58.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.572+0000 7fd59fba4640 1 --2- 172.21.15.67:0/715236035 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd5840b6510 0x7fd5840b8a20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd594006010 tx=0x7fd594005a20 comp rx=0 tx=0).stop 2024-09-21T06:34:58.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.572+0000 7fd59fba4640 1 -- 172.21.15.67:0/715236035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd598108d20 msgr2=0x7fd59819bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:34:58.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.572+0000 7fd59fba4640 1 --2- 172.21.15.67:0/715236035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd598108d20 0x7fd59819bcf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd590009fd0 tx=0x7fd59000b260 comp rx=0 tx=0).stop 2024-09-21T06:34:58.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.572+0000 7fd59fba4640 1 -- 172.21.15.67:0/715236035 shutdown_connections 2024-09-21T06:34:58.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.572+0000 7fd59fba4640 1 --2- 172.21.15.67:0/715236035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5981083d0 0x7fd59819b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:58.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.572+0000 7fd59fba4640 1 --2- 172.21.15.67:0/715236035 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd5840b6510 0x7fd5840b8a20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:58.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.572+0000 7fd59fba4640 1 --2- 172.21.15.67:0/715236035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd598108d20 0x7fd59819bcf0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:58.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.572+0000 7fd59fba4640 1 --2- 172.21.15.67:0/715236035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd598102ae0 0x7fd59819b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:34:58.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.572+0000 7fd59fba4640 1 -- 172.21.15.67:0/715236035 >> 172.21.15.67:0/715236035 conn(0x7fd598079760 msgr2=0x7fd598101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:34:58.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.572+0000 7fd59fba4640 1 -- 172.21.15.67:0/715236035 shutdown_connections 2024-09-21T06:34:58.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:34:58.572+0000 7fd59fba4640 1 -- 172.21.15.67:0/715236035 wait complete. 2024-09-21T06:34:59.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:58 smithi203 ceph-mon[33735]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:59.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:34:58 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/715236035' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T06:34:59.291 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:58 smithi067 ceph-mon[31232]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:59.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:34:58 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/715236035' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T06:34:59.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:58 smithi100 ceph-mon[33401]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:34:59.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:34:58 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/715236035' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T06:34:59.353 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-21T06:34:59.353 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:34:59.372 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T06:34:59.672 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:00.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.170+0000 7fc15f33d640 1 -- 172.21.15.67:0/3168332049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc158108520 msgr2=0x7fc15810ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:00.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.170+0000 7fc15f33d640 1 --2- 172.21.15.67:0/3168332049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc158108520 0x7fc15810ed50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc150009ca0 tx=0x7fc15002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:35:00.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.170+0000 7fc15f33d640 1 -- 172.21.15.67:0/3168332049 shutdown_connections 2024-09-21T06:35:00.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.170+0000 7fc15f33d640 1 --2- 172.21.15.67:0/3168332049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc158107bd0 0x7fc158107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:00.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.170+0000 7fc15f33d640 1 --2- 172.21.15.67:0/3168332049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc158108520 0x7fc15810ed50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:00.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.170+0000 7fc15f33d640 1 --2- 172.21.15.67:0/3168332049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc158102290 0x7fc1581026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:00.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.170+0000 7fc15f33d640 1 -- 172.21.15.67:0/3168332049 >> 172.21.15.67:0/3168332049 conn(0x7fc158079ea0 msgr2=0x7fc15807a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:00.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.170+0000 7fc15f33d640 1 -- 172.21.15.67:0/3168332049 shutdown_connections 2024-09-21T06:35:00.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.170+0000 7fc15f33d640 1 -- 172.21.15.67:0/3168332049 wait complete. 2024-09-21T06:35:00.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.171+0000 7fc15f33d640 1 Processor -- start 2024-09-21T06:35:00.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.171+0000 7fc15f33d640 1 -- start start 2024-09-21T06:35:00.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.171+0000 7fc15f33d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc158102290 0x7fc158110640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:00.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.171+0000 7fc15f33d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc158107bd0 0x7fc158110b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:00.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.171+0000 7fc15f33d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc158108520 0x7fc1581110c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:00.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.171+0000 7fc15f33d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc158113a50 con 0x7fc158102290 2024-09-21T06:35:00.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.171+0000 7fc15f33d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc1581138d0 con 0x7fc158107bd0 2024-09-21T06:35:00.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.171+0000 7fc15f33d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc158113bd0 con 0x7fc158108520 2024-09-21T06:35:00.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.171+0000 7fc15d0d9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc158102290 0x7fc158110640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:00.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.171+0000 7fc15d0d9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc158102290 0x7fc158110640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56126/0 (socket says 172.21.15.67:56126) 2024-09-21T06:35:00.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.171+0000 7fc15d0d9640 1 -- 172.21.15.67:0/3469924808 learned_addr learned my addr 172.21.15.67:0/3469924808 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:00.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.171+0000 7fc15d8da640 1 --2- 172.21.15.67:0/3469924808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc158108520 0x7fc1581110c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:00.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.171+0000 7fc15c8d8640 1 --2- 172.21.15.67:0/3469924808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc158107bd0 0x7fc158110b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:00.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.171+0000 7fc15d0d9640 1 -- 172.21.15.67:0/3469924808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc158108520 msgr2=0x7fc1581110c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:00.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.171+0000 7fc15d0d9640 1 --2- 172.21.15.67:0/3469924808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc158108520 0x7fc1581110c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:00.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.171+0000 7fc15d0d9640 1 -- 172.21.15.67:0/3469924808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc158107bd0 msgr2=0x7fc158110b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:00.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.171+0000 7fc15d0d9640 1 --2- 172.21.15.67:0/3469924808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc158107bd0 0x7fc158110b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:00.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.171+0000 7fc15d0d9640 1 -- 172.21.15.67:0/3469924808 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1581a87c0 con 0x7fc158102290 2024-09-21T06:35:00.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.171+0000 7fc15c8d8640 1 --2- 172.21.15.67:0/3469924808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc158107bd0 0x7fc158110b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:00.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.172+0000 7fc15d0d9640 1 --2- 172.21.15.67:0/3469924808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc158102290 0x7fc158110640 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc15400c890 tx=0x7fc15400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:00.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.172+0000 7fc14e7fc640 1 -- 172.21.15.67:0/3469924808 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc154004220 con 0x7fc158102290 2024-09-21T06:35:00.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.172+0000 7fc14e7fc640 1 -- 172.21.15.67:0/3469924808 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc154007e40 con 0x7fc158102290 2024-09-21T06:35:00.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.172+0000 7fc14e7fc640 1 -- 172.21.15.67:0/3469924808 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc154004a00 con 0x7fc158102290 2024-09-21T06:35:00.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.172+0000 7fc15f33d640 1 -- 172.21.15.67:0/3469924808 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1581a8ab0 con 0x7fc158102290 2024-09-21T06:35:00.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.172+0000 7fc15f33d640 1 -- 172.21.15.67:0/3469924808 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1581a8fa0 con 0x7fc158102290 2024-09-21T06:35:00.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.173+0000 7fc15f33d640 1 -- 172.21.15.67:0/3469924808 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1581026a0 con 0x7fc158102290 2024-09-21T06:35:00.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.175+0000 7fc14e7fc640 1 -- 172.21.15.67:0/3469924808 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fc1540079b0 con 0x7fc158102290 2024-09-21T06:35:00.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.176+0000 7fc14e7fc640 1 --2- 172.21.15.67:0/3469924808 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc1440b6590 0x7fc1440b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:00.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.176+0000 7fc15c8d8640 1 --2- 172.21.15.67:0/3469924808 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc1440b6590 0x7fc1440b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:00.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.176+0000 7fc14e7fc640 1 -- 172.21.15.67:0/3469924808 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7fc1540e76d0 con 0x7fc158102290 2024-09-21T06:35:00.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.177+0000 7fc15c8d8640 1 --2- 172.21.15.67:0/3469924808 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc1440b6590 0x7fc1440b8aa0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc148005fd0 tx=0x7fc148005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:00.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.180+0000 7fc14e7fc640 1 -- 172.21.15.67:0/3469924808 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc154014030 con 0x7fc158102290 2024-09-21T06:35:00.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.382+0000 7fc15f33d640 1 -- 172.21.15.67:0/3469924808 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc158111940 con 0x7fc158102290 2024-09-21T06:35:00.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.382+0000 7fc14e7fc640 1 -- 172.21.15.67:0/3469924808 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v81) ==== 81+0+592 (secure 0 0 0) 0x7fc158111940 con 0x7fc158102290 2024-09-21T06:35:00.384 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/1181476774 2024-09-22T06:27:03.740475+0000 2024-09-21T06:35:00.384 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3100321138 2024-09-22T06:27:03.740475+0000 2024-09-21T06:35:00.384 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/2309364298 2024-09-22T06:27:03.740475+0000 2024-09-21T06:35:00.384 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3458717076 2024-09-22T06:27:03.740475+0000 2024-09-21T06:35:00.384 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/4110206171 2024-09-22T06:26:37.041377+0000 2024-09-21T06:35:00.384 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/1181476774 2024-09-22T06:27:03.740475+0000 2024-09-21T06:35:00.384 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/2910038982 2024-09-22T06:26:37.041377+0000 2024-09-21T06:35:00.384 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/2622729103 2024-09-22T06:26:37.041377+0000 2024-09-21T06:35:00.384 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/2622729103 2024-09-22T06:26:37.041377+0000 2024-09-21T06:35:00.384 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3346784528 2024-09-22T06:26:37.041377+0000 2024-09-21T06:35:00.385 INFO:teuthology.orchestra.run.smithi067.stderr:listed 10 entries 2024-09-21T06:35:00.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.384+0000 7fc15f33d640 1 -- 172.21.15.67:0/3469924808 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc1440b6590 msgr2=0x7fc1440b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:00.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.384+0000 7fc15f33d640 1 --2- 172.21.15.67:0/3469924808 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc1440b6590 0x7fc1440b8aa0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc148005fd0 tx=0x7fc148005850 comp rx=0 tx=0).stop 2024-09-21T06:35:00.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.384+0000 7fc15f33d640 1 -- 172.21.15.67:0/3469924808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc158102290 msgr2=0x7fc158110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:00.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.384+0000 7fc15f33d640 1 --2- 172.21.15.67:0/3469924808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc158102290 0x7fc158110640 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc15400c890 tx=0x7fc15400cd50 comp rx=0 tx=0).stop 2024-09-21T06:35:00.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.384+0000 7fc15f33d640 1 -- 172.21.15.67:0/3469924808 shutdown_connections 2024-09-21T06:35:00.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.384+0000 7fc15f33d640 1 --2- 172.21.15.67:0/3469924808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc158107bd0 0x7fc158110b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:00.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.384+0000 7fc15f33d640 1 --2- 172.21.15.67:0/3469924808 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc1440b6590 0x7fc1440b8aa0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:00.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.384+0000 7fc15f33d640 1 --2- 172.21.15.67:0/3469924808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc158108520 0x7fc1581110c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:00.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.384+0000 7fc15f33d640 1 --2- 172.21.15.67:0/3469924808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc158102290 0x7fc158110640 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:00.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.384+0000 7fc15f33d640 1 -- 172.21.15.67:0/3469924808 >> 172.21.15.67:0/3469924808 conn(0x7fc158079ea0 msgr2=0x7fc158103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:00.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.385+0000 7fc15f33d640 1 -- 172.21.15.67:0/3469924808 shutdown_connections 2024-09-21T06:35:00.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:00.385+0000 7fc15f33d640 1 -- 172.21.15.67:0/3469924808 wait complete. 2024-09-21T06:35:01.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:00 smithi067 ceph-mon[31232]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:35:01.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:00 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3469924808' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T06:35:01.179 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi067.front.sepia.ceph.com... 2024-09-21T06:35:01.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:35:01.179 DEBUG:teuthology.orchestra.run.smithi067:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-21T06:35:01.205 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi067.stderr:ceph-fuse[75179]: fuse finished with error 0 and tester_r 0 2024-09-21T06:35:01.206 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:35:01.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:00 smithi203 ceph-mon[33735]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:35:01.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:00 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3469924808' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T06:35:01.281 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi067.front.sepia.ceph.com 2024-09-21T06:35:01.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:35:01.282 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-21T06:35:01.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:00 smithi100 ceph-mon[33401]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:35:01.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:00 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3469924808' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T06:35:01.311 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:35:01.609 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:02.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.093+0000 7fc19a313640 1 -- 172.21.15.67:0/1569831301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc194109e10 msgr2=0x7fc194110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:02.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.093+0000 7fc19a313640 1 --2- 172.21.15.67:0/1569831301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc194109e10 0x7fc194110640 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009ca0 tx=0x7fc18c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:35:02.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.093+0000 7fc19a313640 1 -- 172.21.15.67:0/1569831301 shutdown_connections 2024-09-21T06:35:02.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.093+0000 7fc19a313640 1 --2- 172.21.15.67:0/1569831301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc194109e10 0x7fc194110640 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:02.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.093+0000 7fc19a313640 1 --2- 172.21.15.67:0/1569831301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1941094c0 0x7fc1941098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:02.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.093+0000 7fc19a313640 1 --2- 172.21.15.67:0/1569831301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc194103b80 0x7fc194103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:02.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.093+0000 7fc19a313640 1 -- 172.21.15.67:0/1569831301 >> 172.21.15.67:0/1569831301 conn(0x7fc1940ff5d0 msgr2=0x7fc194101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:02.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.093+0000 7fc19a313640 1 -- 172.21.15.67:0/1569831301 shutdown_connections 2024-09-21T06:35:02.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.093+0000 7fc19a313640 1 -- 172.21.15.67:0/1569831301 wait complete. 2024-09-21T06:35:02.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.094+0000 7fc19a313640 1 Processor -- start 2024-09-21T06:35:02.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.094+0000 7fc19a313640 1 -- start start 2024-09-21T06:35:02.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.094+0000 7fc19a313640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc194103b80 0x7fc19419a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:02.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.094+0000 7fc19a313640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1941094c0 0x7fc19419a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:02.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.094+0000 7fc19a313640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc194109e10 0x7fc19419fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:02.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.094+0000 7fc19a313640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941139d0 con 0x7fc194109e10 2024-09-21T06:35:02.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.094+0000 7fc19a313640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc194113850 con 0x7fc194103b80 2024-09-21T06:35:02.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.094+0000 7fc19a313640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc194113b50 con 0x7fc1941094c0 2024-09-21T06:35:02.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.094+0000 7fc1988b0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc194109e10 0x7fc19419fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:02.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.094+0000 7fc1988b0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc194109e10 0x7fc19419fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56150/0 (socket says 172.21.15.67:56150) 2024-09-21T06:35:02.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.094+0000 7fc1988b0640 1 -- 172.21.15.67:0/394074616 learned_addr learned my addr 172.21.15.67:0/394074616 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:02.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.094+0000 7fc193fff640 1 --2- 172.21.15.67:0/394074616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc194103b80 0x7fc19419a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:02.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.094+0000 7fc1937fe640 1 --2- 172.21.15.67:0/394074616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1941094c0 0x7fc19419a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:02.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.095+0000 7fc1988b0640 1 -- 172.21.15.67:0/394074616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1941094c0 msgr2=0x7fc19419a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:02.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.095+0000 7fc1988b0640 1 --2- 172.21.15.67:0/394074616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1941094c0 0x7fc19419a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:02.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.095+0000 7fc1988b0640 1 -- 172.21.15.67:0/394074616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc194103b80 msgr2=0x7fc19419a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:02.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.095+0000 7fc1988b0640 1 --2- 172.21.15.67:0/394074616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc194103b80 0x7fc19419a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:02.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.095+0000 7fc1988b0640 1 -- 172.21.15.67:0/394074616 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1941a0310 con 0x7fc194109e10 2024-09-21T06:35:02.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.095+0000 7fc1988b0640 1 --2- 172.21.15.67:0/394074616 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc194109e10 0x7fc19419fbe0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc18c005cc0 tx=0x7fc18c032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:02.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.095+0000 7fc1917fa640 1 -- 172.21.15.67:0/394074616 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c032e00 con 0x7fc194109e10 2024-09-21T06:35:02.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.095+0000 7fc1917fa640 1 -- 172.21.15.67:0/394074616 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc18c0044d0 con 0x7fc194109e10 2024-09-21T06:35:02.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.095+0000 7fc19a313640 1 -- 172.21.15.67:0/394074616 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1941a05a0 con 0x7fc194109e10 2024-09-21T06:35:02.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.095+0000 7fc1917fa640 1 -- 172.21.15.67:0/394074616 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c030b70 con 0x7fc194109e10 2024-09-21T06:35:02.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.095+0000 7fc19a313640 1 -- 172.21.15.67:0/394074616 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1941a6d60 con 0x7fc194109e10 2024-09-21T06:35:02.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.096+0000 7fc19a313640 1 -- 172.21.15.67:0/394074616 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc194103f90 con 0x7fc194109e10 2024-09-21T06:35:02.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.098+0000 7fc1917fa640 1 -- 172.21.15.67:0/394074616 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fc18c030d10 con 0x7fc194109e10 2024-09-21T06:35:02.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.099+0000 7fc1917fa640 1 --2- 172.21.15.67:0/394074616 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc17c0b6510 0x7fc17c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:02.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.099+0000 7fc193fff640 1 --2- 172.21.15.67:0/394074616 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc17c0b6510 0x7fc17c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:02.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.099+0000 7fc1917fa640 1 -- 172.21.15.67:0/394074616 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7fc18c1088e0 con 0x7fc194109e10 2024-09-21T06:35:02.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.099+0000 7fc193fff640 1 --2- 172.21.15.67:0/394074616 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc17c0b6510 0x7fc17c0b8a20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc18400bbe0 tx=0x7fc18400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:02.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.102+0000 7fc1917fa640 1 -- 172.21.15.67:0/394074616 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc18c0cef30 con 0x7fc194109e10 2024-09-21T06:35:02.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.289+0000 7fc19a313640 1 -- 172.21.15.67:0/394074616 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc19402e640 con 0x7fc194109e10 2024-09-21T06:35:02.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.291+0000 7fc1917fa640 1 -- 172.21.15.67:0/394074616 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+24238 (secure 0 0 0) 0x7fc18c0d3310 con 0x7fc194109e10 2024-09-21T06:35:02.292 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:35:02.293 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":81,"fsid":"2a791768-77e2-11ef-baf5-efdc52797490","created":"2024-09-21T06:26:15.222283+0000","modified":"2024-09-21T06:34:42.366026+0000","last_up_change":"2024-09-21T06:33:40.886310+0000","last_in_change":"2024-09-21T06:33:17.339486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:29:50.212825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:34:29.082881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:34:31.301094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:34:37.681356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"18778364-82ed-41c0-92da-ce44cdad7358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6803","nonce":2310759946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6805","nonce":2310759946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6809","nonce":2310759946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6807","nonce":2310759946}]},"public_addr":"172.21.15.67:6803/2310759946","cluster_addr":"172.21.15.67:6805/2310759946","heartbeat_back_addr":"172.21.15.67:6809/2310759946","heartbeat_front_addr":"172.21.15.67:6807/2310759946","state":["exists","up"]},{"osd":1,"uuid":"84e8d0c2-1c45-4673-ad49-f61df667d7f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6801","nonce":161807364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6803","nonce":161807364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6807","nonce":161807364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6805","nonce":161807364}]},"public_addr":"172.21.15.100:6801/161807364","cluster_addr":"172.21.15.100:6803/161807364","heartbeat_back_addr":"172.21.15.100:6807/161807364","heartbeat_front_addr":"172.21.15.100:6805/161807364","state":["exists","up"]},{"osd":2,"uuid":"e9a4d919-a779-4a1b-9b5b-d03a65f28ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6801","nonce":2754787169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6803","nonce":2754787169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6807","nonce":2754787169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6805","nonce":2754787169}]},"public_addr":"172.21.15.203:6801/2754787169","cluster_addr":"172.21.15.203:6803/2754787169","heartbeat_back_addr":"172.21.15.203:6807/2754787169","heartbeat_front_addr":"172.21.15.203:6805/2754787169","state":["exists","up"]},{"osd":3,"uuid":"3f7d9abd-587c-47ad-bf43-11f7dcdf71c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6811","nonce":4007517951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6813","nonce":4007517951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6817","nonce":4007517951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6815","nonce":4007517951}]},"public_addr":"172.21.15.67:6811/4007517951","cluster_addr":"172.21.15.67:6813/4007517951","heartbeat_back_addr":"172.21.15.67:6817/4007517951","heartbeat_front_addr":"172.21.15.67:6815/4007517951","state":["exists","up"]},{"osd":4,"uuid":"94f3a075-6d8a-4907-8418-a8513b35779a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6809","nonce":3773646501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6811","nonce":3773646501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6815","nonce":3773646501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6813","nonce":3773646501}]},"public_addr":"172.21.15.100:6809/3773646501","cluster_addr":"172.21.15.100:6811/3773646501","heartbeat_back_addr":"172.21.15.100:6815/3773646501","heartbeat_front_addr":"172.21.15.100:6813/3773646501","state":["exists","up"]},{"osd":5,"uuid":"5d842edf-06ad-49ae-ac71-863e5657270c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6809","nonce":2096171166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6811","nonce":2096171166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6815","nonce":2096171166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6813","nonce":2096171166}]},"public_addr":"172.21.15.203:6809/2096171166","cluster_addr":"172.21.15.203:6811/2096171166","heartbeat_back_addr":"172.21.15.203:6815/2096171166","heartbeat_front_addr":"172.21.15.203:6813/2096171166","state":["exists","up"]},{"osd":6,"uuid":"85b9beed-c93d-48ba-bc7a-c4cd816fbdff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6819","nonce":3229936280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6821","nonce":3229936280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6825","nonce":3229936280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6823","nonce":3229936280}]},"public_addr":"172.21.15.67:6819/3229936280","cluster_addr":"172.21.15.67:6821/3229936280","heartbeat_back_addr":"172.21.15.67:6825/3229936280","heartbeat_front_addr":"172.21.15.67:6823/3229936280","state":["exists","up"]},{"osd":7,"uuid":"86de30c2-b8ee-4c9d-9c17-8a56517934ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6817","nonce":1622917909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6819","nonce":1622917909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6823","nonce":1622917909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6821","nonce":1622917909}]},"public_addr":"172.21.15.100:6817/1622917909","cluster_addr":"172.21.15.100:6819/1622917909","heartbeat_back_addr":"172.21.15.100:6823/1622917909","heartbeat_front_addr":"172.21.15.100:6821/1622917909","state":["exists","up"]},{"osd":8,"uuid":"0e96cdad-a135-4115-9e2d-9a28306700e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6817","nonce":3452445943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6819","nonce":3452445943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6823","nonce":3452445943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6821","nonce":3452445943}]},"public_addr":"172.21.15.203:6817/3452445943","cluster_addr":"172.21.15.203:6819/3452445943","heartbeat_back_addr":"172.21.15.203:6823/3452445943","heartbeat_front_addr":"172.21.15.203:6821/3452445943","state":["exists","up"]},{"osd":9,"uuid":"95704ed6-6286-4094-9c6d-23919c7005ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6827","nonce":2998307533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6829","nonce":2998307533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6833","nonce":2998307533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6831","nonce":2998307533}]},"public_addr":"172.21.15.67:6827/2998307533","cluster_addr":"172.21.15.67:6829/2998307533","heartbeat_back_addr":"172.21.15.67:6833/2998307533","heartbeat_front_addr":"172.21.15.67:6831/2998307533","state":["exists","up"]},{"osd":10,"uuid":"bd2a7d33-b935-46c8-95c3-d76f28242ffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6825","nonce":2819057142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6827","nonce":2819057142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6831","nonce":2819057142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6829","nonce":2819057142}]},"public_addr":"172.21.15.100:6825/2819057142","cluster_addr":"172.21.15.100:6827/2819057142","heartbeat_back_addr":"172.21.15.100:6831/2819057142","heartbeat_front_addr":"172.21.15.100:6829/2819057142","state":["exists","up"]},{"osd":11,"uuid":"ea591507-10e2-4c08-8d29-d38ba9be96ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6825","nonce":566502600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6827","nonce":566502600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6831","nonce":566502600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6829","nonce":566502600}]},"public_addr":"172.21.15.203:6825/566502600","cluster_addr":"172.21.15.203:6827/566502600","heartbeat_back_addr":"172.21.15.203:6831/566502600","heartbeat_front_addr":"172.21.15.203:6829/566502600","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:28:58.945733+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:29:23.597118+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:29:48.303563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:30:14.139139+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:30:39.021508+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:04.981241+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:30.853841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:56.452967+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:32:22.239036+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:32:47.998803+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:33:13.850759+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:33:39.282150+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6800/1181476774":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/3100321138":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/2309364298":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/3458717076":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/4110206171":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6801/1181476774":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/2910038982":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6801/2622729103":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6800/2622729103":"2024-09-22T06:26:37.041377+0000","172.21.15.67:0/3346784528":"2024-09-22T06:26:37.041377+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:35:02.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.293+0000 7fc19a313640 1 -- 172.21.15.67:0/394074616 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc17c0b6510 msgr2=0x7fc17c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:02.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.293+0000 7fc19a313640 1 --2- 172.21.15.67:0/394074616 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc17c0b6510 0x7fc17c0b8a20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc18400bbe0 tx=0x7fc18400b390 comp rx=0 tx=0).stop 2024-09-21T06:35:02.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.293+0000 7fc19a313640 1 -- 172.21.15.67:0/394074616 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc194109e10 msgr2=0x7fc19419fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:02.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.293+0000 7fc19a313640 1 --2- 172.21.15.67:0/394074616 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc194109e10 0x7fc19419fbe0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc18c005cc0 tx=0x7fc18c032a10 comp rx=0 tx=0).stop 2024-09-21T06:35:02.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.294+0000 7fc19a313640 1 -- 172.21.15.67:0/394074616 shutdown_connections 2024-09-21T06:35:02.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.294+0000 7fc19a313640 1 --2- 172.21.15.67:0/394074616 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc17c0b6510 0x7fc17c0b8a20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:02.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.294+0000 7fc19a313640 1 --2- 172.21.15.67:0/394074616 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc194109e10 0x7fc19419fbe0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:02.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.294+0000 7fc19a313640 1 --2- 172.21.15.67:0/394074616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1941094c0 0x7fc19419a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:02.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.294+0000 7fc19a313640 1 --2- 172.21.15.67:0/394074616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc194103b80 0x7fc19419a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:02.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.294+0000 7fc19a313640 1 -- 172.21.15.67:0/394074616 >> 172.21.15.67:0/394074616 conn(0x7fc1940ff5d0 msgr2=0x7fc1941019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:02.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.294+0000 7fc19a313640 1 -- 172.21.15.67:0/394074616 shutdown_connections 2024-09-21T06:35:02.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:02.294+0000 7fc19a313640 1 -- 172.21.15.67:0/394074616 wait complete. 2024-09-21T06:35:03.104 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:35:03.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:02 smithi067 ceph-mon[31232]: pgmap v310: 129 pgs: 129 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T06:35:03.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:02 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/394074616' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:35:03.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:02 smithi100 ceph-mon[33401]: pgmap v310: 129 pgs: 129 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T06:35:03.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:02 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/394074616' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:35:03.364 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:03.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:02 smithi203 ceph-mon[33735]: pgmap v310: 129 pgs: 129 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T06:35:03.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:02 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/394074616' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:35:03.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.871+0000 7fee73eaf640 1 -- 172.21.15.67:0/443513478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee6c108cf0 msgr2=0x7fee6c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:03.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.871+0000 7fee73eaf640 1 --2- 172.21.15.67:0/443513478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee6c108cf0 0x7fee6c10f520 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fee64009ca0 tx=0x7fee6402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:35:03.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.871+0000 7fee73eaf640 1 -- 172.21.15.67:0/443513478 shutdown_connections 2024-09-21T06:35:03.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.871+0000 7fee73eaf640 1 --2- 172.21.15.67:0/443513478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee6c1083a0 0x7fee6c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:03.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.871+0000 7fee73eaf640 1 --2- 172.21.15.67:0/443513478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee6c108cf0 0x7fee6c10f520 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:03.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.871+0000 7fee73eaf640 1 --2- 172.21.15.67:0/443513478 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee6c102a60 0x7fee6c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:03.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.871+0000 7fee73eaf640 1 -- 172.21.15.67:0/443513478 >> 172.21.15.67:0/443513478 conn(0x7fee6c079730 msgr2=0x7fee6c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:03.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.871+0000 7fee73eaf640 1 -- 172.21.15.67:0/443513478 shutdown_connections 2024-09-21T06:35:03.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.871+0000 7fee73eaf640 1 -- 172.21.15.67:0/443513478 wait complete. 2024-09-21T06:35:03.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.872+0000 7fee73eaf640 1 Processor -- start 2024-09-21T06:35:03.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.872+0000 7fee73eaf640 1 -- start start 2024-09-21T06:35:03.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.872+0000 7fee73eaf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee6c102a60 0x7fee6c19b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:03.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.872+0000 7fee73eaf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee6c1083a0 0x7fee6c19b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:03.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.872+0000 7fee73eaf640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee6c108cf0 0x7fee6c19bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:03.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.872+0000 7fee73eaf640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c112860 con 0x7fee6c108cf0 2024-09-21T06:35:03.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.872+0000 7fee73eaf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c1126e0 con 0x7fee6c102a60 2024-09-21T06:35:03.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.872+0000 7fee73eaf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c1129e0 con 0x7fee6c1083a0 2024-09-21T06:35:03.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.873+0000 7fee7244c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee6c108cf0 0x7fee6c19bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:03.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.873+0000 7fee7244c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee6c108cf0 0x7fee6c19bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56166/0 (socket says 172.21.15.67:56166) 2024-09-21T06:35:03.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.873+0000 7fee7244c640 1 -- 172.21.15.67:0/1950769537 learned_addr learned my addr 172.21.15.67:0/1950769537 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:03.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.873+0000 7fee71c4b640 1 --2- 172.21.15.67:0/1950769537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee6c102a60 0x7fee6c19b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:03.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.873+0000 7fee7144a640 1 --2- 172.21.15.67:0/1950769537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee6c1083a0 0x7fee6c19b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:03.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.873+0000 7fee7244c640 1 -- 172.21.15.67:0/1950769537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee6c1083a0 msgr2=0x7fee6c19b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:03.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.873+0000 7fee7244c640 1 --2- 172.21.15.67:0/1950769537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee6c1083a0 0x7fee6c19b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:03.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.873+0000 7fee7244c640 1 -- 172.21.15.67:0/1950769537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee6c102a60 msgr2=0x7fee6c19b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:03.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.873+0000 7fee7244c640 1 --2- 172.21.15.67:0/1950769537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee6c102a60 0x7fee6c19b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:03.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.873+0000 7fee7244c640 1 -- 172.21.15.67:0/1950769537 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee6c1a6340 con 0x7fee6c108cf0 2024-09-21T06:35:03.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.873+0000 7fee71c4b640 1 --2- 172.21.15.67:0/1950769537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee6c102a60 0x7fee6c19b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:03.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.873+0000 7fee7244c640 1 --2- 172.21.15.67:0/1950769537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee6c108cf0 0x7fee6c19bb90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fee64005cc0 tx=0x7fee64032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:03.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.873+0000 7fee62ffd640 1 -- 172.21.15.67:0/1950769537 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64032e00 con 0x7fee6c108cf0 2024-09-21T06:35:03.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.873+0000 7fee62ffd640 1 -- 172.21.15.67:0/1950769537 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee640044d0 con 0x7fee6c108cf0 2024-09-21T06:35:03.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.873+0000 7fee73eaf640 1 -- 172.21.15.67:0/1950769537 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee6c1a65d0 con 0x7fee6c108cf0 2024-09-21T06:35:03.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.873+0000 7fee62ffd640 1 -- 172.21.15.67:0/1950769537 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64030b70 con 0x7fee6c108cf0 2024-09-21T06:35:03.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.873+0000 7fee73eaf640 1 -- 172.21.15.67:0/1950769537 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee6c1a6b10 con 0x7fee6c108cf0 2024-09-21T06:35:03.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.874+0000 7fee73eaf640 1 -- 172.21.15.67:0/1950769537 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee6c102e70 con 0x7fee6c108cf0 2024-09-21T06:35:03.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.876+0000 7fee62ffd640 1 -- 172.21.15.67:0/1950769537 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fee64030d10 con 0x7fee6c108cf0 2024-09-21T06:35:03.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.877+0000 7fee62ffd640 1 --2- 172.21.15.67:0/1950769537 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fee580b6510 0x7fee580b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:03.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.877+0000 7fee71c4b640 1 --2- 172.21.15.67:0/1950769537 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fee580b6510 0x7fee580b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:03.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.877+0000 7fee62ffd640 1 -- 172.21.15.67:0/1950769537 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7fee641089e0 con 0x7fee6c108cf0 2024-09-21T06:35:03.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.878+0000 7fee71c4b640 1 --2- 172.21.15.67:0/1950769537 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fee580b6510 0x7fee580b8a20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fee68005fd0 tx=0x7fee68005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:03.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:03.881+0000 7fee62ffd640 1 -- 172.21.15.67:0/1950769537 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fee640cf030 con 0x7fee6c108cf0 2024-09-21T06:35:04.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:03 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]} : dispatch 2024-09-21T06:35:04.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:03 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]} : dispatch 2024-09-21T06:35:04.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:03 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]} : dispatch 2024-09-21T06:35:04.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:03 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]} : dispatch 2024-09-21T06:35:04.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:03 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]} : dispatch 2024-09-21T06:35:04.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:03 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]} : dispatch 2024-09-21T06:35:04.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:03 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]} : dispatch 2024-09-21T06:35:04.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:03 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]} : dispatch 2024-09-21T06:35:04.043 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:03 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]} : dispatch 2024-09-21T06:35:04.043 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:03 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 10]} : dispatch 2024-09-21T06:35:04.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:04.076+0000 7fee73eaf640 1 -- 172.21.15.67:0/1950769537 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fee6c02e640 con 0x7fee6c108cf0 2024-09-21T06:35:04.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:04.078+0000 7fee62ffd640 1 -- 172.21.15.67:0/1950769537 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+24709 (secure 0 0 0) 0x7fee640d3410 con 0x7fee6c108cf0 2024-09-21T06:35:04.079 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:35:04.079 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":82,"fsid":"2a791768-77e2-11ef-baf5-efdc52797490","created":"2024-09-21T06:26:15.222283+0000","modified":"2024-09-21T06:35:03.982468+0000","last_up_change":"2024-09-21T06:33:40.886310+0000","last_in_change":"2024-09-21T06:33:17.339486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:29:50.212825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:34:29.082881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:34:31.301094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:34:37.681356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"18778364-82ed-41c0-92da-ce44cdad7358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6803","nonce":2310759946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6805","nonce":2310759946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6809","nonce":2310759946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6807","nonce":2310759946}]},"public_addr":"172.21.15.67:6803/2310759946","cluster_addr":"172.21.15.67:6805/2310759946","heartbeat_back_addr":"172.21.15.67:6809/2310759946","heartbeat_front_addr":"172.21.15.67:6807/2310759946","state":["exists","up"]},{"osd":1,"uuid":"84e8d0c2-1c45-4673-ad49-f61df667d7f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6801","nonce":161807364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6803","nonce":161807364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6807","nonce":161807364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6805","nonce":161807364}]},"public_addr":"172.21.15.100:6801/161807364","cluster_addr":"172.21.15.100:6803/161807364","heartbeat_back_addr":"172.21.15.100:6807/161807364","heartbeat_front_addr":"172.21.15.100:6805/161807364","state":["exists","up"]},{"osd":2,"uuid":"e9a4d919-a779-4a1b-9b5b-d03a65f28ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6801","nonce":2754787169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6803","nonce":2754787169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6807","nonce":2754787169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6805","nonce":2754787169}]},"public_addr":"172.21.15.203:6801/2754787169","cluster_addr":"172.21.15.203:6803/2754787169","heartbeat_back_addr":"172.21.15.203:6807/2754787169","heartbeat_front_addr":"172.21.15.203:6805/2754787169","state":["exists","up"]},{"osd":3,"uuid":"3f7d9abd-587c-47ad-bf43-11f7dcdf71c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6811","nonce":4007517951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6813","nonce":4007517951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6817","nonce":4007517951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6815","nonce":4007517951}]},"public_addr":"172.21.15.67:6811/4007517951","cluster_addr":"172.21.15.67:6813/4007517951","heartbeat_back_addr":"172.21.15.67:6817/4007517951","heartbeat_front_addr":"172.21.15.67:6815/4007517951","state":["exists","up"]},{"osd":4,"uuid":"94f3a075-6d8a-4907-8418-a8513b35779a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6809","nonce":3773646501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6811","nonce":3773646501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6815","nonce":3773646501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6813","nonce":3773646501}]},"public_addr":"172.21.15.100:6809/3773646501","cluster_addr":"172.21.15.100:6811/3773646501","heartbeat_back_addr":"172.21.15.100:6815/3773646501","heartbeat_front_addr":"172.21.15.100:6813/3773646501","state":["exists","up"]},{"osd":5,"uuid":"5d842edf-06ad-49ae-ac71-863e5657270c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6809","nonce":2096171166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6811","nonce":2096171166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6815","nonce":2096171166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6813","nonce":2096171166}]},"public_addr":"172.21.15.203:6809/2096171166","cluster_addr":"172.21.15.203:6811/2096171166","heartbeat_back_addr":"172.21.15.203:6815/2096171166","heartbeat_front_addr":"172.21.15.203:6813/2096171166","state":["exists","up"]},{"osd":6,"uuid":"85b9beed-c93d-48ba-bc7a-c4cd816fbdff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6819","nonce":3229936280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6821","nonce":3229936280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6825","nonce":3229936280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6823","nonce":3229936280}]},"public_addr":"172.21.15.67:6819/3229936280","cluster_addr":"172.21.15.67:6821/3229936280","heartbeat_back_addr":"172.21.15.67:6825/3229936280","heartbeat_front_addr":"172.21.15.67:6823/3229936280","state":["exists","up"]},{"osd":7,"uuid":"86de30c2-b8ee-4c9d-9c17-8a56517934ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6817","nonce":1622917909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6819","nonce":1622917909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6823","nonce":1622917909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6821","nonce":1622917909}]},"public_addr":"172.21.15.100:6817/1622917909","cluster_addr":"172.21.15.100:6819/1622917909","heartbeat_back_addr":"172.21.15.100:6823/1622917909","heartbeat_front_addr":"172.21.15.100:6821/1622917909","state":["exists","up"]},{"osd":8,"uuid":"0e96cdad-a135-4115-9e2d-9a28306700e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6817","nonce":3452445943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6819","nonce":3452445943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6823","nonce":3452445943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6821","nonce":3452445943}]},"public_addr":"172.21.15.203:6817/3452445943","cluster_addr":"172.21.15.203:6819/3452445943","heartbeat_back_addr":"172.21.15.203:6823/3452445943","heartbeat_front_addr":"172.21.15.203:6821/3452445943","state":["exists","up"]},{"osd":9,"uuid":"95704ed6-6286-4094-9c6d-23919c7005ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6827","nonce":2998307533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6829","nonce":2998307533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6833","nonce":2998307533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6831","nonce":2998307533}]},"public_addr":"172.21.15.67:6827/2998307533","cluster_addr":"172.21.15.67:6829/2998307533","heartbeat_back_addr":"172.21.15.67:6833/2998307533","heartbeat_front_addr":"172.21.15.67:6831/2998307533","state":["exists","up"]},{"osd":10,"uuid":"bd2a7d33-b935-46c8-95c3-d76f28242ffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6825","nonce":2819057142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6827","nonce":2819057142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6831","nonce":2819057142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6829","nonce":2819057142}]},"public_addr":"172.21.15.100:6825/2819057142","cluster_addr":"172.21.15.100:6827/2819057142","heartbeat_back_addr":"172.21.15.100:6831/2819057142","heartbeat_front_addr":"172.21.15.100:6829/2819057142","state":["exists","up"]},{"osd":11,"uuid":"ea591507-10e2-4c08-8d29-d38ba9be96ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6825","nonce":566502600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6827","nonce":566502600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6831","nonce":566502600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6829","nonce":566502600}]},"public_addr":"172.21.15.203:6825/566502600","cluster_addr":"172.21.15.203:6827/566502600","heartbeat_back_addr":"172.21.15.203:6831/566502600","heartbeat_front_addr":"172.21.15.203:6829/566502600","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:28:58.945733+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:29:23.597118+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:29:48.303563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:30:14.139139+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:30:39.021508+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:04.981241+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:30.853841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:56.452967+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:32:22.239036+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:32:47.998803+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:33:13.850759+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:33:39.282150+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6800/1181476774":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/3100321138":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/2309364298":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/3458717076":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/4110206171":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6801/1181476774":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/2910038982":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6801/2622729103":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6800/2622729103":"2024-09-22T06:26:37.041377+0000","172.21.15.67:0/3346784528":"2024-09-22T06:26:37.041377+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:35:04.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:04.080+0000 7fee73eaf640 1 -- 172.21.15.67:0/1950769537 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fee580b6510 msgr2=0x7fee580b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:04.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:04.080+0000 7fee73eaf640 1 --2- 172.21.15.67:0/1950769537 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fee580b6510 0x7fee580b8a20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fee68005fd0 tx=0x7fee68005850 comp rx=0 tx=0).stop 2024-09-21T06:35:04.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:04.080+0000 7fee73eaf640 1 -- 172.21.15.67:0/1950769537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee6c108cf0 msgr2=0x7fee6c19bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:04.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:04.080+0000 7fee73eaf640 1 --2- 172.21.15.67:0/1950769537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee6c108cf0 0x7fee6c19bb90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fee64005cc0 tx=0x7fee64032a10 comp rx=0 tx=0).stop 2024-09-21T06:35:04.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:04.080+0000 7fee73eaf640 1 -- 172.21.15.67:0/1950769537 shutdown_connections 2024-09-21T06:35:04.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:04.080+0000 7fee73eaf640 1 --2- 172.21.15.67:0/1950769537 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fee580b6510 0x7fee580b8a20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:04.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:04.080+0000 7fee73eaf640 1 --2- 172.21.15.67:0/1950769537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee6c108cf0 0x7fee6c19bb90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:04.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:04.080+0000 7fee73eaf640 1 --2- 172.21.15.67:0/1950769537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee6c1083a0 0x7fee6c19b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:04.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:04.080+0000 7fee73eaf640 1 --2- 172.21.15.67:0/1950769537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee6c102a60 0x7fee6c19b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:04.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:04.080+0000 7fee73eaf640 1 -- 172.21.15.67:0/1950769537 >> 172.21.15.67:0/1950769537 conn(0x7fee6c079730 msgr2=0x7fee6c1043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:04.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:04.080+0000 7fee73eaf640 1 -- 172.21.15.67:0/1950769537 shutdown_connections 2024-09-21T06:35:04.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:04.080+0000 7fee73eaf640 1 -- 172.21.15.67:0/1950769537 wait complete. 2024-09-21T06:35:04.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:03 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]} : dispatch 2024-09-21T06:35:04.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:03 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]} : dispatch 2024-09-21T06:35:04.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:03 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]} : dispatch 2024-09-21T06:35:04.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:03 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]} : dispatch 2024-09-21T06:35:04.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:03 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]} : dispatch 2024-09-21T06:35:04.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:03 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]} : dispatch 2024-09-21T06:35:04.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:03 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]} : dispatch 2024-09-21T06:35:04.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:03 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]} : dispatch 2024-09-21T06:35:04.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:03 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]} : dispatch 2024-09-21T06:35:04.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:03 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 10]} : dispatch 2024-09-21T06:35:04.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:03 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]} : dispatch 2024-09-21T06:35:04.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:03 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]} : dispatch 2024-09-21T06:35:04.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:03 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]} : dispatch 2024-09-21T06:35:04.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:03 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]} : dispatch 2024-09-21T06:35:04.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:03 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]} : dispatch 2024-09-21T06:35:04.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:03 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]} : dispatch 2024-09-21T06:35:04.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:03 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]} : dispatch 2024-09-21T06:35:04.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:03 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]} : dispatch 2024-09-21T06:35:04.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:03 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]} : dispatch 2024-09-21T06:35:04.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:03 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 10]} : dispatch 2024-09-21T06:35:04.788 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs set cephfs standby_count_wanted 0 2024-09-21T06:35:05.048 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:05.169 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:05 smithi067 ceph-mon[31232]: pgmap v311: 129 pgs: 129 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T06:35:05.169 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:05 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-09-21T06:35:05.169 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:05 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-21T06:35:05.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:05 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-09-21T06:35:05.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:05 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-09-21T06:35:05.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:05 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-09-21T06:35:05.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:05 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]': finished 2024-09-21T06:35:05.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:05 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-09-21T06:35:05.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:05 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-09-21T06:35:05.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:05 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-09-21T06:35:05.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:05 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 10]}]': finished 2024-09-21T06:35:05.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:05 smithi067 ceph-mon[31232]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T06:35:05.171 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:05 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1950769537' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:35:05.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:05 smithi100 ceph-mon[33401]: pgmap v311: 129 pgs: 129 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T06:35:05.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:05 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-09-21T06:35:05.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:05 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-21T06:35:05.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:05 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-09-21T06:35:05.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:05 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-09-21T06:35:05.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:05 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-09-21T06:35:05.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:05 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]': finished 2024-09-21T06:35:05.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:05 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-09-21T06:35:05.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:05 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-09-21T06:35:05.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:05 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-09-21T06:35:05.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:05 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 10]}]': finished 2024-09-21T06:35:05.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:05 smithi100 ceph-mon[33401]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T06:35:05.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:05 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1950769537' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:35:05.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:05 smithi203 ceph-mon[33735]: pgmap v311: 129 pgs: 129 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T06:35:05.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:05 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-09-21T06:35:05.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:05 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-21T06:35:05.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:05 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-09-21T06:35:05.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:05 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-09-21T06:35:05.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:05 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-09-21T06:35:05.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:05 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]': finished 2024-09-21T06:35:05.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:05 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-09-21T06:35:05.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:05 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-09-21T06:35:05.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:05 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-09-21T06:35:05.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:05 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 10]}]': finished 2024-09-21T06:35:05.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:05 smithi203 ceph-mon[33735]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T06:35:05.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:05 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1950769537' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:35:05.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.566+0000 7f87bd1fd640 1 -- 172.21.15.67:0/2804493249 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87b81049b0 msgr2=0x7f87b8104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:05.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.566+0000 7f87bd1fd640 1 --2- 172.21.15.67:0/2804493249 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87b81049b0 0x7f87b8104dc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f87b0009900 tx=0x7f87b002a410 comp rx=0 tx=0).stop 2024-09-21T06:35:05.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.566+0000 7f87bd1fd640 1 -- 172.21.15.67:0/2804493249 shutdown_connections 2024-09-21T06:35:05.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.566+0000 7f87bd1fd640 1 --2- 172.21.15.67:0/2804493249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87b810de10 0x7f87b8110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:05.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.566+0000 7f87bd1fd640 1 --2- 172.21.15.67:0/2804493249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87b8105300 0x7f87b810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:05.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.566+0000 7f87bd1fd640 1 --2- 172.21.15.67:0/2804493249 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87b81049b0 0x7f87b8104dc0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:05.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.566+0000 7f87bd1fd640 1 -- 172.21.15.67:0/2804493249 >> 172.21.15.67:0/2804493249 conn(0x7f87b8079730 msgr2=0x7f87b8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:05.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.566+0000 7f87bd1fd640 1 -- 172.21.15.67:0/2804493249 shutdown_connections 2024-09-21T06:35:05.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.566+0000 7f87bd1fd640 1 -- 172.21.15.67:0/2804493249 wait complete. 2024-09-21T06:35:05.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.567+0000 7f87bd1fd640 1 Processor -- start 2024-09-21T06:35:05.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.567+0000 7f87bd1fd640 1 -- start start 2024-09-21T06:35:05.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.568+0000 7f87bd1fd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87b81049b0 0x7f87b819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:05.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.568+0000 7f87bd1fd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87b8105300 0x7f87b819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:05.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.568+0000 7f87bd1fd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87b810de10 0x7f87b819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:05.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.568+0000 7f87bd1fd640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f87b8113940 con 0x7f87b81049b0 2024-09-21T06:35:05.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.568+0000 7f87bd1fd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f87b81137c0 con 0x7f87b810de10 2024-09-21T06:35:05.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.568+0000 7f87bd1fd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f87b8113ac0 con 0x7f87b8105300 2024-09-21T06:35:05.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.568+0000 7f87b6d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87b81049b0 0x7f87b819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:05.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.568+0000 7f87b6d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87b81049b0 0x7f87b819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56182/0 (socket says 172.21.15.67:56182) 2024-09-21T06:35:05.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.568+0000 7f87b6d9d640 1 -- 172.21.15.67:0/2012571598 learned_addr learned my addr 172.21.15.67:0/2012571598 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:05.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.568+0000 7f87b659c640 1 --2- 172.21.15.67:0/2012571598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87b8105300 0x7f87b819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:05.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.568+0000 7f87b759e640 1 --2- 172.21.15.67:0/2012571598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87b810de10 0x7f87b819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:05.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.568+0000 7f87b6d9d640 1 -- 172.21.15.67:0/2012571598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87b8105300 msgr2=0x7f87b819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:05.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.568+0000 7f87b6d9d640 1 --2- 172.21.15.67:0/2012571598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87b8105300 0x7f87b819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:05.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.568+0000 7f87b6d9d640 1 -- 172.21.15.67:0/2012571598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87b810de10 msgr2=0x7f87b819fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:05.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.568+0000 7f87b6d9d640 1 --2- 172.21.15.67:0/2012571598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87b810de10 0x7f87b819fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:05.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.568+0000 7f87b6d9d640 1 -- 172.21.15.67:0/2012571598 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b81a03e0 con 0x7f87b81049b0 2024-09-21T06:35:05.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.568+0000 7f87b6d9d640 1 --2- 172.21.15.67:0/2012571598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87b81049b0 0x7f87b819a470 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f87b0005940 tx=0x7f87b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:05.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.569+0000 7f87affff640 1 -- 172.21.15.67:0/2012571598 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b003f030 con 0x7f87b81049b0 2024-09-21T06:35:05.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.569+0000 7f87affff640 1 -- 172.21.15.67:0/2012571598 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87b0003c90 con 0x7f87b81049b0 2024-09-21T06:35:05.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.569+0000 7f87bd1fd640 1 -- 172.21.15.67:0/2012571598 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87b81a0670 con 0x7f87b81049b0 2024-09-21T06:35:05.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.569+0000 7f87affff640 1 -- 172.21.15.67:0/2012571598 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b0035070 con 0x7f87b81049b0 2024-09-21T06:35:05.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.569+0000 7f87bd1fd640 1 -- 172.21.15.67:0/2012571598 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87b81a6e30 con 0x7f87b81049b0 2024-09-21T06:35:05.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.570+0000 7f87bd1fd640 1 -- 172.21.15.67:0/2012571598 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87b806b560 con 0x7f87b81049b0 2024-09-21T06:35:05.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.572+0000 7f87affff640 1 -- 172.21.15.67:0/2012571598 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f87b0042030 con 0x7f87b81049b0 2024-09-21T06:35:05.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.572+0000 7f87affff640 1 --2- 172.21.15.67:0/2012571598 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f87a40b6510 0x7f87a40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:05.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.572+0000 7f87b659c640 1 --2- 172.21.15.67:0/2012571598 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f87a40b6510 0x7f87a40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:05.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.573+0000 7f87affff640 1 -- 172.21.15.67:0/2012571598 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f87b0107fc0 con 0x7f87b81049b0 2024-09-21T06:35:05.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.573+0000 7f87b659c640 1 --2- 172.21.15.67:0/2012571598 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f87a40b6510 0x7f87a40b8a20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f87b8102460 tx=0x7f87a000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:05.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.576+0000 7f87affff640 1 -- 172.21.15.67:0/2012571598 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f87b00ce4f0 con 0x7f87b81049b0 2024-09-21T06:35:05.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:05.805+0000 7f87bd1fd640 1 -- 172.21.15.67:0/2012571598 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f87b8108f10 con 0x7f87b81049b0 2024-09-21T06:35:06.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:06.004+0000 7f87affff640 1 -- 172.21.15.67:0/2012571598 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f87b00d28d0 con 0x7f87b81049b0 2024-09-21T06:35:06.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:06.007+0000 7f87bd1fd640 1 -- 172.21.15.67:0/2012571598 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f87a40b6510 msgr2=0x7f87a40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:06.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:06.007+0000 7f87bd1fd640 1 --2- 172.21.15.67:0/2012571598 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f87a40b6510 0x7f87a40b8a20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f87b8102460 tx=0x7f87a000b390 comp rx=0 tx=0).stop 2024-09-21T06:35:06.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:06.007+0000 7f87bd1fd640 1 -- 172.21.15.67:0/2012571598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87b81049b0 msgr2=0x7f87b819a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:06.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:06.007+0000 7f87bd1fd640 1 --2- 172.21.15.67:0/2012571598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87b81049b0 0x7f87b819a470 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f87b0005940 tx=0x7f87b00042f0 comp rx=0 tx=0).stop 2024-09-21T06:35:06.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:06.007+0000 7f87bd1fd640 1 -- 172.21.15.67:0/2012571598 shutdown_connections 2024-09-21T06:35:06.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:06.007+0000 7f87bd1fd640 1 --2- 172.21.15.67:0/2012571598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87b810de10 0x7f87b819fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:06.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:06.007+0000 7f87bd1fd640 1 --2- 172.21.15.67:0/2012571598 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f87a40b6510 0x7f87a40b8a20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:06.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:06.007+0000 7f87bd1fd640 1 --2- 172.21.15.67:0/2012571598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87b8105300 0x7f87b819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:06.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:06.007+0000 7f87bd1fd640 1 --2- 172.21.15.67:0/2012571598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87b81049b0 0x7f87b819a470 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:06.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:06.007+0000 7f87bd1fd640 1 -- 172.21.15.67:0/2012571598 >> 172.21.15.67:0/2012571598 conn(0x7f87b8079730 msgr2=0x7f87b8100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:06.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:06.007+0000 7f87bd1fd640 1 -- 172.21.15.67:0/2012571598 shutdown_connections 2024-09-21T06:35:06.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:06.008+0000 7f87bd1fd640 1 -- 172.21.15.67:0/2012571598 wait complete. 2024-09-21T06:35:06.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:06 smithi067 ceph-mon[31232]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T06:35:06.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:06 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2012571598' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T06:35:06.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:06 smithi100 ceph-mon[33401]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T06:35:06.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:06 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2012571598' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T06:35:06.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:06 smithi203 ceph-mon[33735]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T06:35:06.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:06 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2012571598' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T06:35:06.792 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 1, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-09-21T06:35:06.792 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs set cephfs allow_standby_replay true 2024-09-21T06:35:07.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:07 smithi067 ceph-mon[31232]: pgmap v314: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s 2024-09-21T06:35:07.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:07 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2012571598' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T06:35:07.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:07 smithi067 ceph-mon[31232]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:35:07.051 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:07.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:07 smithi100 ceph-mon[33401]: pgmap v314: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s 2024-09-21T06:35:07.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:07 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2012571598' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T06:35:07.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:07 smithi100 ceph-mon[33401]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:35:07.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:07 smithi203 ceph-mon[33735]: pgmap v314: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s 2024-09-21T06:35:07.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:07 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2012571598' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T06:35:07.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:07 smithi203 ceph-mon[33735]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:35:07.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.542+0000 7f9166331640 1 -- 172.21.15.67:0/3228567415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91601094a0 msgr2=0x7f91601098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:07.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.542+0000 7f9166331640 1 --2- 172.21.15.67:0/3228567415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91601094a0 0x7f91601098b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9154009900 tx=0x7f915402a410 comp rx=0 tx=0).stop 2024-09-21T06:35:07.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.542+0000 7f9166331640 1 -- 172.21.15.67:0/3228567415 shutdown_connections 2024-09-21T06:35:07.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.542+0000 7f9166331640 1 --2- 172.21.15.67:0/3228567415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9160109df0 0x7f9160110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.542+0000 7f9166331640 1 --2- 172.21.15.67:0/3228567415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91601094a0 0x7f91601098b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.542+0000 7f9166331640 1 --2- 172.21.15.67:0/3228567415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9160103b60 0x7f9160103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.542+0000 7f9166331640 1 -- 172.21.15.67:0/3228567415 >> 172.21.15.67:0/3228567415 conn(0x7f916007b850 msgr2=0x7f916007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:07.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.542+0000 7f9166331640 1 -- 172.21.15.67:0/3228567415 shutdown_connections 2024-09-21T06:35:07.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.543+0000 7f9166331640 1 -- 172.21.15.67:0/3228567415 wait complete. 2024-09-21T06:35:07.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.543+0000 7f9166331640 1 Processor -- start 2024-09-21T06:35:07.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.543+0000 7f9166331640 1 -- start start 2024-09-21T06:35:07.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.544+0000 7f9166331640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9160103b60 0x7f916019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:07.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.544+0000 7f9166331640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91601094a0 0x7f916019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:07.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.544+0000 7f9166331640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9160109df0 0x7f916019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:07.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.544+0000 7f9166331640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9160113960 con 0x7f9160109df0 2024-09-21T06:35:07.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.544+0000 7f9166331640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f91601137e0 con 0x7f91601094a0 2024-09-21T06:35:07.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.544+0000 7f9166331640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9160113ae0 con 0x7f9160103b60 2024-09-21T06:35:07.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.544+0000 7f91648ce640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9160109df0 0x7f916019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:07.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.544+0000 7f91648ce640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9160109df0 0x7f916019fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42546/0 (socket says 172.21.15.67:42546) 2024-09-21T06:35:07.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.544+0000 7f91648ce640 1 -- 172.21.15.67:0/3280929838 learned_addr learned my addr 172.21.15.67:0/3280929838 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:07.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.544+0000 7f915ffff640 1 --2- 172.21.15.67:0/3280929838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9160103b60 0x7f916019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:07.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.544+0000 7f915f7fe640 1 --2- 172.21.15.67:0/3280929838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91601094a0 0x7f916019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:07.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.544+0000 7f91648ce640 1 -- 172.21.15.67:0/3280929838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9160103b60 msgr2=0x7f916019a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:07.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.544+0000 7f91648ce640 1 --2- 172.21.15.67:0/3280929838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9160103b60 0x7f916019a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.544+0000 7f91648ce640 1 -- 172.21.15.67:0/3280929838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91601094a0 msgr2=0x7f916019a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:07.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.544+0000 7f91648ce640 1 --2- 172.21.15.67:0/3280929838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91601094a0 0x7f916019a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:07.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.544+0000 7f91648ce640 1 -- 172.21.15.67:0/3280929838 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91601a0380 con 0x7f9160109df0 2024-09-21T06:35:07.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.545+0000 7f91648ce640 1 --2- 172.21.15.67:0/3280929838 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9160109df0 0x7f916019fc50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f915800eaa0 tx=0x7f915800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:07.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.545+0000 7f915d7fa640 1 -- 172.21.15.67:0/3280929838 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915800cdb0 con 0x7f9160109df0 2024-09-21T06:35:07.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.545+0000 7f915d7fa640 1 -- 172.21.15.67:0/3280929838 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9158005020 con 0x7f9160109df0 2024-09-21T06:35:07.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.545+0000 7f9166331640 1 -- 172.21.15.67:0/3280929838 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91601a0670 con 0x7f9160109df0 2024-09-21T06:35:07.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.545+0000 7f915d7fa640 1 -- 172.21.15.67:0/3280929838 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91580116d0 con 0x7f9160109df0 2024-09-21T06:35:07.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.545+0000 7f9166331640 1 -- 172.21.15.67:0/3280929838 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91601a6dd0 con 0x7f9160109df0 2024-09-21T06:35:07.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.546+0000 7f9166331640 1 -- 172.21.15.67:0/3280929838 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9160103f70 con 0x7f9160109df0 2024-09-21T06:35:07.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.547+0000 7f915d7fa640 1 -- 172.21.15.67:0/3280929838 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f91580051d0 con 0x7f9160109df0 2024-09-21T06:35:07.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.548+0000 7f915d7fa640 1 --2- 172.21.15.67:0/3280929838 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f91480b6510 0x7f91480b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:07.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.548+0000 7f915ffff640 1 --2- 172.21.15.67:0/3280929838 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f91480b6510 0x7f91480b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:07.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.548+0000 7f915d7fa640 1 -- 172.21.15.67:0/3280929838 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f91580e7b30 con 0x7f9160109df0 2024-09-21T06:35:07.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.549+0000 7f915ffff640 1 --2- 172.21.15.67:0/3280929838 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f91480b6510 0x7f91480b8a20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9150009bd0 tx=0x7f9150005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:07.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.552+0000 7f915d7fa640 1 -- 172.21.15.67:0/3280929838 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9158016030 con 0x7f9160109df0 2024-09-21T06:35:07.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:07.779+0000 7f9166331640 1 -- 172.21.15.67:0/3280929838 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f916019b540 con 0x7f9160109df0 2024-09-21T06:35:08.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:08.012+0000 7f915d7fa640 1 -- 172.21.15.67:0/3280929838 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7f916019b540 con 0x7f9160109df0 2024-09-21T06:35:08.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:08.014+0000 7f9166331640 1 -- 172.21.15.67:0/3280929838 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f91480b6510 msgr2=0x7f91480b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:08.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:08.014+0000 7f9166331640 1 --2- 172.21.15.67:0/3280929838 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f91480b6510 0x7f91480b8a20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9150009bd0 tx=0x7f9150005f90 comp rx=0 tx=0).stop 2024-09-21T06:35:08.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:08.014+0000 7f9166331640 1 -- 172.21.15.67:0/3280929838 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9160109df0 msgr2=0x7f916019fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:08.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:08.014+0000 7f9166331640 1 --2- 172.21.15.67:0/3280929838 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9160109df0 0x7f916019fc50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f915800eaa0 tx=0x7f915800ef60 comp rx=0 tx=0).stop 2024-09-21T06:35:08.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:08.014+0000 7f9166331640 1 -- 172.21.15.67:0/3280929838 shutdown_connections 2024-09-21T06:35:08.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:08.014+0000 7f9166331640 1 --2- 172.21.15.67:0/3280929838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91601094a0 0x7f916019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:08.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:08.014+0000 7f9166331640 1 --2- 172.21.15.67:0/3280929838 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f91480b6510 0x7f91480b8a20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:08.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:08.014+0000 7f9166331640 1 --2- 172.21.15.67:0/3280929838 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9160109df0 0x7f916019fc50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:08.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:08.014+0000 7f9166331640 1 --2- 172.21.15.67:0/3280929838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9160103b60 0x7f916019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:08.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:08.014+0000 7f9166331640 1 -- 172.21.15.67:0/3280929838 >> 172.21.15.67:0/3280929838 conn(0x7f916007b850 msgr2=0x7f91601055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:08.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:08.014+0000 7f9166331640 1 -- 172.21.15.67:0/3280929838 shutdown_connections 2024-09-21T06:35:08.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:08.015+0000 7f9166331640 1 -- 172.21.15.67:0/3280929838 wait complete. 2024-09-21T06:35:08.041 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:08 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3280929838' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T06:35:08.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:08 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3280929838' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T06:35:08.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:08 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3280929838' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T06:35:08.820 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs set cephfs session_timeout 300 2024-09-21T06:35:09.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:09 smithi067 ceph-mon[31232]: pgmap v315: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s 2024-09-21T06:35:09.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:09 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3280929838' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T06:35:09.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:09 smithi067 ceph-mon[31232]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:35:09.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:09 smithi067 ceph-mon[31232]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-21T06:35:09.087 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:09.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:09 smithi100 ceph-mon[33401]: pgmap v315: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s 2024-09-21T06:35:09.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:09 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3280929838' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T06:35:09.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:09 smithi100 ceph-mon[33401]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:35:09.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:09 smithi100 ceph-mon[33401]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-21T06:35:09.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:09 smithi203 ceph-mon[33735]: pgmap v315: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s 2024-09-21T06:35:09.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:09 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3280929838' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T06:35:09.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:09 smithi203 ceph-mon[33735]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:35:09.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:09 smithi203 ceph-mon[33735]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-21T06:35:09.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.565+0000 7f837f69c640 1 -- 172.21.15.67:0/1139587909 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8378105300 msgr2=0x7f837810d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:09.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.565+0000 7f837f69c640 1 --2- 172.21.15.67:0/1139587909 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8378105300 0x7f837810d850 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f836c009900 tx=0x7f836c02a410 comp rx=0 tx=0).stop 2024-09-21T06:35:09.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.565+0000 7f837f69c640 1 -- 172.21.15.67:0/1139587909 shutdown_connections 2024-09-21T06:35:09.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.565+0000 7f837f69c640 1 --2- 172.21.15.67:0/1139587909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f837810de10 0x7f8378110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:09.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.565+0000 7f837f69c640 1 --2- 172.21.15.67:0/1139587909 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8378105300 0x7f837810d850 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:09.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.565+0000 7f837f69c640 1 --2- 172.21.15.67:0/1139587909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83781049b0 0x7f8378104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:09.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.565+0000 7f837f69c640 1 -- 172.21.15.67:0/1139587909 >> 172.21.15.67:0/1139587909 conn(0x7f8378079730 msgr2=0x7f8378079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:09.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.565+0000 7f837f69c640 1 -- 172.21.15.67:0/1139587909 shutdown_connections 2024-09-21T06:35:09.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.565+0000 7f837f69c640 1 -- 172.21.15.67:0/1139587909 wait complete. 2024-09-21T06:35:09.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.566+0000 7f837f69c640 1 Processor -- start 2024-09-21T06:35:09.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.566+0000 7f837f69c640 1 -- start start 2024-09-21T06:35:09.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.566+0000 7f837f69c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83781049b0 0x7f837819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:09.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.566+0000 7f837f69c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8378105300 0x7f837819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:09.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.566+0000 7f837f69c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f837810de10 0x7f837819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:09.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.566+0000 7f837f69c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8378113920 con 0x7f83781049b0 2024-09-21T06:35:09.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.566+0000 7f837f69c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f83781137a0 con 0x7f837810de10 2024-09-21T06:35:09.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.566+0000 7f837f69c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8378113aa0 con 0x7f8378105300 2024-09-21T06:35:09.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.566+0000 7f837d438640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83781049b0 0x7f837819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:09.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.566+0000 7f837d438640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83781049b0 0x7f837819a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42568/0 (socket says 172.21.15.67:42568) 2024-09-21T06:35:09.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.566+0000 7f837d438640 1 -- 172.21.15.67:0/2848937752 learned_addr learned my addr 172.21.15.67:0/2848937752 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:09.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.566+0000 7f837dc39640 1 --2- 172.21.15.67:0/2848937752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f837810de10 0x7f837819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:09.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.567+0000 7f837cc37640 1 --2- 172.21.15.67:0/2848937752 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8378105300 0x7f837819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:09.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.567+0000 7f837d438640 1 -- 172.21.15.67:0/2848937752 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8378105300 msgr2=0x7f837819a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:09.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.567+0000 7f837d438640 1 --2- 172.21.15.67:0/2848937752 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8378105300 0x7f837819a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:09.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.567+0000 7f837d438640 1 -- 172.21.15.67:0/2848937752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f837810de10 msgr2=0x7f837819fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:09.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.567+0000 7f837d438640 1 --2- 172.21.15.67:0/2848937752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f837810de10 0x7f837819fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:09.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.567+0000 7f837d438640 1 -- 172.21.15.67:0/2848937752 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83781a03c0 con 0x7f83781049b0 2024-09-21T06:35:09.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.567+0000 7f837dc39640 1 --2- 172.21.15.67:0/2848937752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f837810de10 0x7f837819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:09.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.567+0000 7f837d438640 1 --2- 172.21.15.67:0/2848937752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83781049b0 0x7f837819a450 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f837000d770 tx=0x7f837000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:09.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.567+0000 7f83767fc640 1 -- 172.21.15.67:0/2848937752 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8370017070 con 0x7f83781049b0 2024-09-21T06:35:09.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.567+0000 7f83767fc640 1 -- 172.21.15.67:0/2848937752 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8370009c30 con 0x7f83781049b0 2024-09-21T06:35:09.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.567+0000 7f837f69c640 1 -- 172.21.15.67:0/2848937752 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83781a06b0 con 0x7f83781049b0 2024-09-21T06:35:09.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.567+0000 7f83767fc640 1 -- 172.21.15.67:0/2848937752 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8370003c50 con 0x7f83781049b0 2024-09-21T06:35:09.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.567+0000 7f837f69c640 1 -- 172.21.15.67:0/2848937752 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83781a6e10 con 0x7f83781049b0 2024-09-21T06:35:09.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.568+0000 7f837f69c640 1 -- 172.21.15.67:0/2848937752 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8378104dc0 con 0x7f83781049b0 2024-09-21T06:35:09.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.570+0000 7f83767fc640 1 -- 172.21.15.67:0/2848937752 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f8370003df0 con 0x7f83781049b0 2024-09-21T06:35:09.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.571+0000 7f83767fc640 1 --2- 172.21.15.67:0/2848937752 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f83600b6510 0x7f83600b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:09.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.571+0000 7f837cc37640 1 --2- 172.21.15.67:0/2848937752 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f83600b6510 0x7f83600b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:09.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.571+0000 7f83767fc640 1 -- 172.21.15.67:0/2848937752 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f83700e7520 con 0x7f83781049b0 2024-09-21T06:35:09.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.572+0000 7f837cc37640 1 --2- 172.21.15.67:0/2848937752 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f83600b6510 0x7f83600b8a20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f836c0057c0 tx=0x7f836c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:09.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.575+0000 7f83767fc640 1 -- 172.21.15.67:0/2848937752 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8370009de0 con 0x7f83781049b0 2024-09-21T06:35:09.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:09.804+0000 7f837f69c640 1 -- 172.21.15.67:0/2848937752 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f837819b580 con 0x7f83781049b0 2024-09-21T06:35:10.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:10.021+0000 7f83767fc640 1 -- 172.21.15.67:0/2848937752 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) ==== 115+0+0 (secure 0 0 0) 0x7f83700b6020 con 0x7f83781049b0 2024-09-21T06:35:10.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:10.024+0000 7f837f69c640 1 -- 172.21.15.67:0/2848937752 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f83600b6510 msgr2=0x7f83600b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:10.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:10.024+0000 7f837f69c640 1 --2- 172.21.15.67:0/2848937752 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f83600b6510 0x7f83600b8a20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f836c0057c0 tx=0x7f836c000bc0 comp rx=0 tx=0).stop 2024-09-21T06:35:10.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:10.024+0000 7f837f69c640 1 -- 172.21.15.67:0/2848937752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83781049b0 msgr2=0x7f837819a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:10.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:10.024+0000 7f837f69c640 1 --2- 172.21.15.67:0/2848937752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83781049b0 0x7f837819a450 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f837000d770 tx=0x7f837000dc30 comp rx=0 tx=0).stop 2024-09-21T06:35:10.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:10.025+0000 7f837f69c640 1 -- 172.21.15.67:0/2848937752 shutdown_connections 2024-09-21T06:35:10.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:10.025+0000 7f837f69c640 1 --2- 172.21.15.67:0/2848937752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f837810de10 0x7f837819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:10.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:10.025+0000 7f837f69c640 1 --2- 172.21.15.67:0/2848937752 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f83600b6510 0x7f83600b8a20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:10.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:10.025+0000 7f837f69c640 1 --2- 172.21.15.67:0/2848937752 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8378105300 0x7f837819a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:10.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:10.025+0000 7f837f69c640 1 --2- 172.21.15.67:0/2848937752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83781049b0 0x7f837819a450 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:10.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:10.025+0000 7f837f69c640 1 -- 172.21.15.67:0/2848937752 >> 172.21.15.67:0/2848937752 conn(0x7f8378079730 msgr2=0x7f83780ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:10.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:10.025+0000 7f837f69c640 1 -- 172.21.15.67:0/2848937752 shutdown_connections 2024-09-21T06:35:10.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:10.025+0000 7f837f69c640 1 -- 172.21.15.67:0/2848937752 wait complete. 2024-09-21T06:35:10.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:10 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2848937752' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T06:35:10.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:10 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2848937752' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T06:35:10.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:10 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2848937752' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T06:35:10.799 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-09-21T06:35:10.799 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs subvolumegroup create cephfs qa 2024-09-21T06:35:11.062 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:11.291 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:11 smithi067 ceph-mon[31232]: pgmap v316: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-09-21T06:35:11.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:11 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2848937752' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T06:35:11.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:11 smithi067 ceph-mon[31232]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-21T06:35:11.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:11 smithi100 ceph-mon[33401]: pgmap v316: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-09-21T06:35:11.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:11 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2848937752' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T06:35:11.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:11 smithi100 ceph-mon[33401]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-21T06:35:11.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:11 smithi203 ceph-mon[33735]: pgmap v316: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-09-21T06:35:11.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:11 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2848937752' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T06:35:11.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:11 smithi203 ceph-mon[33735]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-21T06:35:11.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.558+0000 7f3ffb5ac640 1 -- 172.21.15.67:0/252017552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ff41083d0 msgr2=0x7f3ff41087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:11.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.558+0000 7f3ffb5ac640 1 --2- 172.21.15.67:0/252017552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ff41083d0 0x7f3ff41087e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4009900 tx=0x7f3fe402a410 comp rx=0 tx=0).stop 2024-09-21T06:35:11.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.558+0000 7f3ffb5ac640 1 -- 172.21.15.67:0/252017552 shutdown_connections 2024-09-21T06:35:11.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.558+0000 7f3ffb5ac640 1 --2- 172.21.15.67:0/252017552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ff4108d20 0x7f3ff410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:11.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.558+0000 7f3ffb5ac640 1 --2- 172.21.15.67:0/252017552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ff41083d0 0x7f3ff41087e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:11.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.558+0000 7f3ffb5ac640 1 --2- 172.21.15.67:0/252017552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ff4102a90 0x7f3ff4102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:11.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.558+0000 7f3ffb5ac640 1 -- 172.21.15.67:0/252017552 >> 172.21.15.67:0/252017552 conn(0x7f3ff4079750 msgr2=0x7f3ff4079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:11.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.558+0000 7f3ffb5ac640 1 -- 172.21.15.67:0/252017552 shutdown_connections 2024-09-21T06:35:11.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.558+0000 7f3ffb5ac640 1 -- 172.21.15.67:0/252017552 wait complete. 2024-09-21T06:35:11.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.559+0000 7f3ffb5ac640 1 Processor -- start 2024-09-21T06:35:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.559+0000 7f3ffb5ac640 1 -- start start 2024-09-21T06:35:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.559+0000 7f3ffb5ac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ff4102a90 0x7f3ff419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.559+0000 7f3ffb5ac640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ff41083d0 0x7f3ff419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.559+0000 7f3ffb5ac640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ff4108d20 0x7f3ff419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.559+0000 7f3ffb5ac640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff4113960 con 0x7f3ff4108d20 2024-09-21T06:35:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.559+0000 7f3ffb5ac640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff41137e0 con 0x7f3ff4102a90 2024-09-21T06:35:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.559+0000 7f3ffb5ac640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff4113ae0 con 0x7f3ff41083d0 2024-09-21T06:35:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.559+0000 7f3ff9b49640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ff4108d20 0x7f3ff419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.559+0000 7f3ff9348640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ff4102a90 0x7f3ff419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.559+0000 7f3ff9b49640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ff4108d20 0x7f3ff419fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42576/0 (socket says 172.21.15.67:42576) 2024-09-21T06:35:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.559+0000 7f3ff9b49640 1 -- 172.21.15.67:0/142921257 learned_addr learned my addr 172.21.15.67:0/142921257 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:11.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.559+0000 7f3ff8b47640 1 --2- 172.21.15.67:0/142921257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ff41083d0 0x7f3ff419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:11.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.560+0000 7f3ff9b49640 1 -- 172.21.15.67:0/142921257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ff41083d0 msgr2=0x7f3ff419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:11.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.560+0000 7f3ff9b49640 1 --2- 172.21.15.67:0/142921257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ff41083d0 0x7f3ff419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:11.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.560+0000 7f3ff9b49640 1 -- 172.21.15.67:0/142921257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ff4102a90 msgr2=0x7f3ff419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:11.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.560+0000 7f3ff9b49640 1 --2- 172.21.15.67:0/142921257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ff4102a90 0x7f3ff419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:11.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.560+0000 7f3ff9b49640 1 -- 172.21.15.67:0/142921257 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff41a0400 con 0x7f3ff4108d20 2024-09-21T06:35:11.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.560+0000 7f3ff9348640 1 --2- 172.21.15.67:0/142921257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ff4102a90 0x7f3ff419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:35:11.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.560+0000 7f3ff9b49640 1 --2- 172.21.15.67:0/142921257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ff4108d20 0x7f3ff419fcd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3fec009fd0 tx=0x7f3fec00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:11.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.560+0000 7f3fea7fc640 1 -- 172.21.15.67:0/142921257 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec017070 con 0x7f3ff4108d20 2024-09-21T06:35:11.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.560+0000 7f3fea7fc640 1 -- 172.21.15.67:0/142921257 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3fec003e00 con 0x7f3ff4108d20 2024-09-21T06:35:11.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.560+0000 7f3ffb5ac640 1 -- 172.21.15.67:0/142921257 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff41a06f0 con 0x7f3ff4108d20 2024-09-21T06:35:11.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.560+0000 7f3fea7fc640 1 -- 172.21.15.67:0/142921257 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec010460 con 0x7f3ff4108d20 2024-09-21T06:35:11.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.560+0000 7f3ffb5ac640 1 -- 172.21.15.67:0/142921257 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff41a6e50 con 0x7f3ff4108d20 2024-09-21T06:35:11.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.561+0000 7f3ffb5ac640 1 -- 172.21.15.67:0/142921257 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ff4103d10 con 0x7f3ff4108d20 2024-09-21T06:35:11.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.563+0000 7f3fea7fc640 1 -- 172.21.15.67:0/142921257 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f3fec003970 con 0x7f3ff4108d20 2024-09-21T06:35:11.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.564+0000 7f3fea7fc640 1 --2- 172.21.15.67:0/142921257 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f3fe00b6510 0x7f3fe00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:11.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.564+0000 7f3ff9348640 1 --2- 172.21.15.67:0/142921257 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f3fe00b6510 0x7f3fe00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:11.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.564+0000 7f3fea7fc640 1 -- 172.21.15.67:0/142921257 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f3fec0e76c0 con 0x7f3ff4108d20 2024-09-21T06:35:11.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.564+0000 7f3ff9348640 1 --2- 172.21.15.67:0/142921257 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f3fe00b6510 0x7f3fe00b8a20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009fc0 tx=0x7f3ff0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:11.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.568+0000 7f3fea7fc640 1 -- 172.21.15.67:0/142921257 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3fec0b1fd0 con 0x7f3ff4108d20 2024-09-21T06:35:11.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.785+0000 7f3ffb5ac640 1 -- 172.21.15.67:0/142921257 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f3ff4063800 con 0x7f3fe00b6510 2024-09-21T06:35:11.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.806+0000 7f3fea7fc640 1 -- 172.21.15.67:0/142921257 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3ff4063800 con 0x7f3fe00b6510 2024-09-21T06:35:11.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.809+0000 7f3ffb5ac640 1 -- 172.21.15.67:0/142921257 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f3fe00b6510 msgr2=0x7f3fe00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:11.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.809+0000 7f3ffb5ac640 1 --2- 172.21.15.67:0/142921257 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f3fe00b6510 0x7f3fe00b8a20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009fc0 tx=0x7f3ff0005f90 comp rx=0 tx=0).stop 2024-09-21T06:35:11.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.809+0000 7f3ffb5ac640 1 -- 172.21.15.67:0/142921257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ff4108d20 msgr2=0x7f3ff419fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:11.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.809+0000 7f3ffb5ac640 1 --2- 172.21.15.67:0/142921257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ff4108d20 0x7f3ff419fcd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3fec009fd0 tx=0x7f3fec00b260 comp rx=0 tx=0).stop 2024-09-21T06:35:11.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.809+0000 7f3ffb5ac640 1 -- 172.21.15.67:0/142921257 shutdown_connections 2024-09-21T06:35:11.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.809+0000 7f3ffb5ac640 1 --2- 172.21.15.67:0/142921257 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f3fe00b6510 0x7f3fe00b8a20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:11.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.809+0000 7f3ffb5ac640 1 --2- 172.21.15.67:0/142921257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ff4108d20 0x7f3ff419fcd0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:11.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.809+0000 7f3ffb5ac640 1 --2- 172.21.15.67:0/142921257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ff41083d0 0x7f3ff419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:11.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.809+0000 7f3ffb5ac640 1 --2- 172.21.15.67:0/142921257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ff4102a90 0x7f3ff419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:11.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.809+0000 7f3ffb5ac640 1 -- 172.21.15.67:0/142921257 >> 172.21.15.67:0/142921257 conn(0x7f3ff4079750 msgr2=0x7f3ff4104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:11.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.809+0000 7f3ffb5ac640 1 -- 172.21.15.67:0/142921257 shutdown_connections 2024-09-21T06:35:11.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:11.809+0000 7f3ffb5ac640 1 -- 172.21.15.67:0/142921257 wait complete. 2024-09-21T06:35:12.042 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:35:11 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: 2024-09-21T06:35:11.791+0000 7fbf9a013640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:35:12.042 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:35:11 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: 2024-09-21T06:35:11.791+0000 7fbf9a013640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:35:12.042 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:35:11 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: 2024-09-21T06:35:11.791+0000 7fbf9a013640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:35:12.042 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:35:11 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: 2024-09-21T06:35:11.791+0000 7fbf9a013640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:35:12.042 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:35:11 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: 2024-09-21T06:35:11.791+0000 7fbf9a013640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:35:12.657 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-21T06:35:12.918 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:13.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:13 smithi067 ceph-mon[31232]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:35:13.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:13 smithi067 ceph-mon[31232]: pgmap v317: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 127 B/s wr, 1 op/s 2024-09-21T06:35:13.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:13 smithi067 ceph-mon[31232]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-21T06:35:13.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:13 smithi100 ceph-mon[33401]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:35:13.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:13 smithi100 ceph-mon[33401]: pgmap v317: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 127 B/s wr, 1 op/s 2024-09-21T06:35:13.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:13 smithi100 ceph-mon[33401]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-21T06:35:13.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.438+0000 7f72984dc640 1 -- 172.21.15.67:0/3197901943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7290102ae0 msgr2=0x7f7290102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:13.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.438+0000 7f72984dc640 1 --2- 172.21.15.67:0/3197901943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7290102ae0 0x7f7290102ef0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f728c009900 tx=0x7f728c02a410 comp rx=0 tx=0).stop 2024-09-21T06:35:13.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.438+0000 7f72984dc640 1 -- 172.21.15.67:0/3197901943 shutdown_connections 2024-09-21T06:35:13.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.438+0000 7f72984dc640 1 --2- 172.21.15.67:0/3197901943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72901083d0 0x7f72901087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.438+0000 7f72984dc640 1 --2- 172.21.15.67:0/3197901943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7290108d20 0x7f729010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.438+0000 7f72984dc640 1 --2- 172.21.15.67:0/3197901943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7290102ae0 0x7f7290102ef0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.438+0000 7f72984dc640 1 -- 172.21.15.67:0/3197901943 >> 172.21.15.67:0/3197901943 conn(0x7f7290079760 msgr2=0x7f7290079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:13.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.438+0000 7f72984dc640 1 -- 172.21.15.67:0/3197901943 shutdown_connections 2024-09-21T06:35:13.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.439+0000 7f72984dc640 1 -- 172.21.15.67:0/3197901943 wait complete. 2024-09-21T06:35:13.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.439+0000 7f72984dc640 1 Processor -- start 2024-09-21T06:35:13.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.439+0000 7f72984dc640 1 -- start start 2024-09-21T06:35:13.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.439+0000 7f72984dc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7290102ae0 0x7f729019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.439+0000 7f72984dc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72901083d0 0x7f729019a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.439+0000 7f72984dc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7290108d20 0x7f729019fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.440+0000 7f72984dc640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7290113960 con 0x7f7290102ae0 2024-09-21T06:35:13.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.440+0000 7f72984dc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f72901137e0 con 0x7f72901083d0 2024-09-21T06:35:13.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.440+0000 7f72984dc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7290113ae0 con 0x7f7290108d20 2024-09-21T06:35:13.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.440+0000 7f7296278640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7290102ae0 0x7f729019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.440+0000 7f7296278640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7290102ae0 0x7f729019a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42612/0 (socket says 172.21.15.67:42612) 2024-09-21T06:35:13.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.440+0000 7f7296278640 1 -- 172.21.15.67:0/3601285497 learned_addr learned my addr 172.21.15.67:0/3601285497 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:13.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.440+0000 7f7296a79640 1 --2- 172.21.15.67:0/3601285497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7290108d20 0x7f729019fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.440+0000 7f7295a77640 1 --2- 172.21.15.67:0/3601285497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72901083d0 0x7f729019a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.440+0000 7f7296278640 1 -- 172.21.15.67:0/3601285497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7290108d20 msgr2=0x7f729019fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:13.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.440+0000 7f7296278640 1 --2- 172.21.15.67:0/3601285497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7290108d20 0x7f729019fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.440+0000 7f7296278640 1 -- 172.21.15.67:0/3601285497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72901083d0 msgr2=0x7f729019a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:13.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.440+0000 7f7296278640 1 --2- 172.21.15.67:0/3601285497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72901083d0 0x7f729019a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.440+0000 7f7296278640 1 -- 172.21.15.67:0/3601285497 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72901a0260 con 0x7f7290102ae0 2024-09-21T06:35:13.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.440+0000 7f7296278640 1 --2- 172.21.15.67:0/3601285497 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7290102ae0 0x7f729019a3b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f728c009a20 tx=0x7f728c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:13.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.440+0000 7f72837fe640 1 -- 172.21.15.67:0/3601285497 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c03f030 con 0x7f7290102ae0 2024-09-21T06:35:13.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.440+0000 7f72837fe640 1 -- 172.21.15.67:0/3601285497 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f728c0035f0 con 0x7f7290102ae0 2024-09-21T06:35:13.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.440+0000 7f72984dc640 1 -- 172.21.15.67:0/3601285497 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72901a04f0 con 0x7f7290102ae0 2024-09-21T06:35:13.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.441+0000 7f72837fe640 1 -- 172.21.15.67:0/3601285497 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c035070 con 0x7f7290102ae0 2024-09-21T06:35:13.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.441+0000 7f72984dc640 1 -- 172.21.15.67:0/3601285497 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72901a0a20 con 0x7f7290102ae0 2024-09-21T06:35:13.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.442+0000 7f72984dc640 1 -- 172.21.15.67:0/3601285497 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7290103d10 con 0x7f7290102ae0 2024-09-21T06:35:13.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.443+0000 7f72837fe640 1 -- 172.21.15.67:0/3601285497 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f728c003aa0 con 0x7f7290102ae0 2024-09-21T06:35:13.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.445+0000 7f72837fe640 1 --2- 172.21.15.67:0/3601285497 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f727c0b6650 0x7f727c0b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:13.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.445+0000 7f7295a77640 1 --2- 172.21.15.67:0/3601285497 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f727c0b6650 0x7f727c0b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:13.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.445+0000 7f72837fe640 1 -- 172.21.15.67:0/3601285497 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f728c108790 con 0x7f7290102ae0 2024-09-21T06:35:13.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.445+0000 7f7295a77640 1 --2- 172.21.15.67:0/3601285497 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f727c0b6650 0x7f727c0b8b60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7290079ed0 tx=0x7f728400a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:13.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.449+0000 7f72837fe640 1 -- 172.21.15.67:0/3601285497 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f72901a0a20 con 0x7f7290102ae0 2024-09-21T06:35:13.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:13 smithi203 ceph-mon[33735]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:35:13.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:13 smithi203 ceph-mon[33735]: pgmap v317: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 127 B/s wr, 1 op/s 2024-09-21T06:35:13.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:13 smithi203 ceph-mon[33735]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-21T06:35:13.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.664+0000 7f72984dc640 1 -- 172.21.15.67:0/3601285497 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f729010ab50 con 0x7f727c0b6650 2024-09-21T06:35:13.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.692+0000 7f72837fe640 1 -- 172.21.15.67:0/3601285497 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f729010ab50 con 0x7f727c0b6650 2024-09-21T06:35:13.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.694+0000 7f72984dc640 1 -- 172.21.15.67:0/3601285497 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f727c0b6650 msgr2=0x7f727c0b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:13.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.694+0000 7f72984dc640 1 --2- 172.21.15.67:0/3601285497 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f727c0b6650 0x7f727c0b8b60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7290079ed0 tx=0x7f728400a4c0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.694+0000 7f72984dc640 1 -- 172.21.15.67:0/3601285497 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7290102ae0 msgr2=0x7f729019a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:13.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.694+0000 7f72984dc640 1 --2- 172.21.15.67:0/3601285497 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7290102ae0 0x7f729019a3b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f728c009a20 tx=0x7f728c004270 comp rx=0 tx=0).stop 2024-09-21T06:35:13.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.695+0000 7f72984dc640 1 -- 172.21.15.67:0/3601285497 shutdown_connections 2024-09-21T06:35:13.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.695+0000 7f72984dc640 1 --2- 172.21.15.67:0/3601285497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72901083d0 0x7f729019a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.695+0000 7f72984dc640 1 --2- 172.21.15.67:0/3601285497 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f727c0b6650 0x7f727c0b8b60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.695+0000 7f72984dc640 1 --2- 172.21.15.67:0/3601285497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7290108d20 0x7f729019fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.695+0000 7f72984dc640 1 --2- 172.21.15.67:0/3601285497 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7290102ae0 0x7f729019a3b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:13.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.695+0000 7f72984dc640 1 -- 172.21.15.67:0/3601285497 >> 172.21.15.67:0/3601285497 conn(0x7f7290079760 msgr2=0x7f7290101610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:13.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.695+0000 7f72984dc640 1 -- 172.21.15.67:0/3601285497 shutdown_connections 2024-09-21T06:35:13.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:13.695+0000 7f72984dc640 1 -- 172.21.15.67:0/3601285497 wait complete. 2024-09-21T06:35:14.527 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-21T06:35:14.795 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:15.206 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:15 smithi067 ceph-mon[31232]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:35:15.207 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:15 smithi067 ceph-mon[31232]: pgmap v318: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 103 B/s wr, 1 op/s 2024-09-21T06:35:15.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:15 smithi100 ceph-mon[33401]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:35:15.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:15 smithi100 ceph-mon[33401]: pgmap v318: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 103 B/s wr, 1 op/s 2024-09-21T06:35:15.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.303+0000 7f45a5cc8640 1 -- 172.21.15.67:0/1239724042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45a01083a0 msgr2=0x7f45a01087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:15.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.303+0000 7f45a5cc8640 1 --2- 172.21.15.67:0/1239724042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45a01083a0 0x7f45a01087b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4590009900 tx=0x7f459002a410 comp rx=0 tx=0).stop 2024-09-21T06:35:15.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.303+0000 7f45a5cc8640 1 -- 172.21.15.67:0/1239724042 shutdown_connections 2024-09-21T06:35:15.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.303+0000 7f45a5cc8640 1 --2- 172.21.15.67:0/1239724042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45a0108cf0 0x7f45a010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:15.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.303+0000 7f45a5cc8640 1 --2- 172.21.15.67:0/1239724042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45a01083a0 0x7f45a01087b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:15.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.303+0000 7f45a5cc8640 1 --2- 172.21.15.67:0/1239724042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45a0102a60 0x7f45a0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:15.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.303+0000 7f45a5cc8640 1 -- 172.21.15.67:0/1239724042 >> 172.21.15.67:0/1239724042 conn(0x7f45a0079730 msgr2=0x7f45a0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:15.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.303+0000 7f45a5cc8640 1 -- 172.21.15.67:0/1239724042 shutdown_connections 2024-09-21T06:35:15.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.303+0000 7f45a5cc8640 1 -- 172.21.15.67:0/1239724042 wait complete. 2024-09-21T06:35:15.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.304+0000 7f45a5cc8640 1 Processor -- start 2024-09-21T06:35:15.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.304+0000 7f45a5cc8640 1 -- start start 2024-09-21T06:35:15.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.304+0000 7f45a5cc8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45a0102a60 0x7f45a019b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:15.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.304+0000 7f45a5cc8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45a01083a0 0x7f45a019b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:15.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.304+0000 7f45a5cc8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45a0108cf0 0x7f45a019bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:15.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.304+0000 7f45a5cc8640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f45a0112810 con 0x7f45a0102a60 2024-09-21T06:35:15.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.304+0000 7f45a5cc8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f45a0112690 con 0x7f45a0108cf0 2024-09-21T06:35:15.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.304+0000 7f45a5cc8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f45a0112990 con 0x7f45a01083a0 2024-09-21T06:35:15.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.304+0000 7f459f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45a0102a60 0x7f45a019b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:15.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.304+0000 7f459f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45a0102a60 0x7f45a019b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42628/0 (socket says 172.21.15.67:42628) 2024-09-21T06:35:15.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.304+0000 7f459f7fe640 1 -- 172.21.15.67:0/1405972521 learned_addr learned my addr 172.21.15.67:0/1405972521 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:15.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.304+0000 7f459ffff640 1 --2- 172.21.15.67:0/1405972521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45a0108cf0 0x7f45a019bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:15.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.304+0000 7f459effd640 1 --2- 172.21.15.67:0/1405972521 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45a01083a0 0x7f45a019b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:15.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.305+0000 7f459f7fe640 1 -- 172.21.15.67:0/1405972521 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45a01083a0 msgr2=0x7f45a019b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:15.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.305+0000 7f459f7fe640 1 --2- 172.21.15.67:0/1405972521 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45a01083a0 0x7f45a019b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:15.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.305+0000 7f459f7fe640 1 -- 172.21.15.67:0/1405972521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45a0108cf0 msgr2=0x7f45a019bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:15.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.305+0000 7f459f7fe640 1 --2- 172.21.15.67:0/1405972521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45a0108cf0 0x7f45a019bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:15.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.305+0000 7f459f7fe640 1 -- 172.21.15.67:0/1405972521 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a01a6460 con 0x7f45a0102a60 2024-09-21T06:35:15.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.305+0000 7f459f7fe640 1 --2- 172.21.15.67:0/1405972521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45a0102a60 0x7f45a019b1a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f458800d950 tx=0x7f458800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:15.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.305+0000 7f459cff9640 1 -- 172.21.15.67:0/1405972521 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4588017070 con 0x7f45a0102a60 2024-09-21T06:35:15.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.305+0000 7f45a5cc8640 1 -- 172.21.15.67:0/1405972521 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45a01a6750 con 0x7f45a0102a60 2024-09-21T06:35:15.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.305+0000 7f459cff9640 1 -- 172.21.15.67:0/1405972521 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4588009c30 con 0x7f45a0102a60 2024-09-21T06:35:15.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.305+0000 7f459cff9640 1 -- 172.21.15.67:0/1405972521 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458800f620 con 0x7f45a0102a60 2024-09-21T06:35:15.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.305+0000 7f45a5cc8640 1 -- 172.21.15.67:0/1405972521 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45a01a6cb0 con 0x7f45a0102a60 2024-09-21T06:35:15.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.306+0000 7f45a5cc8640 1 -- 172.21.15.67:0/1405972521 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45a0102e70 con 0x7f45a0102a60 2024-09-21T06:35:15.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.308+0000 7f459cff9640 1 -- 172.21.15.67:0/1405972521 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f4588010460 con 0x7f45a0102a60 2024-09-21T06:35:15.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.308+0000 7f459cff9640 1 --2- 172.21.15.67:0/1405972521 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f458c0b65a0 0x7f458c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:15.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.309+0000 7f459effd640 1 --2- 172.21.15.67:0/1405972521 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f458c0b65a0 0x7f458c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:15.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.309+0000 7f459cff9640 1 -- 172.21.15.67:0/1405972521 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f45880e7370 con 0x7f45a0102a60 2024-09-21T06:35:15.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.309+0000 7f459effd640 1 --2- 172.21.15.67:0/1405972521 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f458c0b65a0 0x7f458c0b8ab0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4590004770 tx=0x7f45900058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:15.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.312+0000 7f459cff9640 1 -- 172.21.15.67:0/1405972521 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4588009de0 con 0x7f45a0102a60 2024-09-21T06:35:15.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:15 smithi203 ceph-mon[33735]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:35:15.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:15 smithi203 ceph-mon[33735]: pgmap v318: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 103 B/s wr, 1 op/s 2024-09-21T06:35:15.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.531+0000 7f45a5cc8640 1 -- 172.21.15.67:0/1405972521 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f45a0063800 con 0x7f458c0b65a0 2024-09-21T06:35:15.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.539+0000 7f459cff9640 1 -- 172.21.15.67:0/1405972521 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f45a0063800 con 0x7f458c0b65a0 2024-09-21T06:35:15.541 INFO:teuthology.orchestra.run.smithi067.stdout:/volumes/qa/sv_0/d337545a-42eb-483d-9071-3b1bcc855266 2024-09-21T06:35:15.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.541+0000 7f45a5cc8640 1 -- 172.21.15.67:0/1405972521 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f458c0b65a0 msgr2=0x7f458c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:15.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.541+0000 7f45a5cc8640 1 --2- 172.21.15.67:0/1405972521 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f458c0b65a0 0x7f458c0b8ab0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4590004770 tx=0x7f45900058d0 comp rx=0 tx=0).stop 2024-09-21T06:35:15.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.541+0000 7f45a5cc8640 1 -- 172.21.15.67:0/1405972521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45a0102a60 msgr2=0x7f45a019b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:15.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.541+0000 7f45a5cc8640 1 --2- 172.21.15.67:0/1405972521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45a0102a60 0x7f45a019b1a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f458800d950 tx=0x7f458800de10 comp rx=0 tx=0).stop 2024-09-21T06:35:15.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.541+0000 7f45a5cc8640 1 -- 172.21.15.67:0/1405972521 shutdown_connections 2024-09-21T06:35:15.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.541+0000 7f45a5cc8640 1 --2- 172.21.15.67:0/1405972521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45a0108cf0 0x7f45a019bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:15.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.541+0000 7f45a5cc8640 1 --2- 172.21.15.67:0/1405972521 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f458c0b65a0 0x7f458c0b8ab0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:15.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.541+0000 7f45a5cc8640 1 --2- 172.21.15.67:0/1405972521 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45a01083a0 0x7f45a019b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:15.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.541+0000 7f45a5cc8640 1 --2- 172.21.15.67:0/1405972521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45a0102a60 0x7f45a019b1a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:15.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.541+0000 7f45a5cc8640 1 -- 172.21.15.67:0/1405972521 >> 172.21.15.67:0/1405972521 conn(0x7f45a0079730 msgr2=0x7f45a0104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:15.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.541+0000 7f45a5cc8640 1 -- 172.21.15.67:0/1405972521 shutdown_connections 2024-09-21T06:35:15.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:15.541+0000 7f45a5cc8640 1 -- 172.21.15.67:0/1405972521 wait complete. 2024-09-21T06:35:16.225 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:16 smithi067 ceph-mon[31232]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:35:16.253 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-21T06:35:16.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:16 smithi100 ceph-mon[33401]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:35:16.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:16 smithi203 ceph-mon[33735]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:35:16.511 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:17.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.015+0000 7f89656fb640 1 -- 172.21.15.67:0/3945663340 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8960102a80 msgr2=0x7f8960102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:17.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.015+0000 7f89656fb640 1 --2- 172.21.15.67:0/3945663340 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8960102a80 0x7f8960102e90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8950009900 tx=0x7f895002a410 comp rx=0 tx=0).stop 2024-09-21T06:35:17.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.015+0000 7f89656fb640 1 -- 172.21.15.67:0/3945663340 shutdown_connections 2024-09-21T06:35:17.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.015+0000 7f89656fb640 1 --2- 172.21.15.67:0/3945663340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89601083c0 0x7f89601087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:17.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.015+0000 7f89656fb640 1 --2- 172.21.15.67:0/3945663340 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8960108d10 0x7f896010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:17.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.015+0000 7f89656fb640 1 --2- 172.21.15.67:0/3945663340 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8960102a80 0x7f8960102e90 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:17.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.015+0000 7f89656fb640 1 -- 172.21.15.67:0/3945663340 >> 172.21.15.67:0/3945663340 conn(0x7f8960079740 msgr2=0x7f8960079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:17.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.015+0000 7f89656fb640 1 -- 172.21.15.67:0/3945663340 shutdown_connections 2024-09-21T06:35:17.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.015+0000 7f89656fb640 1 -- 172.21.15.67:0/3945663340 wait complete. 2024-09-21T06:35:17.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.016+0000 7f89656fb640 1 Processor -- start 2024-09-21T06:35:17.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.016+0000 7f89656fb640 1 -- start start 2024-09-21T06:35:17.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.016+0000 7f89656fb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8960102a80 0x7f896019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:17.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.016+0000 7f89656fb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89601083c0 0x7f896019a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:17.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.016+0000 7f89656fb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8960108d10 0x7f896019fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:17.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.016+0000 7f89656fb640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8960113950 con 0x7f8960102a80 2024-09-21T06:35:17.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.016+0000 7f89656fb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f89601137d0 con 0x7f8960108d10 2024-09-21T06:35:17.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.016+0000 7f89656fb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8960113ad0 con 0x7f89601083c0 2024-09-21T06:35:17.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.016+0000 7f895effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8960102a80 0x7f896019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:17.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.016+0000 7f895effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8960102a80 0x7f896019a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42650/0 (socket says 172.21.15.67:42650) 2024-09-21T06:35:17.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.016+0000 7f895effd640 1 -- 172.21.15.67:0/1743694134 learned_addr learned my addr 172.21.15.67:0/1743694134 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:17.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.016+0000 7f895f7fe640 1 --2- 172.21.15.67:0/1743694134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8960108d10 0x7f896019fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:17.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.016+0000 7f8957fff640 1 --2- 172.21.15.67:0/1743694134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89601083c0 0x7f896019a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:17.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.017+0000 7f895effd640 1 -- 172.21.15.67:0/1743694134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89601083c0 msgr2=0x7f896019a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:17.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.017+0000 7f895effd640 1 --2- 172.21.15.67:0/1743694134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89601083c0 0x7f896019a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:17.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.017+0000 7f895effd640 1 -- 172.21.15.67:0/1743694134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8960108d10 msgr2=0x7f896019fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:17.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.017+0000 7f895effd640 1 --2- 172.21.15.67:0/1743694134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8960108d10 0x7f896019fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:17.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.017+0000 7f895effd640 1 -- 172.21.15.67:0/1743694134 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89601a02f0 con 0x7f8960102a80 2024-09-21T06:35:17.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.017+0000 7f895effd640 1 --2- 172.21.15.67:0/1743694134 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8960102a80 0x7f896019a380 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8950009230 tx=0x7f8950004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:17.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.017+0000 7f895cff9640 1 -- 172.21.15.67:0/1743694134 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895003f030 con 0x7f8960102a80 2024-09-21T06:35:17.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.017+0000 7f895cff9640 1 -- 172.21.15.67:0/1743694134 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8950003a80 con 0x7f8960102a80 2024-09-21T06:35:17.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.017+0000 7f89656fb640 1 -- 172.21.15.67:0/1743694134 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89601a0580 con 0x7f8960102a80 2024-09-21T06:35:17.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.017+0000 7f895cff9640 1 -- 172.21.15.67:0/1743694134 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8950035070 con 0x7f8960102a80 2024-09-21T06:35:17.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.017+0000 7f89656fb640 1 -- 172.21.15.67:0/1743694134 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89601a6d40 con 0x7f8960102a80 2024-09-21T06:35:17.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.018+0000 7f89656fb640 1 -- 172.21.15.67:0/1743694134 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8960103d00 con 0x7f8960102a80 2024-09-21T06:35:17.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.020+0000 7f895cff9640 1 -- 172.21.15.67:0/1743694134 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f8950042030 con 0x7f8960102a80 2024-09-21T06:35:17.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.020+0000 7f895cff9640 1 --2- 172.21.15.67:0/1743694134 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f894c0b65a0 0x7f894c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:17.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.021+0000 7f895cff9640 1 -- 172.21.15.67:0/1743694134 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f8950107cb0 con 0x7f8960102a80 2024-09-21T06:35:17.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.021+0000 7f8957fff640 1 --2- 172.21.15.67:0/1743694134 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f894c0b65a0 0x7f894c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:17.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.021+0000 7f8957fff640 1 --2- 172.21.15.67:0/1743694134 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f894c0b65a0 0x7f894c0b8ab0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8960100c70 tx=0x7f894800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:17.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.025+0000 7f895cff9640 1 -- 172.21.15.67:0/1743694134 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f89500d2460 con 0x7f8960102a80 2024-09-21T06:35:17.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.239+0000 7f89656fb640 1 -- 172.21.15.67:0/1743694134 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f8960063800 con 0x7f894c0b65a0 2024-09-21T06:35:17.242 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:17 smithi067 ceph-mon[31232]: pgmap v319: 129 pgs: 129 active+clean; 603 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.5 KiB/s wr, 3 op/s 2024-09-21T06:35:17.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.266+0000 7f895cff9640 1 -- 172.21.15.67:0/1743694134 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8960063800 con 0x7f894c0b65a0 2024-09-21T06:35:17.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.268+0000 7f89656fb640 1 -- 172.21.15.67:0/1743694134 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f894c0b65a0 msgr2=0x7f894c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:17.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.268+0000 7f89656fb640 1 --2- 172.21.15.67:0/1743694134 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f894c0b65a0 0x7f894c0b8ab0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8960100c70 tx=0x7f894800b410 comp rx=0 tx=0).stop 2024-09-21T06:35:17.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.268+0000 7f89656fb640 1 -- 172.21.15.67:0/1743694134 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8960102a80 msgr2=0x7f896019a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:17.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.268+0000 7f89656fb640 1 --2- 172.21.15.67:0/1743694134 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8960102a80 0x7f896019a380 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8950009230 tx=0x7f8950004270 comp rx=0 tx=0).stop 2024-09-21T06:35:17.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.268+0000 7f89656fb640 1 -- 172.21.15.67:0/1743694134 shutdown_connections 2024-09-21T06:35:17.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.268+0000 7f89656fb640 1 --2- 172.21.15.67:0/1743694134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8960108d10 0x7f896019fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:17.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.268+0000 7f89656fb640 1 --2- 172.21.15.67:0/1743694134 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f894c0b65a0 0x7f894c0b8ab0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:17.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.268+0000 7f89656fb640 1 --2- 172.21.15.67:0/1743694134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89601083c0 0x7f896019a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:17.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.268+0000 7f89656fb640 1 --2- 172.21.15.67:0/1743694134 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8960102a80 0x7f896019a380 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:17.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.268+0000 7f89656fb640 1 -- 172.21.15.67:0/1743694134 >> 172.21.15.67:0/1743694134 conn(0x7f8960079740 msgr2=0x7f89601014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:17.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.268+0000 7f89656fb640 1 -- 172.21.15.67:0/1743694134 shutdown_connections 2024-09-21T06:35:17.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:17.268+0000 7f89656fb640 1 -- 172.21.15.67:0/1743694134 wait complete. 2024-09-21T06:35:17.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:17 smithi100 ceph-mon[33401]: pgmap v319: 129 pgs: 129 active+clean; 603 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.5 KiB/s wr, 3 op/s 2024-09-21T06:35:17.383 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:17 smithi203 ceph-mon[33735]: pgmap v319: 129 pgs: 129 active+clean; 603 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.5 KiB/s wr, 3 op/s 2024-09-21T06:35:18.000 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-21T06:35:18.261 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:18.291 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:18 smithi067 ceph-mon[31232]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:35:18.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:18 smithi100 ceph-mon[33401]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:35:18.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:18 smithi203 ceph-mon[33735]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:35:18.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.777+0000 7f4a3fc62640 1 -- 172.21.15.67:0/398839346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a3806f0d0 msgr2=0x7f4a3806f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:18.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.777+0000 7f4a3fc62640 1 --2- 172.21.15.67:0/398839346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a3806f0d0 0x7f4a3806f4e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c009900 tx=0x7f4a2c02a410 comp rx=0 tx=0).stop 2024-09-21T06:35:18.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.777+0000 7f4a3fc62640 1 -- 172.21.15.67:0/398839346 shutdown_connections 2024-09-21T06:35:18.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.777+0000 7f4a3fc62640 1 --2- 172.21.15.67:0/398839346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a3811cf80 0x7f4a3811f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:18.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.777+0000 7f4a3fc62640 1 --2- 172.21.15.67:0/398839346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a3806fa20 0x7f4a3811ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:18.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.777+0000 7f4a3fc62640 1 --2- 172.21.15.67:0/398839346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a3806f0d0 0x7f4a3806f4e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:18.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.777+0000 7f4a3fc62640 1 -- 172.21.15.67:0/398839346 >> 172.21.15.67:0/398839346 conn(0x7f4a3810a3c0 msgr2=0x7f4a3810c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:18.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.777+0000 7f4a3fc62640 1 -- 172.21.15.67:0/398839346 shutdown_connections 2024-09-21T06:35:18.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.777+0000 7f4a3fc62640 1 -- 172.21.15.67:0/398839346 wait complete. 2024-09-21T06:35:18.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.778+0000 7f4a3fc62640 1 Processor -- start 2024-09-21T06:35:18.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.778+0000 7f4a3fc62640 1 -- start start 2024-09-21T06:35:18.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.778+0000 7f4a3fc62640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a3806f0d0 0x7f4a381a9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:18.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.778+0000 7f4a3fc62640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a3806fa20 0x7f4a381a97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:18.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.778+0000 7f4a3fc62640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a3811cf80 0x7f4a381aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:18.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.778+0000 7f4a3fc62640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4a38122780 con 0x7f4a3806f0d0 2024-09-21T06:35:18.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.778+0000 7f4a3fc62640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4a38122600 con 0x7f4a3811cf80 2024-09-21T06:35:18.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.778+0000 7f4a3fc62640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4a38122900 con 0x7f4a3806fa20 2024-09-21T06:35:18.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.778+0000 7f4a3d9fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a3806f0d0 0x7f4a381a9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:18.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.778+0000 7f4a3d9fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a3806f0d0 0x7f4a381a9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54798/0 (socket says 172.21.15.67:54798) 2024-09-21T06:35:18.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.778+0000 7f4a3d9fe640 1 -- 172.21.15.67:0/2061394952 learned_addr learned my addr 172.21.15.67:0/2061394952 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:18.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.778+0000 7f4a3e1ff640 1 --2- 172.21.15.67:0/2061394952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a3811cf80 0x7f4a381aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:18.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.778+0000 7f4a3d1fd640 1 --2- 172.21.15.67:0/2061394952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a3806fa20 0x7f4a381a97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:18.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.779+0000 7f4a3d9fe640 1 -- 172.21.15.67:0/2061394952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a3806fa20 msgr2=0x7f4a381a97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:18.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.779+0000 7f4a3d9fe640 1 --2- 172.21.15.67:0/2061394952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a3806fa20 0x7f4a381a97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:18.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.779+0000 7f4a3d9fe640 1 -- 172.21.15.67:0/2061394952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a3811cf80 msgr2=0x7f4a381aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:18.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.779+0000 7f4a3d9fe640 1 --2- 172.21.15.67:0/2061394952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a3811cf80 0x7f4a381aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:18.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.779+0000 7f4a3d9fe640 1 -- 172.21.15.67:0/2061394952 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a381af0c0 con 0x7f4a3806f0d0 2024-09-21T06:35:18.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.779+0000 7f4a3d9fe640 1 --2- 172.21.15.67:0/2061394952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a3806f0d0 0x7f4a381a9210 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c02a930 tx=0x7f4a2c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:18.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.779+0000 7f4a2affd640 1 -- 172.21.15.67:0/2061394952 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c03f070 con 0x7f4a3806f0d0 2024-09-21T06:35:18.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.779+0000 7f4a2affd640 1 -- 172.21.15.67:0/2061394952 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4a2c003e40 con 0x7f4a3806f0d0 2024-09-21T06:35:18.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.779+0000 7f4a3fc62640 1 -- 172.21.15.67:0/2061394952 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a381af350 con 0x7f4a3806f0d0 2024-09-21T06:35:18.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.779+0000 7f4a2affd640 1 -- 172.21.15.67:0/2061394952 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c035030 con 0x7f4a3806f0d0 2024-09-21T06:35:18.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.779+0000 7f4a3fc62640 1 -- 172.21.15.67:0/2061394952 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a381af830 con 0x7f4a3806f0d0 2024-09-21T06:35:18.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.780+0000 7f4a3fc62640 1 -- 172.21.15.67:0/2061394952 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a3806f4e0 con 0x7f4a3806f0d0 2024-09-21T06:35:18.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.782+0000 7f4a2affd640 1 -- 172.21.15.67:0/2061394952 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f4a2c042070 con 0x7f4a3806f0d0 2024-09-21T06:35:18.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.782+0000 7f4a2affd640 1 --2- 172.21.15.67:0/2061394952 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f4a240b65d0 0x7f4a240b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:18.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.783+0000 7f4a3d1fd640 1 --2- 172.21.15.67:0/2061394952 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f4a240b65d0 0x7f4a240b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:18.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.783+0000 7f4a2affd640 1 -- 172.21.15.67:0/2061394952 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f4a2c108350 con 0x7f4a3806f0d0 2024-09-21T06:35:18.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.783+0000 7f4a3d1fd640 1 --2- 172.21.15.67:0/2061394952 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f4a240b65d0 0x7f4a240b8ae0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4a38071170 tx=0x7f4a3400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:18.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:18.786+0000 7f4a2affd640 1 -- 172.21.15.67:0/2061394952 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4a2c0d2b00 con 0x7f4a3806f0d0 2024-09-21T06:35:19.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:19.004+0000 7f4a3fc62640 1 -- 172.21.15.67:0/2061394952 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f4a38118140 con 0x7f4a240b65d0 2024-09-21T06:35:19.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:19.009+0000 7f4a2affd640 1 -- 172.21.15.67:0/2061394952 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f4a38118140 con 0x7f4a240b65d0 2024-09-21T06:35:19.011 INFO:teuthology.orchestra.run.smithi067.stdout:/volumes/qa/sv_1/8d8d2f61-b8d5-4ff8-8d8f-ccb717d55c51 2024-09-21T06:35:19.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:19.011+0000 7f4a3fc62640 1 -- 172.21.15.67:0/2061394952 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f4a240b65d0 msgr2=0x7f4a240b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:19.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:19.011+0000 7f4a3fc62640 1 --2- 172.21.15.67:0/2061394952 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f4a240b65d0 0x7f4a240b8ae0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4a38071170 tx=0x7f4a3400b390 comp rx=0 tx=0).stop 2024-09-21T06:35:19.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:19.011+0000 7f4a3fc62640 1 -- 172.21.15.67:0/2061394952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a3806f0d0 msgr2=0x7f4a381a9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:19.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:19.011+0000 7f4a3fc62640 1 --2- 172.21.15.67:0/2061394952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a3806f0d0 0x7f4a381a9210 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c02a930 tx=0x7f4a2c003c20 comp rx=0 tx=0).stop 2024-09-21T06:35:19.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:19.011+0000 7f4a3fc62640 1 -- 172.21.15.67:0/2061394952 shutdown_connections 2024-09-21T06:35:19.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:19.011+0000 7f4a3fc62640 1 --2- 172.21.15.67:0/2061394952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a3811cf80 0x7f4a381aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:19.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:19.011+0000 7f4a3fc62640 1 --2- 172.21.15.67:0/2061394952 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f4a240b65d0 0x7f4a240b8ae0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:19.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:19.011+0000 7f4a3fc62640 1 --2- 172.21.15.67:0/2061394952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a3806fa20 0x7f4a381a97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:19.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:19.011+0000 7f4a3fc62640 1 --2- 172.21.15.67:0/2061394952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a3806f0d0 0x7f4a381a9210 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:19.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:19.011+0000 7f4a3fc62640 1 -- 172.21.15.67:0/2061394952 >> 172.21.15.67:0/2061394952 conn(0x7f4a3810a3c0 msgr2=0x7f4a3811efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:19.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:19.011+0000 7f4a3fc62640 1 -- 172.21.15.67:0/2061394952 shutdown_connections 2024-09-21T06:35:19.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:19.011+0000 7f4a3fc62640 1 -- 172.21.15.67:0/2061394952 wait complete. 2024-09-21T06:35:19.291 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:19 smithi067 ceph-mon[31232]: pgmap v320: 129 pgs: 129 active+clean; 603 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.4 KiB/s wr, 3 op/s 2024-09-21T06:35:19.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:19 smithi203 ceph-mon[33735]: pgmap v320: 129 pgs: 129 active+clean; 603 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.4 KiB/s wr, 3 op/s 2024-09-21T06:35:19.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:19 smithi100 ceph-mon[33401]: pgmap v320: 129 pgs: 129 active+clean; 603 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.4 KiB/s wr, 3 op/s 2024-09-21T06:35:19.685 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:35:19.946 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:20.349 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:20 smithi067 ceph-mon[31232]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:35:20.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.439+0000 7f2bb49b0640 1 -- 172.21.15.67:0/3926776917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bb01094f0 msgr2=0x7f2bb0109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:20.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.439+0000 7f2bb49b0640 1 --2- 172.21.15.67:0/3926776917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bb01094f0 0x7f2bb0109900 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009900 tx=0x7f2ba402a3f0 comp rx=0 tx=0).stop 2024-09-21T06:35:20.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.440+0000 7f2bb49b0640 1 -- 172.21.15.67:0/3926776917 shutdown_connections 2024-09-21T06:35:20.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.440+0000 7f2bb49b0640 1 --2- 172.21.15.67:0/3926776917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bb0109e40 0x7f2bb0110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:20.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.440+0000 7f2bb49b0640 1 --2- 172.21.15.67:0/3926776917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bb01094f0 0x7f2bb0109900 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:20.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.440+0000 7f2bb49b0640 1 --2- 172.21.15.67:0/3926776917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bb0103bb0 0x7f2bb0103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:20.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.440+0000 7f2bb49b0640 1 -- 172.21.15.67:0/3926776917 >> 172.21.15.67:0/3926776917 conn(0x7f2bb00ff600 msgr2=0x7f2bb0101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:20.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.440+0000 7f2bb49b0640 1 -- 172.21.15.67:0/3926776917 shutdown_connections 2024-09-21T06:35:20.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.440+0000 7f2bb49b0640 1 -- 172.21.15.67:0/3926776917 wait complete. 2024-09-21T06:35:20.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.441+0000 7f2bb49b0640 1 Processor -- start 2024-09-21T06:35:20.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.441+0000 7f2bb49b0640 1 -- start start 2024-09-21T06:35:20.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.441+0000 7f2bb49b0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bb0103bb0 0x7f2bb0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:20.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.441+0000 7f2bb49b0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bb01094f0 0x7f2bb0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:20.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.441+0000 7f2bb49b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bb0109e40 0x7f2bb0076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:20.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.441+0000 7f2bb49b0640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb01139b0 con 0x7f2bb01094f0 2024-09-21T06:35:20.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.441+0000 7f2bb49b0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb0113830 con 0x7f2bb0109e40 2024-09-21T06:35:20.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.441+0000 7f2bb49b0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb0113b30 con 0x7f2bb0103bb0 2024-09-21T06:35:20.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.441+0000 7f2badd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bb01094f0 0x7f2bb0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:20.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.441+0000 7f2badd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bb01094f0 0x7f2bb0075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54810/0 (socket says 172.21.15.67:54810) 2024-09-21T06:35:20.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.441+0000 7f2badd9b640 1 -- 172.21.15.67:0/1005278105 learned_addr learned my addr 172.21.15.67:0/1005278105 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:20.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.441+0000 7f2baed9d640 1 --2- 172.21.15.67:0/1005278105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bb0109e40 0x7f2bb0076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:20.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.441+0000 7f2bae59c640 1 --2- 172.21.15.67:0/1005278105 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bb0103bb0 0x7f2bb0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:20.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.441+0000 7f2badd9b640 1 -- 172.21.15.67:0/1005278105 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bb0103bb0 msgr2=0x7f2bb0075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:20.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.441+0000 7f2badd9b640 1 --2- 172.21.15.67:0/1005278105 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bb0103bb0 0x7f2bb0075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:20.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.441+0000 7f2badd9b640 1 -- 172.21.15.67:0/1005278105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bb0109e40 msgr2=0x7f2bb0076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:20.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.441+0000 7f2badd9b640 1 --2- 172.21.15.67:0/1005278105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bb0109e40 0x7f2bb0076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:20.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.441+0000 7f2badd9b640 1 -- 172.21.15.67:0/1005278105 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb01a86a0 con 0x7f2bb01094f0 2024-09-21T06:35:20.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.442+0000 7f2badd9b640 1 --2- 172.21.15.67:0/1005278105 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bb01094f0 0x7f2bb0075c50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4005940 tx=0x7f2ba40040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:20.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.442+0000 7f2b9f7fe640 1 -- 172.21.15.67:0/1005278105 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba403f030 con 0x7f2bb01094f0 2024-09-21T06:35:20.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.442+0000 7f2b9f7fe640 1 -- 172.21.15.67:0/1005278105 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ba4003a80 con 0x7f2bb01094f0 2024-09-21T06:35:20.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.442+0000 7f2bb49b0640 1 -- 172.21.15.67:0/1005278105 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bb01a8930 con 0x7f2bb01094f0 2024-09-21T06:35:20.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.442+0000 7f2b9f7fe640 1 -- 172.21.15.67:0/1005278105 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4035070 con 0x7f2bb01094f0 2024-09-21T06:35:20.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.442+0000 7f2bb49b0640 1 -- 172.21.15.67:0/1005278105 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bb01a8e60 con 0x7f2bb01094f0 2024-09-21T06:35:20.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.443+0000 7f2bb49b0640 1 -- 172.21.15.67:0/1005278105 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bb0103fc0 con 0x7f2bb01094f0 2024-09-21T06:35:20.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.445+0000 7f2b9f7fe640 1 -- 172.21.15.67:0/1005278105 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f2ba4042030 con 0x7f2bb01094f0 2024-09-21T06:35:20.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.445+0000 7f2b9f7fe640 1 --2- 172.21.15.67:0/1005278105 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f2b980b6620 0x7f2b980b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:20.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.445+0000 7f2bae59c640 1 --2- 172.21.15.67:0/1005278105 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f2b980b6620 0x7f2b980b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:20.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.445+0000 7f2b9f7fe640 1 -- 172.21.15.67:0/1005278105 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f2ba4108150 con 0x7f2bb01094f0 2024-09-21T06:35:20.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.446+0000 7f2bae59c640 1 --2- 172.21.15.67:0/1005278105 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f2b980b6620 0x7f2b980b8b30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0005fd0 tx=0x7f2ba0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:20.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.449+0000 7f2b9f7fe640 1 -- 172.21.15.67:0/1005278105 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2ba40ce520 con 0x7f2bb01094f0 2024-09-21T06:35:20.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:20 smithi203 ceph-mon[33735]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:35:20.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:20 smithi100 ceph-mon[33401]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:35:20.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.677+0000 7f2bb49b0640 1 -- 172.21.15.67:0/1005278105 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bb0076a10 con 0x7f2bb01094f0 2024-09-21T06:35:20.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.678+0000 7f2b9f7fe640 1 -- 172.21.15.67:0/1005278105 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7f2ba40d2900 con 0x7f2bb01094f0 2024-09-21T06:35:20.681 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:35:20.681 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:35:20.681 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:35:20.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.681+0000 7f2bb49b0640 1 -- 172.21.15.67:0/1005278105 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f2b980b6620 msgr2=0x7f2b980b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:20.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.681+0000 7f2bb49b0640 1 --2- 172.21.15.67:0/1005278105 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f2b980b6620 0x7f2b980b8b30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0005fd0 tx=0x7f2ba0005850 comp rx=0 tx=0).stop 2024-09-21T06:35:20.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.681+0000 7f2bb49b0640 1 -- 172.21.15.67:0/1005278105 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bb01094f0 msgr2=0x7f2bb0075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:20.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.681+0000 7f2bb49b0640 1 --2- 172.21.15.67:0/1005278105 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bb01094f0 0x7f2bb0075c50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4005940 tx=0x7f2ba40040e0 comp rx=0 tx=0).stop 2024-09-21T06:35:20.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.681+0000 7f2bb49b0640 1 -- 172.21.15.67:0/1005278105 shutdown_connections 2024-09-21T06:35:20.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.681+0000 7f2bb49b0640 1 --2- 172.21.15.67:0/1005278105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bb0109e40 0x7f2bb0076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:20.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.681+0000 7f2bb49b0640 1 --2- 172.21.15.67:0/1005278105 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f2b980b6620 0x7f2b980b8b30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:20.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.681+0000 7f2bb49b0640 1 --2- 172.21.15.67:0/1005278105 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bb01094f0 0x7f2bb0075c50 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:20.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.681+0000 7f2bb49b0640 1 --2- 172.21.15.67:0/1005278105 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bb0103bb0 0x7f2bb0075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:20.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.681+0000 7f2bb49b0640 1 -- 172.21.15.67:0/1005278105 >> 172.21.15.67:0/1005278105 conn(0x7f2bb00ff600 msgr2=0x7f2bb0101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:20.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.681+0000 7f2bb49b0640 1 -- 172.21.15.67:0/1005278105 shutdown_connections 2024-09-21T06:35:20.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:20.681+0000 7f2bb49b0640 1 -- 172.21.15.67:0/1005278105 wait complete. 2024-09-21T06:35:21.356 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:21 smithi067 ceph-mon[31232]: pgmap v321: 129 pgs: 129 active+clean; 603 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.4 KiB/s wr, 3 op/s 2024-09-21T06:35:21.356 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:21 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1005278105' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:35:21.385 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:35:21.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:21 smithi203 ceph-mon[33735]: pgmap v321: 129 pgs: 129 active+clean; 603 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.4 KiB/s wr, 3 op/s 2024-09-21T06:35:21.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:21 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1005278105' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:35:21.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:21 smithi100 ceph-mon[33401]: pgmap v321: 129 pgs: 129 active+clean; 603 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.4 KiB/s wr, 3 op/s 2024-09-21T06:35:21.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:21 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1005278105' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:35:21.643 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:22.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.128+0000 7f383fa97640 1 -- 172.21.15.67:0/2606892162 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3838102a80 msgr2=0x7f3838102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:22.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.128+0000 7f383fa97640 1 --2- 172.21.15.67:0/2606892162 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3838102a80 0x7f3838102e90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3834009900 tx=0x7f383402a410 comp rx=0 tx=0).stop 2024-09-21T06:35:22.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.128+0000 7f383fa97640 1 -- 172.21.15.67:0/2606892162 shutdown_connections 2024-09-21T06:35:22.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.128+0000 7f383fa97640 1 --2- 172.21.15.67:0/2606892162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38381083c0 0x7f38381087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:22.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.128+0000 7f383fa97640 1 --2- 172.21.15.67:0/2606892162 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3838108d10 0x7f383810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:22.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.128+0000 7f383fa97640 1 --2- 172.21.15.67:0/2606892162 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3838102a80 0x7f3838102e90 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:22.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.128+0000 7f383fa97640 1 -- 172.21.15.67:0/2606892162 >> 172.21.15.67:0/2606892162 conn(0x7f3838079740 msgr2=0x7f3838079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:22.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.128+0000 7f383fa97640 1 -- 172.21.15.67:0/2606892162 shutdown_connections 2024-09-21T06:35:22.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.128+0000 7f383fa97640 1 -- 172.21.15.67:0/2606892162 wait complete. 2024-09-21T06:35:22.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.129+0000 7f383fa97640 1 Processor -- start 2024-09-21T06:35:22.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.129+0000 7f383fa97640 1 -- start start 2024-09-21T06:35:22.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.129+0000 7f383fa97640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3838102a80 0x7f383819a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:22.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.129+0000 7f383fa97640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38381083c0 0x7f383819a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:22.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.129+0000 7f383fa97640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3838108d10 0x7f383819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:22.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.129+0000 7f383fa97640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3838113980 con 0x7f38381083c0 2024-09-21T06:35:22.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.129+0000 7f383fa97640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3838113800 con 0x7f3838108d10 2024-09-21T06:35:22.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.129+0000 7f383fa97640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3838113b00 con 0x7f3838102a80 2024-09-21T06:35:22.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.129+0000 7f383d032640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38381083c0 0x7f383819a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:22.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.129+0000 7f383d833640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3838102a80 0x7f383819a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:22.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.129+0000 7f383d032640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38381083c0 0x7f383819a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54828/0 (socket says 172.21.15.67:54828) 2024-09-21T06:35:22.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.129+0000 7f383d032640 1 -- 172.21.15.67:0/4141321059 learned_addr learned my addr 172.21.15.67:0/4141321059 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:22.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.129+0000 7f383e034640 1 --2- 172.21.15.67:0/4141321059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3838108d10 0x7f383819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:22.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.130+0000 7f383d032640 1 -- 172.21.15.67:0/4141321059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3838102a80 msgr2=0x7f383819a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:22.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.130+0000 7f383d032640 1 --2- 172.21.15.67:0/4141321059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3838102a80 0x7f383819a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:22.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.130+0000 7f383d032640 1 -- 172.21.15.67:0/4141321059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3838108d10 msgr2=0x7f383819fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:22.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.130+0000 7f383d032640 1 --2- 172.21.15.67:0/4141321059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3838108d10 0x7f383819fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:22.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.130+0000 7f383d032640 1 -- 172.21.15.67:0/4141321059 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38381a0260 con 0x7f38381083c0 2024-09-21T06:35:22.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.130+0000 7f383d032640 1 --2- 172.21.15.67:0/4141321059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38381083c0 0x7f383819a950 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f382c00c970 tx=0x7f382c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:22.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.130+0000 7f382affd640 1 -- 172.21.15.67:0/4141321059 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c007bb0 con 0x7f38381083c0 2024-09-21T06:35:22.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.130+0000 7f383fa97640 1 -- 172.21.15.67:0/4141321059 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38381a0550 con 0x7f38381083c0 2024-09-21T06:35:22.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.130+0000 7f382affd640 1 -- 172.21.15.67:0/4141321059 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f382c007d50 con 0x7f38381083c0 2024-09-21T06:35:22.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.130+0000 7f382affd640 1 -- 172.21.15.67:0/4141321059 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c004e10 con 0x7f38381083c0 2024-09-21T06:35:22.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.130+0000 7f383fa97640 1 -- 172.21.15.67:0/4141321059 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38381a6d70 con 0x7f38381083c0 2024-09-21T06:35:22.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.131+0000 7f383fa97640 1 -- 172.21.15.67:0/4141321059 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3838103d00 con 0x7f38381083c0 2024-09-21T06:35:22.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.133+0000 7f382affd640 1 -- 172.21.15.67:0/4141321059 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f382c004fb0 con 0x7f38381083c0 2024-09-21T06:35:22.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.134+0000 7f382affd640 1 --2- 172.21.15.67:0/4141321059 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f38240b65d0 0x7f38240b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:22.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.134+0000 7f383d833640 1 --2- 172.21.15.67:0/4141321059 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f38240b65d0 0x7f38240b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:22.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.134+0000 7f382affd640 1 -- 172.21.15.67:0/4141321059 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f382c0e7840 con 0x7f38381083c0 2024-09-21T06:35:22.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.134+0000 7f383d833640 1 --2- 172.21.15.67:0/4141321059 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f38240b65d0 0x7f38240b8ae0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3834006010 tx=0x7f3834000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:22.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.137+0000 7f382affd640 1 -- 172.21.15.67:0/4141321059 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f382c014030 con 0x7f38381083c0 2024-09-21T06:35:22.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.329+0000 7f383fa97640 1 -- 172.21.15.67:0/4141321059 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3838102e90 con 0x7f38381083c0 2024-09-21T06:35:22.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.330+0000 7f382affd640 1 -- 172.21.15.67:0/4141321059 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+24709 (secure 0 0 0) 0x7f3838102e90 con 0x7f38381083c0 2024-09-21T06:35:22.333 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:35:22.333 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":83,"fsid":"2a791768-77e2-11ef-baf5-efdc52797490","created":"2024-09-21T06:26:15.222283+0000","modified":"2024-09-21T06:35:04.990843+0000","last_up_change":"2024-09-21T06:33:40.886310+0000","last_in_change":"2024-09-21T06:33:17.339486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:29:50.212825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:34:29.082881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:34:31.301094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:34:37.681356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"18778364-82ed-41c0-92da-ce44cdad7358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6803","nonce":2310759946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6805","nonce":2310759946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6809","nonce":2310759946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6807","nonce":2310759946}]},"public_addr":"172.21.15.67:6803/2310759946","cluster_addr":"172.21.15.67:6805/2310759946","heartbeat_back_addr":"172.21.15.67:6809/2310759946","heartbeat_front_addr":"172.21.15.67:6807/2310759946","state":["exists","up"]},{"osd":1,"uuid":"84e8d0c2-1c45-4673-ad49-f61df667d7f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6801","nonce":161807364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6803","nonce":161807364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6807","nonce":161807364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6805","nonce":161807364}]},"public_addr":"172.21.15.100:6801/161807364","cluster_addr":"172.21.15.100:6803/161807364","heartbeat_back_addr":"172.21.15.100:6807/161807364","heartbeat_front_addr":"172.21.15.100:6805/161807364","state":["exists","up"]},{"osd":2,"uuid":"e9a4d919-a779-4a1b-9b5b-d03a65f28ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6801","nonce":2754787169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6803","nonce":2754787169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6807","nonce":2754787169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6805","nonce":2754787169}]},"public_addr":"172.21.15.203:6801/2754787169","cluster_addr":"172.21.15.203:6803/2754787169","heartbeat_back_addr":"172.21.15.203:6807/2754787169","heartbeat_front_addr":"172.21.15.203:6805/2754787169","state":["exists","up"]},{"osd":3,"uuid":"3f7d9abd-587c-47ad-bf43-11f7dcdf71c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6811","nonce":4007517951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6813","nonce":4007517951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6817","nonce":4007517951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6815","nonce":4007517951}]},"public_addr":"172.21.15.67:6811/4007517951","cluster_addr":"172.21.15.67:6813/4007517951","heartbeat_back_addr":"172.21.15.67:6817/4007517951","heartbeat_front_addr":"172.21.15.67:6815/4007517951","state":["exists","up"]},{"osd":4,"uuid":"94f3a075-6d8a-4907-8418-a8513b35779a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6809","nonce":3773646501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6811","nonce":3773646501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6815","nonce":3773646501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6813","nonce":3773646501}]},"public_addr":"172.21.15.100:6809/3773646501","cluster_addr":"172.21.15.100:6811/3773646501","heartbeat_back_addr":"172.21.15.100:6815/3773646501","heartbeat_front_addr":"172.21.15.100:6813/3773646501","state":["exists","up"]},{"osd":5,"uuid":"5d842edf-06ad-49ae-ac71-863e5657270c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6809","nonce":2096171166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6811","nonce":2096171166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6815","nonce":2096171166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6813","nonce":2096171166}]},"public_addr":"172.21.15.203:6809/2096171166","cluster_addr":"172.21.15.203:6811/2096171166","heartbeat_back_addr":"172.21.15.203:6815/2096171166","heartbeat_front_addr":"172.21.15.203:6813/2096171166","state":["exists","up"]},{"osd":6,"uuid":"85b9beed-c93d-48ba-bc7a-c4cd816fbdff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6819","nonce":3229936280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6821","nonce":3229936280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6825","nonce":3229936280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6823","nonce":3229936280}]},"public_addr":"172.21.15.67:6819/3229936280","cluster_addr":"172.21.15.67:6821/3229936280","heartbeat_back_addr":"172.21.15.67:6825/3229936280","heartbeat_front_addr":"172.21.15.67:6823/3229936280","state":["exists","up"]},{"osd":7,"uuid":"86de30c2-b8ee-4c9d-9c17-8a56517934ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6817","nonce":1622917909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6819","nonce":1622917909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6823","nonce":1622917909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6821","nonce":1622917909}]},"public_addr":"172.21.15.100:6817/1622917909","cluster_addr":"172.21.15.100:6819/1622917909","heartbeat_back_addr":"172.21.15.100:6823/1622917909","heartbeat_front_addr":"172.21.15.100:6821/1622917909","state":["exists","up"]},{"osd":8,"uuid":"0e96cdad-a135-4115-9e2d-9a28306700e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6817","nonce":3452445943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6819","nonce":3452445943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6823","nonce":3452445943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6821","nonce":3452445943}]},"public_addr":"172.21.15.203:6817/3452445943","cluster_addr":"172.21.15.203:6819/3452445943","heartbeat_back_addr":"172.21.15.203:6823/3452445943","heartbeat_front_addr":"172.21.15.203:6821/3452445943","state":["exists","up"]},{"osd":9,"uuid":"95704ed6-6286-4094-9c6d-23919c7005ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6827","nonce":2998307533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6829","nonce":2998307533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6833","nonce":2998307533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6831","nonce":2998307533}]},"public_addr":"172.21.15.67:6827/2998307533","cluster_addr":"172.21.15.67:6829/2998307533","heartbeat_back_addr":"172.21.15.67:6833/2998307533","heartbeat_front_addr":"172.21.15.67:6831/2998307533","state":["exists","up"]},{"osd":10,"uuid":"bd2a7d33-b935-46c8-95c3-d76f28242ffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6825","nonce":2819057142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6827","nonce":2819057142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6831","nonce":2819057142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6829","nonce":2819057142}]},"public_addr":"172.21.15.100:6825/2819057142","cluster_addr":"172.21.15.100:6827/2819057142","heartbeat_back_addr":"172.21.15.100:6831/2819057142","heartbeat_front_addr":"172.21.15.100:6829/2819057142","state":["exists","up"]},{"osd":11,"uuid":"ea591507-10e2-4c08-8d29-d38ba9be96ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6825","nonce":566502600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6827","nonce":566502600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6831","nonce":566502600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6829","nonce":566502600}]},"public_addr":"172.21.15.203:6825/566502600","cluster_addr":"172.21.15.203:6827/566502600","heartbeat_back_addr":"172.21.15.203:6831/566502600","heartbeat_front_addr":"172.21.15.203:6829/566502600","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:28:58.945733+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:29:23.597118+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:29:48.303563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:30:14.139139+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:30:39.021508+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:04.981241+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:30.853841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:56.452967+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:32:22.239036+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:32:47.998803+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:33:13.850759+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:33:39.282150+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6800/1181476774":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/3100321138":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/2309364298":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/3458717076":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/4110206171":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6801/1181476774":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/2910038982":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6801/2622729103":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6800/2622729103":"2024-09-22T06:26:37.041377+0000","172.21.15.67:0/3346784528":"2024-09-22T06:26:37.041377+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:35:22.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.333+0000 7f383fa97640 1 -- 172.21.15.67:0/4141321059 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f38240b65d0 msgr2=0x7f38240b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:22.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.333+0000 7f383fa97640 1 --2- 172.21.15.67:0/4141321059 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f38240b65d0 0x7f38240b8ae0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3834006010 tx=0x7f3834000bc0 comp rx=0 tx=0).stop 2024-09-21T06:35:22.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.333+0000 7f383fa97640 1 -- 172.21.15.67:0/4141321059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38381083c0 msgr2=0x7f383819a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:22.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.333+0000 7f383fa97640 1 --2- 172.21.15.67:0/4141321059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38381083c0 0x7f383819a950 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f382c00c970 tx=0x7f382c00ce30 comp rx=0 tx=0).stop 2024-09-21T06:35:22.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.333+0000 7f383fa97640 1 -- 172.21.15.67:0/4141321059 shutdown_connections 2024-09-21T06:35:22.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.333+0000 7f383fa97640 1 --2- 172.21.15.67:0/4141321059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3838108d10 0x7f383819fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:22.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.333+0000 7f383fa97640 1 --2- 172.21.15.67:0/4141321059 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f38240b65d0 0x7f38240b8ae0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:22.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.333+0000 7f383fa97640 1 --2- 172.21.15.67:0/4141321059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38381083c0 0x7f383819a950 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:22.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.333+0000 7f383fa97640 1 --2- 172.21.15.67:0/4141321059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3838102a80 0x7f383819a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:22.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.333+0000 7f383fa97640 1 -- 172.21.15.67:0/4141321059 >> 172.21.15.67:0/4141321059 conn(0x7f3838079740 msgr2=0x7f38381015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:22.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.333+0000 7f383fa97640 1 -- 172.21.15.67:0/4141321059 shutdown_connections 2024-09-21T06:35:22.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:22.333+0000 7f383fa97640 1 -- 172.21.15.67:0/4141321059 wait complete. 2024-09-21T06:35:23.062 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T06:35:23.062 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T06:35:23.291 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:23 smithi067 ceph-mon[31232]: pgmap v322: 129 pgs: 129 active+clean; 625 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.6 KiB/s wr, 5 op/s 2024-09-21T06:35:23.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:23 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/4141321059' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:35:23.317 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:23.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:23 smithi203 ceph-mon[33735]: pgmap v322: 129 pgs: 129 active+clean; 625 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.6 KiB/s wr, 5 op/s 2024-09-21T06:35:23.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:23 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/4141321059' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:35:23.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:23 smithi100 ceph-mon[33401]: pgmap v322: 129 pgs: 129 active+clean; 625 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.6 KiB/s wr, 5 op/s 2024-09-21T06:35:23.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:23 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/4141321059' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:35:23.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.796+0000 7f9c58f40640 1 -- 172.21.15.67:0/1471078928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c54103bb0 msgr2=0x7f9c54103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:23.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.796+0000 7f9c58f40640 1 --2- 172.21.15.67:0/1471078928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c54103bb0 0x7f9c54103fc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009900 tx=0x7f9c4402a3f0 comp rx=0 tx=0).stop 2024-09-21T06:35:23.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.796+0000 7f9c58f40640 1 -- 172.21.15.67:0/1471078928 shutdown_connections 2024-09-21T06:35:23.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.796+0000 7f9c58f40640 1 --2- 172.21.15.67:0/1471078928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c541094f0 0x7f9c54109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:23.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.796+0000 7f9c58f40640 1 --2- 172.21.15.67:0/1471078928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c54109e40 0x7f9c54110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:23.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.796+0000 7f9c58f40640 1 --2- 172.21.15.67:0/1471078928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c54103bb0 0x7f9c54103fc0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:23.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.796+0000 7f9c58f40640 1 -- 172.21.15.67:0/1471078928 >> 172.21.15.67:0/1471078928 conn(0x7f9c540ff600 msgr2=0x7f9c54101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:23.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.796+0000 7f9c58f40640 1 -- 172.21.15.67:0/1471078928 shutdown_connections 2024-09-21T06:35:23.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.796+0000 7f9c58f40640 1 -- 172.21.15.67:0/1471078928 wait complete. 2024-09-21T06:35:23.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.797+0000 7f9c58f40640 1 Processor -- start 2024-09-21T06:35:23.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.797+0000 7f9c58f40640 1 -- start start 2024-09-21T06:35:23.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.797+0000 7f9c58f40640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c54103bb0 0x7f9c5419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:23.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.797+0000 7f9c58f40640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c541094f0 0x7f9c5419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:23.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.797+0000 7f9c58f40640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c54109e40 0x7f9c5419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:23.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.797+0000 7f9c58f40640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9c541139b0 con 0x7f9c541094f0 2024-09-21T06:35:23.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.797+0000 7f9c58f40640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9c54113830 con 0x7f9c54109e40 2024-09-21T06:35:23.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.797+0000 7f9c58f40640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9c54113b30 con 0x7f9c54103bb0 2024-09-21T06:35:23.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.797+0000 7f9c51d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c541094f0 0x7f9c5419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:23.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.797+0000 7f9c51d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c541094f0 0x7f9c5419a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54846/0 (socket says 172.21.15.67:54846) 2024-09-21T06:35:23.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.797+0000 7f9c51d9b640 1 -- 172.21.15.67:0/3886324495 learned_addr learned my addr 172.21.15.67:0/3886324495 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:23.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.798+0000 7f9c52d9d640 1 --2- 172.21.15.67:0/3886324495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c54109e40 0x7f9c5419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:23.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.798+0000 7f9c5259c640 1 --2- 172.21.15.67:0/3886324495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c54103bb0 0x7f9c5419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:23.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.798+0000 7f9c51d9b640 1 -- 172.21.15.67:0/3886324495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c54103bb0 msgr2=0x7f9c5419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:23.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.798+0000 7f9c51d9b640 1 --2- 172.21.15.67:0/3886324495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c54103bb0 0x7f9c5419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:23.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.798+0000 7f9c51d9b640 1 -- 172.21.15.67:0/3886324495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c54109e40 msgr2=0x7f9c5419fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:23.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.798+0000 7f9c51d9b640 1 --2- 172.21.15.67:0/3886324495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c54109e40 0x7f9c5419fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:23.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.798+0000 7f9c51d9b640 1 -- 172.21.15.67:0/3886324495 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c541a02d0 con 0x7f9c541094f0 2024-09-21T06:35:23.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.798+0000 7f9c5259c640 1 --2- 172.21.15.67:0/3886324495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c54103bb0 0x7f9c5419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:23.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.798+0000 7f9c52d9d640 1 --2- 172.21.15.67:0/3886324495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c54109e40 0x7f9c5419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:23.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.798+0000 7f9c51d9b640 1 --2- 172.21.15.67:0/3886324495 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c541094f0 0x7f9c5419a9c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9c4800c8c0 tx=0x7f9c4800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:23.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.798+0000 7f9c437fe640 1 -- 172.21.15.67:0/3886324495 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c48007bb0 con 0x7f9c541094f0 2024-09-21T06:35:23.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.798+0000 7f9c437fe640 1 -- 172.21.15.67:0/3886324495 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c48007d50 con 0x7f9c541094f0 2024-09-21T06:35:23.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.798+0000 7f9c58f40640 1 -- 172.21.15.67:0/3886324495 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c541a05c0 con 0x7f9c541094f0 2024-09-21T06:35:23.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.798+0000 7f9c437fe640 1 -- 172.21.15.67:0/3886324495 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c480186a0 con 0x7f9c541094f0 2024-09-21T06:35:23.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.798+0000 7f9c58f40640 1 -- 172.21.15.67:0/3886324495 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c541a6de0 con 0x7f9c541094f0 2024-09-21T06:35:23.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.799+0000 7f9c58f40640 1 -- 172.21.15.67:0/3886324495 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c54103fc0 con 0x7f9c541094f0 2024-09-21T06:35:23.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.801+0000 7f9c437fe640 1 -- 172.21.15.67:0/3886324495 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f9c4800fa00 con 0x7f9c541094f0 2024-09-21T06:35:23.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.802+0000 7f9c437fe640 1 --2- 172.21.15.67:0/3886324495 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9c3c0b65d0 0x7f9c3c0b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:23.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.802+0000 7f9c5259c640 1 --2- 172.21.15.67:0/3886324495 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9c3c0b65d0 0x7f9c3c0b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:23.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.802+0000 7f9c437fe640 1 -- 172.21.15.67:0/3886324495 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f9c480e7bd0 con 0x7f9c541094f0 2024-09-21T06:35:23.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.803+0000 7f9c5259c640 1 --2- 172.21.15.67:0/3886324495 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9c3c0b65d0 0x7f9c3c0b8ae0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9c44000c00 tx=0x7f9c44005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:23.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.805+0000 7f9c437fe640 1 -- 172.21.15.67:0/3886324495 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9c48014030 con 0x7f9c541094f0 2024-09-21T06:35:23.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.991+0000 7f9c58f40640 1 -- 172.21.15.67:0/3886324495 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9c54063800 con 0x7f9c3c0b65d0 2024-09-21T06:35:24.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:23.998+0000 7f9c437fe640 1 -- 172.21.15.67:0/3886324495 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340526 (secure 0 0 0) 0x7f9c54063800 con 0x7f9c3c0b65d0 2024-09-21T06:35:24.003 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:35:24.004 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-09-21T06:35:24.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:24.001+0000 7f9c58f40640 1 -- 172.21.15.67:0/3886324495 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9c3c0b65d0 msgr2=0x7f9c3c0b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:24.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:24.001+0000 7f9c58f40640 1 --2- 172.21.15.67:0/3886324495 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9c3c0b65d0 0x7f9c3c0b8ae0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9c44000c00 tx=0x7f9c44005a20 comp rx=0 tx=0).stop 2024-09-21T06:35:24.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:24.001+0000 7f9c58f40640 1 -- 172.21.15.67:0/3886324495 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c541094f0 msgr2=0x7f9c5419a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:24.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:24.001+0000 7f9c58f40640 1 --2- 172.21.15.67:0/3886324495 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c541094f0 0x7f9c5419a9c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9c4800c8c0 tx=0x7f9c4800cd80 comp rx=0 tx=0).stop 2024-09-21T06:35:24.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:24.001+0000 7f9c58f40640 1 -- 172.21.15.67:0/3886324495 shutdown_connections 2024-09-21T06:35:24.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:24.001+0000 7f9c58f40640 1 --2- 172.21.15.67:0/3886324495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c54109e40 0x7f9c5419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:24.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:24.001+0000 7f9c58f40640 1 --2- 172.21.15.67:0/3886324495 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9c3c0b65d0 0x7f9c3c0b8ae0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:24.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:24.001+0000 7f9c58f40640 1 --2- 172.21.15.67:0/3886324495 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c541094f0 0x7f9c5419a9c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:24.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:24.001+0000 7f9c58f40640 1 --2- 172.21.15.67:0/3886324495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c54103bb0 0x7f9c5419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:24.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:24.001+0000 7f9c58f40640 1 -- 172.21.15.67:0/3886324495 >> 172.21.15.67:0/3886324495 conn(0x7f9c540ff600 msgr2=0x7f9c54101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:24.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:24.002+0000 7f9c58f40640 1 -- 172.21.15.67:0/3886324495 shutdown_connections 2024-09-21T06:35:24.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:24.002+0000 7f9c58f40640 1 -- 172.21.15.67:0/3886324495 wait complete. 2024-09-21T06:35:24.728 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-09-21T06:35:23.849362+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":640019,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":113,"num_read_kb":133,"num_write":196,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364388,"kb_used_data":10964,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577436,"statfs":{"total":1151940427776,"available":1151567294464,"internally_reserved":0,"allocated":11227136,"data_stored":7503887,"data_compressed":390735,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":45011,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":56,"num_write":24,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003431"},"pg_stats":[{"pgid":"2.1f","version":"75'4","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.114370+0000","last_change":"2024-09-21T06:34:30.633366+0000","last_active":"2024-09-21T06:35:05.114370+0000","last_peered":"2024-09-21T06:35:05.114370+0000","last_clean":"2024-09-21T06:35:05.114370+0000","last_became_active":"2024-09-21T06:34:30.632929+0000","last_became_peered":"2024-09-21T06:34:30.632929+0000","last_unstale":"2024-09-21T06:35:05.114370+0000","last_undegraded":"2024-09-21T06:35:05.114370+0000","last_fullsized":"2024-09-21T06:35:05.114370+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:47.722461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.272950+0000","last_change":"2024-09-21T06:34:32.632989+0000","last_active":"2024-09-21T06:35:05.272950+0000","last_peered":"2024-09-21T06:35:05.272950+0000","last_clean":"2024-09-21T06:35:05.272950+0000","last_became_active":"2024-09-21T06:34:32.632024+0000","last_became_peered":"2024-09-21T06:34:32.632024+0000","last_unstale":"2024-09-21T06:35:05.272950+0000","last_undegraded":"2024-09-21T06:35:05.272950+0000","last_fullsized":"2024-09-21T06:35:05.272950+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:13.669613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.274531+0000","last_change":"2024-09-21T06:35:05.274434+0000","last_active":"2024-09-21T06:35:05.274531+0000","last_peered":"2024-09-21T06:35:05.274531+0000","last_clean":"2024-09-21T06:35:05.274531+0000","last_became_active":"2024-09-21T06:35:05.273833+0000","last_became_peered":"2024-09-21T06:35:05.273833+0000","last_unstale":"2024-09-21T06:35:05.274531+0000","last_undegraded":"2024-09-21T06:35:05.274531+0000","last_fullsized":"2024-09-21T06:35:05.274531+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:48.847861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.273266+0000","last_change":"2024-09-21T06:34:30.946309+0000","last_active":"2024-09-21T06:35:05.273266+0000","last_peered":"2024-09-21T06:35:05.273266+0000","last_clean":"2024-09-21T06:35:05.273266+0000","last_became_active":"2024-09-21T06:34:30.945913+0000","last_became_peered":"2024-09-21T06:34:30.945913+0000","last_unstale":"2024-09-21T06:35:05.273266+0000","last_undegraded":"2024-09-21T06:35:05.273266+0000","last_fullsized":"2024-09-21T06:35:05.273266+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:21.676524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000669+0000","last_change":"2024-09-21T06:34:32.637636+0000","last_active":"2024-09-21T06:35:05.000669+0000","last_peered":"2024-09-21T06:35:05.000669+0000","last_clean":"2024-09-21T06:35:05.000669+0000","last_became_active":"2024-09-21T06:34:32.636534+0000","last_became_peered":"2024-09-21T06:34:32.636534+0000","last_unstale":"2024-09-21T06:35:05.000669+0000","last_undegraded":"2024-09-21T06:35:05.000669+0000","last_fullsized":"2024-09-21T06:35:05.000669+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:05.802118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.274538+0000","last_change":"2024-09-21T06:35:05.274419+0000","last_active":"2024-09-21T06:35:05.274538+0000","last_peered":"2024-09-21T06:35:05.274538+0000","last_clean":"2024-09-21T06:35:05.274538+0000","last_became_active":"2024-09-21T06:35:05.273865+0000","last_became_peered":"2024-09-21T06:35:05.273865+0000","last_unstale":"2024-09-21T06:35:05.274538+0000","last_undegraded":"2024-09-21T06:35:05.274538+0000","last_fullsized":"2024-09-21T06:35:05.274538+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:04.490355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.004199+0000","last_change":"2024-09-21T06:34:30.636856+0000","last_active":"2024-09-21T06:35:05.004199+0000","last_peered":"2024-09-21T06:35:05.004199+0000","last_clean":"2024-09-21T06:35:05.004199+0000","last_became_active":"2024-09-21T06:34:30.636460+0000","last_became_peered":"2024-09-21T06:34:30.636460+0000","last_unstale":"2024-09-21T06:35:05.004199+0000","last_undegraded":"2024-09-21T06:35:05.004199+0000","last_fullsized":"2024-09-21T06:35:05.004199+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:58.917851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995796+0000","last_change":"2024-09-21T06:34:32.650303+0000","last_active":"2024-09-21T06:35:04.995796+0000","last_peered":"2024-09-21T06:35:04.995796+0000","last_clean":"2024-09-21T06:35:04.995796+0000","last_became_active":"2024-09-21T06:34:32.647189+0000","last_became_peered":"2024-09-21T06:34:32.647189+0000","last_unstale":"2024-09-21T06:35:04.995796+0000","last_undegraded":"2024-09-21T06:35:04.995796+0000","last_fullsized":"2024-09-21T06:35:04.995796+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:14.889384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.996942+0000","last_change":"2024-09-21T06:34:39.713139+0000","last_active":"2024-09-21T06:35:04.996942+0000","last_peered":"2024-09-21T06:35:04.996942+0000","last_clean":"2024-09-21T06:35:04.996942+0000","last_became_active":"2024-09-21T06:34:39.712379+0000","last_became_peered":"2024-09-21T06:34:39.712379+0000","last_unstale":"2024-09-21T06:35:04.996942+0000","last_undegraded":"2024-09-21T06:35:04.996942+0000","last_fullsized":"2024-09-21T06:35:04.996942+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:53.100529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.004039+0000","last_change":"2024-09-21T06:34:30.633850+0000","last_active":"2024-09-21T06:35:05.004039+0000","last_peered":"2024-09-21T06:35:05.004039+0000","last_clean":"2024-09-21T06:35:05.004039+0000","last_became_active":"2024-09-21T06:34:30.633468+0000","last_became_peered":"2024-09-21T06:34:30.633468+0000","last_unstale":"2024-09-21T06:35:05.004039+0000","last_undegraded":"2024-09-21T06:35:05.004039+0000","last_fullsized":"2024-09-21T06:35:05.004039+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:59.095803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.005361+0000","last_change":"2024-09-21T06:35:05.005240+0000","last_active":"2024-09-21T06:35:05.005361+0000","last_peered":"2024-09-21T06:35:05.005361+0000","last_clean":"2024-09-21T06:35:05.005361+0000","last_became_active":"2024-09-21T06:35:05.004849+0000","last_became_peered":"2024-09-21T06:35:05.004849+0000","last_unstale":"2024-09-21T06:35:05.005361+0000","last_undegraded":"2024-09-21T06:35:05.005361+0000","last_fullsized":"2024-09-21T06:35:05.005361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:28.849604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.274638+0000","last_change":"2024-09-21T06:35:05.274437+0000","last_active":"2024-09-21T06:35:05.274638+0000","last_peered":"2024-09-21T06:35:05.274638+0000","last_clean":"2024-09-21T06:35:05.274638+0000","last_became_active":"2024-09-21T06:35:05.273865+0000","last_became_peered":"2024-09-21T06:35:05.273865+0000","last_unstale":"2024-09-21T06:35:05.274638+0000","last_undegraded":"2024-09-21T06:35:05.274638+0000","last_fullsized":"2024-09-21T06:35:05.274638+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:43.339504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029725+0000","last_change":"2024-09-21T06:34:32.645630+0000","last_active":"2024-09-21T06:35:05.029725+0000","last_peered":"2024-09-21T06:35:05.029725+0000","last_clean":"2024-09-21T06:35:05.029725+0000","last_became_active":"2024-09-21T06:34:32.645237+0000","last_became_peered":"2024-09-21T06:34:32.645237+0000","last_unstale":"2024-09-21T06:35:05.029725+0000","last_undegraded":"2024-09-21T06:35:05.029725+0000","last_fullsized":"2024-09-21T06:35:05.029725+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:25.064318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.003627+0000","last_change":"2024-09-21T06:34:30.636609+0000","last_active":"2024-09-21T06:35:05.003627+0000","last_peered":"2024-09-21T06:35:05.003627+0000","last_clean":"2024-09-21T06:35:05.003627+0000","last_became_active":"2024-09-21T06:34:30.636217+0000","last_became_peered":"2024-09-21T06:34:30.636217+0000","last_unstale":"2024-09-21T06:35:05.003627+0000","last_undegraded":"2024-09-21T06:35:05.003627+0000","last_fullsized":"2024-09-21T06:35:05.003627+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:32.541307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997116+0000","last_change":"2024-09-21T06:34:39.714177+0000","last_active":"2024-09-21T06:35:04.997116+0000","last_peered":"2024-09-21T06:35:04.997116+0000","last_clean":"2024-09-21T06:35:04.997116+0000","last_became_active":"2024-09-21T06:34:39.713713+0000","last_became_peered":"2024-09-21T06:34:39.713713+0000","last_unstale":"2024-09-21T06:35:04.997116+0000","last_undegraded":"2024-09-21T06:35:04.997116+0000","last_fullsized":"2024-09-21T06:35:04.997116+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:07.042240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.272625+0000","last_change":"2024-09-21T06:34:32.636297+0000","last_active":"2024-09-21T06:35:05.272625+0000","last_peered":"2024-09-21T06:35:05.272625+0000","last_clean":"2024-09-21T06:35:05.272625+0000","last_became_active":"2024-09-21T06:34:32.635930+0000","last_became_peered":"2024-09-21T06:34:32.635930+0000","last_unstale":"2024-09-21T06:35:05.272625+0000","last_undegraded":"2024-09-21T06:35:05.272625+0000","last_fullsized":"2024-09-21T06:35:05.272625+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:43.091682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995838+0000","last_change":"2024-09-21T06:34:30.631405+0000","last_active":"2024-09-21T06:35:04.995838+0000","last_peered":"2024-09-21T06:35:04.995838+0000","last_clean":"2024-09-21T06:35:04.995838+0000","last_became_active":"2024-09-21T06:34:30.630512+0000","last_became_peered":"2024-09-21T06:34:30.630512+0000","last_unstale":"2024-09-21T06:35:04.995838+0000","last_undegraded":"2024-09-21T06:35:04.995838+0000","last_fullsized":"2024-09-21T06:35:04.995838+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:43.412695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"83'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997680+0000","last_change":"2024-09-21T06:34:39.710573+0000","last_active":"2024-09-21T06:35:04.997680+0000","last_peered":"2024-09-21T06:35:04.997680+0000","last_clean":"2024-09-21T06:35:04.997680+0000","last_became_active":"2024-09-21T06:34:39.708969+0000","last_became_peered":"2024-09-21T06:34:39.708969+0000","last_unstale":"2024-09-21T06:35:04.997680+0000","last_undegraded":"2024-09-21T06:35:04.997680+0000","last_fullsized":"2024-09-21T06:35:04.997680+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:34.847587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000559+0000","last_change":"2024-09-21T06:34:32.641085+0000","last_active":"2024-09-21T06:35:05.000559+0000","last_peered":"2024-09-21T06:35:05.000559+0000","last_clean":"2024-09-21T06:35:05.000559+0000","last_became_active":"2024-09-21T06:34:32.640723+0000","last_became_peered":"2024-09-21T06:34:32.640723+0000","last_unstale":"2024-09-21T06:35:05.000559+0000","last_undegraded":"2024-09-21T06:35:05.000559+0000","last_fullsized":"2024-09-21T06:35:05.000559+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:21.410262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.273157+0000","last_change":"2024-09-21T06:34:30.626719+0000","last_active":"2024-09-21T06:35:05.273157+0000","last_peered":"2024-09-21T06:35:05.273157+0000","last_clean":"2024-09-21T06:35:05.273157+0000","last_became_active":"2024-09-21T06:34:30.626312+0000","last_became_peered":"2024-09-21T06:34:30.626312+0000","last_unstale":"2024-09-21T06:35:05.273157+0000","last_undegraded":"2024-09-21T06:35:05.273157+0000","last_fullsized":"2024-09-21T06:35:05.273157+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:53.938914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.114020+0000","last_change":"2024-09-21T06:34:39.711172+0000","last_active":"2024-09-21T06:35:05.114020+0000","last_peered":"2024-09-21T06:35:05.114020+0000","last_clean":"2024-09-21T06:35:05.114020+0000","last_became_active":"2024-09-21T06:34:39.708424+0000","last_became_peered":"2024-09-21T06:34:39.708424+0000","last_unstale":"2024-09-21T06:35:05.114020+0000","last_undegraded":"2024-09-21T06:35:05.114020+0000","last_fullsized":"2024-09-21T06:35:05.114020+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:43.918355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.003179+0000","last_change":"2024-09-21T06:34:32.641463+0000","last_active":"2024-09-21T06:35:05.003179+0000","last_peered":"2024-09-21T06:35:05.003179+0000","last_clean":"2024-09-21T06:35:05.003179+0000","last_became_active":"2024-09-21T06:34:32.641086+0000","last_became_peered":"2024-09-21T06:34:32.641086+0000","last_unstale":"2024-09-21T06:35:05.003179+0000","last_undegraded":"2024-09-21T06:35:05.003179+0000","last_fullsized":"2024-09-21T06:35:05.003179+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:58.991374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995541+0000","last_change":"2024-09-21T06:34:30.946698+0000","last_active":"2024-09-21T06:35:04.995541+0000","last_peered":"2024-09-21T06:35:04.995541+0000","last_clean":"2024-09-21T06:35:04.995541+0000","last_became_active":"2024-09-21T06:34:30.946325+0000","last_became_peered":"2024-09-21T06:34:30.946325+0000","last_unstale":"2024-09-21T06:35:04.995541+0000","last_undegraded":"2024-09-21T06:35:04.995541+0000","last_fullsized":"2024-09-21T06:35:04.995541+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:22.639265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997471+0000","last_change":"2024-09-21T06:34:39.712429+0000","last_active":"2024-09-21T06:35:04.997471+0000","last_peered":"2024-09-21T06:35:04.997471+0000","last_clean":"2024-09-21T06:35:04.997471+0000","last_became_active":"2024-09-21T06:34:39.711823+0000","last_became_peered":"2024-09-21T06:34:39.711823+0000","last_unstale":"2024-09-21T06:35:04.997471+0000","last_undegraded":"2024-09-21T06:35:04.997471+0000","last_fullsized":"2024-09-21T06:35:04.997471+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:03:49.513991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.274438+0000","last_change":"2024-09-21T06:35:05.274326+0000","last_active":"2024-09-21T06:35:05.274438+0000","last_peered":"2024-09-21T06:35:05.274438+0000","last_clean":"2024-09-21T06:35:05.274438+0000","last_became_active":"2024-09-21T06:35:05.273884+0000","last_became_peered":"2024-09-21T06:35:05.273884+0000","last_unstale":"2024-09-21T06:35:05.274438+0000","last_undegraded":"2024-09-21T06:35:05.274438+0000","last_fullsized":"2024-09-21T06:35:05.274438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:03.219765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997399+0000","last_change":"2024-09-21T06:34:30.676794+0000","last_active":"2024-09-21T06:35:04.997399+0000","last_peered":"2024-09-21T06:35:04.997399+0000","last_clean":"2024-09-21T06:35:04.997399+0000","last_became_active":"2024-09-21T06:34:30.676400+0000","last_became_peered":"2024-09-21T06:34:30.676400+0000","last_unstale":"2024-09-21T06:35:04.997399+0000","last_undegraded":"2024-09-21T06:35:04.997399+0000","last_fullsized":"2024-09-21T06:35:04.997399+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:23.681363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995836+0000","last_change":"2024-09-21T06:34:39.708980+0000","last_active":"2024-09-21T06:35:04.995836+0000","last_peered":"2024-09-21T06:35:04.995836+0000","last_clean":"2024-09-21T06:35:04.995836+0000","last_became_active":"2024-09-21T06:34:39.707748+0000","last_became_peered":"2024-09-21T06:34:39.707748+0000","last_unstale":"2024-09-21T06:35:04.995836+0000","last_undegraded":"2024-09-21T06:35:04.995836+0000","last_fullsized":"2024-09-21T06:35:04.995836+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:16.812226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.273262+0000","last_change":"2024-09-21T06:34:32.638337+0000","last_active":"2024-09-21T06:35:05.273262+0000","last_peered":"2024-09-21T06:35:05.273262+0000","last_clean":"2024-09-21T06:35:05.273262+0000","last_became_active":"2024-09-21T06:34:32.638013+0000","last_became_peered":"2024-09-21T06:34:32.638013+0000","last_unstale":"2024-09-21T06:35:05.273262+0000","last_undegraded":"2024-09-21T06:35:05.273262+0000","last_fullsized":"2024-09-21T06:35:05.273262+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:26.321190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"83'5","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:17.661508+0000","last_change":"2024-09-21T06:34:30.631086+0000","last_active":"2024-09-21T06:35:17.661508+0000","last_peered":"2024-09-21T06:35:17.661508+0000","last_clean":"2024-09-21T06:35:17.661508+0000","last_became_active":"2024-09-21T06:34:30.630706+0000","last_became_peered":"2024-09-21T06:34:30.630706+0000","last_unstale":"2024-09-21T06:35:17.661508+0000","last_undegraded":"2024-09-21T06:35:17.661508+0000","last_fullsized":"2024-09-21T06:35:17.661508+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:30.260359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.272633+0000","last_change":"2024-09-21T06:34:39.709097+0000","last_active":"2024-09-21T06:35:05.272633+0000","last_peered":"2024-09-21T06:35:05.272633+0000","last_clean":"2024-09-21T06:35:05.272633+0000","last_became_active":"2024-09-21T06:34:39.708609+0000","last_became_peered":"2024-09-21T06:34:39.708609+0000","last_unstale":"2024-09-21T06:35:05.272633+0000","last_undegraded":"2024-09-21T06:35:05.272633+0000","last_fullsized":"2024-09-21T06:35:05.272633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:48.842174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995363+0000","last_change":"2024-09-21T06:34:32.636816+0000","last_active":"2024-09-21T06:35:04.995363+0000","last_peered":"2024-09-21T06:35:04.995363+0000","last_clean":"2024-09-21T06:35:04.995363+0000","last_became_active":"2024-09-21T06:34:32.636429+0000","last_became_peered":"2024-09-21T06:34:32.636429+0000","last_unstale":"2024-09-21T06:35:04.995363+0000","last_undegraded":"2024-09-21T06:35:04.995363+0000","last_fullsized":"2024-09-21T06:35:04.995363+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:49.010491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.004246+0000","last_change":"2024-09-21T06:34:30.947481+0000","last_active":"2024-09-21T06:35:05.004246+0000","last_peered":"2024-09-21T06:35:05.004246+0000","last_clean":"2024-09-21T06:35:05.004246+0000","last_became_active":"2024-09-21T06:34:30.947040+0000","last_became_peered":"2024-09-21T06:34:30.947040+0000","last_unstale":"2024-09-21T06:35:05.004246+0000","last_undegraded":"2024-09-21T06:35:05.004246+0000","last_fullsized":"2024-09-21T06:35:05.004246+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:52.813476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000546+0000","last_change":"2024-09-21T06:34:39.708674+0000","last_active":"2024-09-21T06:35:05.000546+0000","last_peered":"2024-09-21T06:35:05.000546+0000","last_clean":"2024-09-21T06:35:05.000546+0000","last_became_active":"2024-09-21T06:34:39.708176+0000","last_became_peered":"2024-09-21T06:34:39.708176+0000","last_unstale":"2024-09-21T06:35:05.000546+0000","last_undegraded":"2024-09-21T06:35:05.000546+0000","last_fullsized":"2024-09-21T06:35:05.000546+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:57.070792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000844+0000","last_change":"2024-09-21T06:34:32.636381+0000","last_active":"2024-09-21T06:35:05.000844+0000","last_peered":"2024-09-21T06:35:05.000844+0000","last_clean":"2024-09-21T06:35:05.000844+0000","last_became_active":"2024-09-21T06:34:32.635937+0000","last_became_peered":"2024-09-21T06:34:32.635937+0000","last_unstale":"2024-09-21T06:35:05.000844+0000","last_undegraded":"2024-09-21T06:35:05.000844+0000","last_fullsized":"2024-09-21T06:35:05.000844+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:52.870676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.114033+0000","last_change":"2024-09-21T06:34:30.947160+0000","last_active":"2024-09-21T06:35:05.114033+0000","last_peered":"2024-09-21T06:35:05.114033+0000","last_clean":"2024-09-21T06:35:05.114033+0000","last_became_active":"2024-09-21T06:34:30.946736+0000","last_became_peered":"2024-09-21T06:34:30.946736+0000","last_unstale":"2024-09-21T06:35:05.114033+0000","last_undegraded":"2024-09-21T06:35:05.114033+0000","last_fullsized":"2024-09-21T06:35:05.114033+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:47.425481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.006145+0000","last_change":"2024-09-21T06:35:05.006032+0000","last_active":"2024-09-21T06:35:05.006145+0000","last_peered":"2024-09-21T06:35:05.006145+0000","last_clean":"2024-09-21T06:35:05.006145+0000","last_became_active":"2024-09-21T06:35:05.005429+0000","last_became_peered":"2024-09-21T06:35:05.005429+0000","last_unstale":"2024-09-21T06:35:05.006145+0000","last_undegraded":"2024-09-21T06:35:05.006145+0000","last_fullsized":"2024-09-21T06:35:05.006145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:04.138097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000613+0000","last_change":"2024-09-21T06:34:32.648256+0000","last_active":"2024-09-21T06:35:05.000613+0000","last_peered":"2024-09-21T06:35:05.000613+0000","last_clean":"2024-09-21T06:35:05.000613+0000","last_became_active":"2024-09-21T06:34:32.647881+0000","last_became_peered":"2024-09-21T06:34:32.647881+0000","last_unstale":"2024-09-21T06:35:05.000613+0000","last_undegraded":"2024-09-21T06:35:05.000613+0000","last_fullsized":"2024-09-21T06:35:05.000613+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:14.590834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029849+0000","last_change":"2024-09-21T06:34:30.629918+0000","last_active":"2024-09-21T06:35:05.029849+0000","last_peered":"2024-09-21T06:35:05.029849+0000","last_clean":"2024-09-21T06:35:05.029849+0000","last_became_active":"2024-09-21T06:34:30.628785+0000","last_became_peered":"2024-09-21T06:34:30.628785+0000","last_unstale":"2024-09-21T06:35:05.029849+0000","last_undegraded":"2024-09-21T06:35:05.029849+0000","last_fullsized":"2024-09-21T06:35:05.029849+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:05.333406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995102+0000","last_change":"2024-09-21T06:34:39.710073+0000","last_active":"2024-09-21T06:35:04.995102+0000","last_peered":"2024-09-21T06:35:04.995102+0000","last_clean":"2024-09-21T06:35:04.995102+0000","last_became_active":"2024-09-21T06:34:39.708986+0000","last_became_peered":"2024-09-21T06:34:39.708986+0000","last_unstale":"2024-09-21T06:35:04.995102+0000","last_undegraded":"2024-09-21T06:35:04.995102+0000","last_fullsized":"2024-09-21T06:35:04.995102+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:46.379902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.998454+0000","last_change":"2024-09-21T06:34:32.644224+0000","last_active":"2024-09-21T06:35:04.998454+0000","last_peered":"2024-09-21T06:35:04.998454+0000","last_clean":"2024-09-21T06:35:04.998454+0000","last_became_active":"2024-09-21T06:34:32.643850+0000","last_became_peered":"2024-09-21T06:34:32.643850+0000","last_unstale":"2024-09-21T06:35:04.998454+0000","last_undegraded":"2024-09-21T06:35:04.998454+0000","last_fullsized":"2024-09-21T06:35:04.998454+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:24:33.188537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.994875+0000","last_change":"2024-09-21T06:34:30.946334+0000","last_active":"2024-09-21T06:35:04.994875+0000","last_peered":"2024-09-21T06:35:04.994875+0000","last_clean":"2024-09-21T06:35:04.994875+0000","last_became_active":"2024-09-21T06:34:30.945908+0000","last_became_peered":"2024-09-21T06:34:30.945908+0000","last_unstale":"2024-09-21T06:35:04.994875+0000","last_undegraded":"2024-09-21T06:35:04.994875+0000","last_fullsized":"2024-09-21T06:35:04.994875+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:46.746527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995193+0000","last_change":"2024-09-21T06:34:39.707909+0000","last_active":"2024-09-21T06:35:04.995193+0000","last_peered":"2024-09-21T06:35:04.995193+0000","last_clean":"2024-09-21T06:35:04.995193+0000","last_became_active":"2024-09-21T06:34:39.707492+0000","last_became_peered":"2024-09-21T06:34:39.707492+0000","last_unstale":"2024-09-21T06:35:04.995193+0000","last_undegraded":"2024-09-21T06:35:04.995193+0000","last_fullsized":"2024-09-21T06:35:04.995193+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:56.201212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.994774+0000","last_change":"2024-09-21T06:34:32.635870+0000","last_active":"2024-09-21T06:35:04.994774+0000","last_peered":"2024-09-21T06:35:04.994774+0000","last_clean":"2024-09-21T06:35:04.994774+0000","last_became_active":"2024-09-21T06:34:32.635252+0000","last_became_peered":"2024-09-21T06:34:32.635252+0000","last_unstale":"2024-09-21T06:35:04.994774+0000","last_undegraded":"2024-09-21T06:35:04.994774+0000","last_fullsized":"2024-09-21T06:35:04.994774+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:10.613033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029607+0000","last_change":"2024-09-21T06:34:30.634360+0000","last_active":"2024-09-21T06:35:05.029607+0000","last_peered":"2024-09-21T06:35:05.029607+0000","last_clean":"2024-09-21T06:35:05.029607+0000","last_became_active":"2024-09-21T06:34:30.633845+0000","last_became_peered":"2024-09-21T06:34:30.633845+0000","last_unstale":"2024-09-21T06:35:05.029607+0000","last_undegraded":"2024-09-21T06:35:05.029607+0000","last_fullsized":"2024-09-21T06:35:05.029607+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:34:53.388670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999643+0000","last_change":"2024-09-21T06:34:39.719620+0000","last_active":"2024-09-21T06:35:04.999643+0000","last_peered":"2024-09-21T06:35:04.999643+0000","last_clean":"2024-09-21T06:35:04.999643+0000","last_became_active":"2024-09-21T06:34:39.718786+0000","last_became_peered":"2024-09-21T06:34:39.718786+0000","last_unstale":"2024-09-21T06:35:04.999643+0000","last_undegraded":"2024-09-21T06:35:04.999643+0000","last_fullsized":"2024-09-21T06:35:04.999643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:43.524932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.273070+0000","last_change":"2024-09-21T06:34:32.634970+0000","last_active":"2024-09-21T06:35:05.273070+0000","last_peered":"2024-09-21T06:35:05.273070+0000","last_clean":"2024-09-21T06:35:05.273070+0000","last_became_active":"2024-09-21T06:34:32.634611+0000","last_became_peered":"2024-09-21T06:34:32.634611+0000","last_unstale":"2024-09-21T06:35:05.273070+0000","last_undegraded":"2024-09-21T06:35:05.273070+0000","last_fullsized":"2024-09-21T06:35:05.273070+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:37.855849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997918+0000","last_change":"2024-09-21T06:34:30.628123+0000","last_active":"2024-09-21T06:35:04.997918+0000","last_peered":"2024-09-21T06:35:04.997918+0000","last_clean":"2024-09-21T06:35:04.997918+0000","last_became_active":"2024-09-21T06:34:30.627498+0000","last_became_peered":"2024-09-21T06:34:30.627498+0000","last_unstale":"2024-09-21T06:35:04.997918+0000","last_undegraded":"2024-09-21T06:35:04.997918+0000","last_fullsized":"2024-09-21T06:35:04.997918+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:42.576441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":110,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029689+0000","last_change":"2024-09-21T06:33:16.856623+0000","last_active":"2024-09-21T06:35:05.029689+0000","last_peered":"2024-09-21T06:35:05.029689+0000","last_clean":"2024-09-21T06:35:05.029689+0000","last_became_active":"2024-09-21T06:33:16.850787+0000","last_became_peered":"2024-09-21T06:33:16.850787+0000","last_unstale":"2024-09-21T06:35:05.029689+0000","last_undegraded":"2024-09-21T06:35:05.029689+0000","last_fullsized":"2024-09-21T06:35:05.029689+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:50.283252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:50.283252+0000","last_clean_scrub_stamp":"2024-09-21T06:29:50.283252+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:29:20.711208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029992+0000","last_change":"2024-09-21T06:34:39.706472+0000","last_active":"2024-09-21T06:35:05.029992+0000","last_peered":"2024-09-21T06:35:05.029992+0000","last_clean":"2024-09-21T06:35:05.029992+0000","last_became_active":"2024-09-21T06:34:39.705724+0000","last_became_peered":"2024-09-21T06:34:39.705724+0000","last_unstale":"2024-09-21T06:35:05.029992+0000","last_undegraded":"2024-09-21T06:35:05.029992+0000","last_fullsized":"2024-09-21T06:35:05.029992+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:43.238704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.030085+0000","last_change":"2024-09-21T06:34:32.645944+0000","last_active":"2024-09-21T06:35:05.030085+0000","last_peered":"2024-09-21T06:35:05.030085+0000","last_clean":"2024-09-21T06:35:05.030085+0000","last_became_active":"2024-09-21T06:34:32.645134+0000","last_became_peered":"2024-09-21T06:34:32.645134+0000","last_unstale":"2024-09-21T06:35:05.030085+0000","last_undegraded":"2024-09-21T06:35:05.030085+0000","last_fullsized":"2024-09-21T06:35:05.030085+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:45.254164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.996320+0000","last_change":"2024-09-21T06:34:30.946296+0000","last_active":"2024-09-21T06:35:04.996320+0000","last_peered":"2024-09-21T06:35:04.996320+0000","last_clean":"2024-09-21T06:35:04.996320+0000","last_became_active":"2024-09-21T06:34:30.945777+0000","last_became_peered":"2024-09-21T06:34:30.945777+0000","last_unstale":"2024-09-21T06:35:04.996320+0000","last_undegraded":"2024-09-21T06:35:04.996320+0000","last_fullsized":"2024-09-21T06:35:04.996320+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:32:25.988512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995681+0000","last_change":"2024-09-21T06:34:39.705876+0000","last_active":"2024-09-21T06:35:04.995681+0000","last_peered":"2024-09-21T06:35:04.995681+0000","last_clean":"2024-09-21T06:35:04.995681+0000","last_became_active":"2024-09-21T06:34:39.705132+0000","last_became_peered":"2024-09-21T06:34:39.705132+0000","last_unstale":"2024-09-21T06:35:04.995681+0000","last_undegraded":"2024-09-21T06:35:04.995681+0000","last_fullsized":"2024-09-21T06:35:04.995681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:18.180244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029892+0000","last_change":"2024-09-21T06:34:32.638242+0000","last_active":"2024-09-21T06:35:05.029892+0000","last_peered":"2024-09-21T06:35:05.029892+0000","last_clean":"2024-09-21T06:35:05.029892+0000","last_became_active":"2024-09-21T06:34:32.637816+0000","last_became_peered":"2024-09-21T06:34:32.637816+0000","last_unstale":"2024-09-21T06:35:05.029892+0000","last_undegraded":"2024-09-21T06:35:05.029892+0000","last_fullsized":"2024-09-21T06:35:05.029892+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:40.926147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000042+0000","last_change":"2024-09-21T06:34:30.632173+0000","last_active":"2024-09-21T06:35:05.000042+0000","last_peered":"2024-09-21T06:35:05.000042+0000","last_clean":"2024-09-21T06:35:05.000042+0000","last_became_active":"2024-09-21T06:34:30.631638+0000","last_became_peered":"2024-09-21T06:34:30.631638+0000","last_unstale":"2024-09-21T06:35:05.000042+0000","last_undegraded":"2024-09-21T06:35:05.000042+0000","last_fullsized":"2024-09-21T06:35:05.000042+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:29.763272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000157+0000","last_change":"2024-09-21T06:34:39.714620+0000","last_active":"2024-09-21T06:35:05.000157+0000","last_peered":"2024-09-21T06:35:05.000157+0000","last_clean":"2024-09-21T06:35:05.000157+0000","last_became_active":"2024-09-21T06:34:39.714199+0000","last_became_peered":"2024-09-21T06:34:39.714199+0000","last_unstale":"2024-09-21T06:35:05.000157+0000","last_undegraded":"2024-09-21T06:35:05.000157+0000","last_fullsized":"2024-09-21T06:35:05.000157+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:37.041249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.030100+0000","last_change":"2024-09-21T06:34:32.645001+0000","last_active":"2024-09-21T06:35:05.030100+0000","last_peered":"2024-09-21T06:35:05.030100+0000","last_clean":"2024-09-21T06:35:05.030100+0000","last_became_active":"2024-09-21T06:34:32.644584+0000","last_became_peered":"2024-09-21T06:34:32.644584+0000","last_unstale":"2024-09-21T06:35:05.030100+0000","last_undegraded":"2024-09-21T06:35:05.030100+0000","last_fullsized":"2024-09-21T06:35:05.030100+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:47.098850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995420+0000","last_change":"2024-09-21T06:34:30.632593+0000","last_active":"2024-09-21T06:35:04.995420+0000","last_peered":"2024-09-21T06:35:04.995420+0000","last_clean":"2024-09-21T06:35:04.995420+0000","last_became_active":"2024-09-21T06:34:30.632170+0000","last_became_peered":"2024-09-21T06:34:30.632170+0000","last_unstale":"2024-09-21T06:35:04.995420+0000","last_undegraded":"2024-09-21T06:35:04.995420+0000","last_fullsized":"2024-09-21T06:35:04.995420+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:56.170655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.998214+0000","last_change":"2024-09-21T06:34:39.710172+0000","last_active":"2024-09-21T06:35:04.998214+0000","last_peered":"2024-09-21T06:35:04.998214+0000","last_clean":"2024-09-21T06:35:04.998214+0000","last_became_active":"2024-09-21T06:34:39.708876+0000","last_became_peered":"2024-09-21T06:34:39.708876+0000","last_unstale":"2024-09-21T06:35:04.998214+0000","last_undegraded":"2024-09-21T06:35:04.998214+0000","last_fullsized":"2024-09-21T06:35:04.998214+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:32.371873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000997+0000","last_change":"2024-09-21T06:34:32.651056+0000","last_active":"2024-09-21T06:35:05.000997+0000","last_peered":"2024-09-21T06:35:05.000997+0000","last_clean":"2024-09-21T06:35:05.000997+0000","last_became_active":"2024-09-21T06:34:32.650676+0000","last_became_peered":"2024-09-21T06:34:32.650676+0000","last_unstale":"2024-09-21T06:35:05.000997+0000","last_undegraded":"2024-09-21T06:35:05.000997+0000","last_fullsized":"2024-09-21T06:35:05.000997+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:34.599654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.004248+0000","last_change":"2024-09-21T06:34:30.632307+0000","last_active":"2024-09-21T06:35:05.004248+0000","last_peered":"2024-09-21T06:35:05.004248+0000","last_clean":"2024-09-21T06:35:05.004248+0000","last_became_active":"2024-09-21T06:34:30.631920+0000","last_became_peered":"2024-09-21T06:34:30.631920+0000","last_unstale":"2024-09-21T06:35:05.004248+0000","last_undegraded":"2024-09-21T06:35:05.004248+0000","last_fullsized":"2024-09-21T06:35:05.004248+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:46.940898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.998452+0000","last_change":"2024-09-21T06:34:39.715056+0000","last_active":"2024-09-21T06:35:04.998452+0000","last_peered":"2024-09-21T06:35:04.998452+0000","last_clean":"2024-09-21T06:35:04.998452+0000","last_became_active":"2024-09-21T06:34:39.714563+0000","last_became_peered":"2024-09-21T06:34:39.714563+0000","last_unstale":"2024-09-21T06:35:04.998452+0000","last_undegraded":"2024-09-21T06:35:04.998452+0000","last_fullsized":"2024-09-21T06:35:04.998452+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:48.567057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.998840+0000","last_change":"2024-09-21T06:34:32.646934+0000","last_active":"2024-09-21T06:35:04.998840+0000","last_peered":"2024-09-21T06:35:04.998840+0000","last_clean":"2024-09-21T06:35:04.998840+0000","last_became_active":"2024-09-21T06:34:32.646559+0000","last_became_peered":"2024-09-21T06:34:32.646559+0000","last_unstale":"2024-09-21T06:35:04.998840+0000","last_undegraded":"2024-09-21T06:35:04.998840+0000","last_fullsized":"2024-09-21T06:35:04.998840+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:37.802590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.004016+0000","last_change":"2024-09-21T06:34:30.946769+0000","last_active":"2024-09-21T06:35:05.004016+0000","last_peered":"2024-09-21T06:35:05.004016+0000","last_clean":"2024-09-21T06:35:05.004016+0000","last_became_active":"2024-09-21T06:34:30.946327+0000","last_became_peered":"2024-09-21T06:34:30.946327+0000","last_unstale":"2024-09-21T06:35:05.004016+0000","last_undegraded":"2024-09-21T06:35:05.004016+0000","last_fullsized":"2024-09-21T06:35:05.004016+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:52.055398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995322+0000","last_change":"2024-09-21T06:34:39.713398+0000","last_active":"2024-09-21T06:35:04.995322+0000","last_peered":"2024-09-21T06:35:04.995322+0000","last_clean":"2024-09-21T06:35:04.995322+0000","last_became_active":"2024-09-21T06:34:39.712954+0000","last_became_peered":"2024-09-21T06:34:39.712954+0000","last_unstale":"2024-09-21T06:35:04.995322+0000","last_undegraded":"2024-09-21T06:35:04.995322+0000","last_fullsized":"2024-09-21T06:35:04.995322+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:58.881014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029465+0000","last_change":"2024-09-21T06:34:32.640879+0000","last_active":"2024-09-21T06:35:05.029465+0000","last_peered":"2024-09-21T06:35:05.029465+0000","last_clean":"2024-09-21T06:35:05.029465+0000","last_became_active":"2024-09-21T06:34:32.640526+0000","last_became_peered":"2024-09-21T06:34:32.640526+0000","last_unstale":"2024-09-21T06:35:05.029465+0000","last_undegraded":"2024-09-21T06:35:05.029465+0000","last_fullsized":"2024-09-21T06:35:05.029465+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:12.327195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997068+0000","last_change":"2024-09-21T06:34:30.631449+0000","last_active":"2024-09-21T06:35:04.997068+0000","last_peered":"2024-09-21T06:35:04.997068+0000","last_clean":"2024-09-21T06:35:04.997068+0000","last_became_active":"2024-09-21T06:34:30.631081+0000","last_became_peered":"2024-09-21T06:34:30.631081+0000","last_unstale":"2024-09-21T06:35:04.997068+0000","last_undegraded":"2024-09-21T06:35:04.997068+0000","last_fullsized":"2024-09-21T06:35:04.997068+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:09.202696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.114180+0000","last_change":"2024-09-21T06:34:39.707379+0000","last_active":"2024-09-21T06:35:05.114180+0000","last_peered":"2024-09-21T06:35:05.114180+0000","last_clean":"2024-09-21T06:35:05.114180+0000","last_became_active":"2024-09-21T06:34:39.706979+0000","last_became_peered":"2024-09-21T06:34:39.706979+0000","last_unstale":"2024-09-21T06:35:05.114180+0000","last_undegraded":"2024-09-21T06:35:05.114180+0000","last_fullsized":"2024-09-21T06:35:05.114180+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:24.394233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029782+0000","last_change":"2024-09-21T06:34:32.649458+0000","last_active":"2024-09-21T06:35:05.029782+0000","last_peered":"2024-09-21T06:35:05.029782+0000","last_clean":"2024-09-21T06:35:05.029782+0000","last_became_active":"2024-09-21T06:34:32.649051+0000","last_became_peered":"2024-09-21T06:34:32.649051+0000","last_unstale":"2024-09-21T06:35:05.029782+0000","last_undegraded":"2024-09-21T06:35:05.029782+0000","last_fullsized":"2024-09-21T06:35:05.029782+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:12.631446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999624+0000","last_change":"2024-09-21T06:34:30.634592+0000","last_active":"2024-09-21T06:35:04.999624+0000","last_peered":"2024-09-21T06:35:04.999624+0000","last_clean":"2024-09-21T06:35:04.999624+0000","last_became_active":"2024-09-21T06:34:30.634251+0000","last_became_peered":"2024-09-21T06:34:30.634251+0000","last_unstale":"2024-09-21T06:35:04.999624+0000","last_undegraded":"2024-09-21T06:35:04.999624+0000","last_fullsized":"2024-09-21T06:35:04.999624+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:32.817467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000642+0000","last_change":"2024-09-21T06:34:39.719416+0000","last_active":"2024-09-21T06:35:05.000642+0000","last_peered":"2024-09-21T06:35:05.000642+0000","last_clean":"2024-09-21T06:35:05.000642+0000","last_became_active":"2024-09-21T06:34:39.718910+0000","last_became_peered":"2024-09-21T06:34:39.718910+0000","last_unstale":"2024-09-21T06:35:05.000642+0000","last_undegraded":"2024-09-21T06:35:05.000642+0000","last_fullsized":"2024-09-21T06:35:05.000642+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:41.413250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.994958+0000","last_change":"2024-09-21T06:34:32.640727+0000","last_active":"2024-09-21T06:35:04.994958+0000","last_peered":"2024-09-21T06:35:04.994958+0000","last_clean":"2024-09-21T06:35:04.994958+0000","last_became_active":"2024-09-21T06:34:32.640296+0000","last_became_peered":"2024-09-21T06:34:32.640296+0000","last_unstale":"2024-09-21T06:35:04.994958+0000","last_undegraded":"2024-09-21T06:35:04.994958+0000","last_fullsized":"2024-09-21T06:35:04.994958+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:35.124059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.996682+0000","last_change":"2024-09-21T06:34:30.633332+0000","last_active":"2024-09-21T06:35:04.996682+0000","last_peered":"2024-09-21T06:35:04.996682+0000","last_clean":"2024-09-21T06:35:04.996682+0000","last_became_active":"2024-09-21T06:34:30.632641+0000","last_became_peered":"2024-09-21T06:34:30.632641+0000","last_unstale":"2024-09-21T06:35:04.996682+0000","last_undegraded":"2024-09-21T06:35:04.996682+0000","last_fullsized":"2024-09-21T06:35:04.996682+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:30:23.930313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999766+0000","last_change":"2024-09-21T06:34:39.712504+0000","last_active":"2024-09-21T06:35:04.999766+0000","last_peered":"2024-09-21T06:35:04.999766+0000","last_clean":"2024-09-21T06:35:04.999766+0000","last_became_active":"2024-09-21T06:34:39.711638+0000","last_became_peered":"2024-09-21T06:34:39.711638+0000","last_unstale":"2024-09-21T06:35:04.999766+0000","last_undegraded":"2024-09-21T06:35:04.999766+0000","last_fullsized":"2024-09-21T06:35:04.999766+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:26:09.003367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.004007+0000","last_change":"2024-09-21T06:34:32.639582+0000","last_active":"2024-09-21T06:35:05.004007+0000","last_peered":"2024-09-21T06:35:05.004007+0000","last_clean":"2024-09-21T06:35:05.004007+0000","last_became_active":"2024-09-21T06:34:32.639023+0000","last_became_peered":"2024-09-21T06:34:32.639023+0000","last_unstale":"2024-09-21T06:35:05.004007+0000","last_undegraded":"2024-09-21T06:35:05.004007+0000","last_fullsized":"2024-09-21T06:35:05.004007+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:26.773389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"83'24","reported_seq":90,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:20.052467+0000","last_change":"2024-09-21T06:34:30.626280+0000","last_active":"2024-09-21T06:35:20.052467+0000","last_peered":"2024-09-21T06:35:20.052467+0000","last_clean":"2024-09-21T06:35:20.052467+0000","last_became_active":"2024-09-21T06:34:30.625662+0000","last_became_peered":"2024-09-21T06:34:30.625662+0000","last_unstale":"2024-09-21T06:35:20.052467+0000","last_undegraded":"2024-09-21T06:35:20.052467+0000","last_fullsized":"2024-09-21T06:35:20.052467+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:10.541881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48619,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":31,"num_read_kb":63,"num_write":24,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.113723+0000","last_change":"2024-09-21T06:34:39.708999+0000","last_active":"2024-09-21T06:35:05.113723+0000","last_peered":"2024-09-21T06:35:05.113723+0000","last_clean":"2024-09-21T06:35:05.113723+0000","last_became_active":"2024-09-21T06:34:39.708580+0000","last_became_peered":"2024-09-21T06:34:39.708580+0000","last_unstale":"2024-09-21T06:35:05.113723+0000","last_undegraded":"2024-09-21T06:35:05.113723+0000","last_fullsized":"2024-09-21T06:35:05.113723+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:07.648358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995798+0000","last_change":"2024-09-21T06:34:32.651157+0000","last_active":"2024-09-21T06:35:04.995798+0000","last_peered":"2024-09-21T06:35:04.995798+0000","last_clean":"2024-09-21T06:35:04.995798+0000","last_became_active":"2024-09-21T06:34:32.650804+0000","last_became_peered":"2024-09-21T06:34:32.650804+0000","last_unstale":"2024-09-21T06:35:04.995798+0000","last_undegraded":"2024-09-21T06:35:04.995798+0000","last_fullsized":"2024-09-21T06:35:04.995798+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:49.758496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999901+0000","last_change":"2024-09-21T06:34:30.947142+0000","last_active":"2024-09-21T06:35:04.999901+0000","last_peered":"2024-09-21T06:35:04.999901+0000","last_clean":"2024-09-21T06:35:04.999901+0000","last_became_active":"2024-09-21T06:34:30.946748+0000","last_became_peered":"2024-09-21T06:34:30.946748+0000","last_unstale":"2024-09-21T06:35:04.999901+0000","last_undegraded":"2024-09-21T06:35:04.999901+0000","last_fullsized":"2024-09-21T06:35:04.999901+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:39.222286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997754+0000","last_change":"2024-09-21T06:34:39.709266+0000","last_active":"2024-09-21T06:35:04.997754+0000","last_peered":"2024-09-21T06:35:04.997754+0000","last_clean":"2024-09-21T06:35:04.997754+0000","last_became_active":"2024-09-21T06:34:39.708753+0000","last_became_peered":"2024-09-21T06:34:39.708753+0000","last_unstale":"2024-09-21T06:35:04.997754+0000","last_undegraded":"2024-09-21T06:35:04.997754+0000","last_fullsized":"2024-09-21T06:35:04.997754+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:48.711681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000708+0000","last_change":"2024-09-21T06:34:32.637976+0000","last_active":"2024-09-21T06:35:05.000708+0000","last_peered":"2024-09-21T06:35:05.000708+0000","last_clean":"2024-09-21T06:35:05.000708+0000","last_became_active":"2024-09-21T06:34:32.637267+0000","last_became_peered":"2024-09-21T06:34:32.637267+0000","last_unstale":"2024-09-21T06:35:05.000708+0000","last_undegraded":"2024-09-21T06:35:05.000708+0000","last_fullsized":"2024-09-21T06:35:05.000708+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:31.340583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.030056+0000","last_change":"2024-09-21T06:34:30.631059+0000","last_active":"2024-09-21T06:35:05.030056+0000","last_peered":"2024-09-21T06:35:05.030056+0000","last_clean":"2024-09-21T06:35:05.030056+0000","last_became_active":"2024-09-21T06:34:30.630633+0000","last_became_peered":"2024-09-21T06:34:30.630633+0000","last_unstale":"2024-09-21T06:35:05.030056+0000","last_undegraded":"2024-09-21T06:35:05.030056+0000","last_fullsized":"2024-09-21T06:35:05.030056+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:09.049438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.113457+0000","last_change":"2024-09-21T06:34:39.713831+0000","last_active":"2024-09-21T06:35:05.113457+0000","last_peered":"2024-09-21T06:35:05.113457+0000","last_clean":"2024-09-21T06:35:05.113457+0000","last_became_active":"2024-09-21T06:34:39.713403+0000","last_became_peered":"2024-09-21T06:34:39.713403+0000","last_unstale":"2024-09-21T06:35:05.113457+0000","last_undegraded":"2024-09-21T06:35:05.113457+0000","last_fullsized":"2024-09-21T06:35:05.113457+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:22.201063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.996367+0000","last_change":"2024-09-21T06:34:32.646924+0000","last_active":"2024-09-21T06:35:04.996367+0000","last_peered":"2024-09-21T06:35:04.996367+0000","last_clean":"2024-09-21T06:35:04.996367+0000","last_became_active":"2024-09-21T06:34:32.646531+0000","last_became_peered":"2024-09-21T06:34:32.646531+0000","last_unstale":"2024-09-21T06:35:04.996367+0000","last_undegraded":"2024-09-21T06:35:04.996367+0000","last_fullsized":"2024-09-21T06:35:04.996367+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:48.550193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.001234+0000","last_change":"2024-09-21T06:34:30.631901+0000","last_active":"2024-09-21T06:35:05.001234+0000","last_peered":"2024-09-21T06:35:05.001234+0000","last_clean":"2024-09-21T06:35:05.001234+0000","last_became_active":"2024-09-21T06:34:30.631450+0000","last_became_peered":"2024-09-21T06:34:30.631450+0000","last_unstale":"2024-09-21T06:35:05.001234+0000","last_undegraded":"2024-09-21T06:35:05.001234+0000","last_fullsized":"2024-09-21T06:35:05.001234+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:20.709456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.272748+0000","last_change":"2024-09-21T06:34:39.711951+0000","last_active":"2024-09-21T06:35:05.272748+0000","last_peered":"2024-09-21T06:35:05.272748+0000","last_clean":"2024-09-21T06:35:05.272748+0000","last_became_active":"2024-09-21T06:34:39.710783+0000","last_became_peered":"2024-09-21T06:34:39.710783+0000","last_unstale":"2024-09-21T06:35:05.272748+0000","last_undegraded":"2024-09-21T06:35:05.272748+0000","last_fullsized":"2024-09-21T06:35:05.272748+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:27:58.501155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000355+0000","last_change":"2024-09-21T06:34:39.709639+0000","last_active":"2024-09-21T06:35:05.000355+0000","last_peered":"2024-09-21T06:35:05.000355+0000","last_clean":"2024-09-21T06:35:05.000355+0000","last_became_active":"2024-09-21T06:34:39.709241+0000","last_became_peered":"2024-09-21T06:34:39.709241+0000","last_unstale":"2024-09-21T06:35:05.000355+0000","last_undegraded":"2024-09-21T06:35:05.000355+0000","last_fullsized":"2024-09-21T06:35:05.000355+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.357399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.113674+0000","last_change":"2024-09-21T06:34:32.635365+0000","last_active":"2024-09-21T06:35:05.113674+0000","last_peered":"2024-09-21T06:35:05.113674+0000","last_clean":"2024-09-21T06:35:05.113674+0000","last_became_active":"2024-09-21T06:34:32.635012+0000","last_became_peered":"2024-09-21T06:34:32.635012+0000","last_unstale":"2024-09-21T06:35:05.113674+0000","last_undegraded":"2024-09-21T06:35:05.113674+0000","last_fullsized":"2024-09-21T06:35:05.113674+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:54.671074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.998620+0000","last_change":"2024-09-21T06:34:30.633347+0000","last_active":"2024-09-21T06:35:04.998620+0000","last_peered":"2024-09-21T06:35:04.998620+0000","last_clean":"2024-09-21T06:35:04.998620+0000","last_became_active":"2024-09-21T06:34:30.632513+0000","last_became_peered":"2024-09-21T06:34:30.632513+0000","last_unstale":"2024-09-21T06:35:04.998620+0000","last_undegraded":"2024-09-21T06:35:04.998620+0000","last_fullsized":"2024-09-21T06:35:04.998620+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:12.439109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.001150+0000","last_change":"2024-09-21T06:34:39.710204+0000","last_active":"2024-09-21T06:35:05.001150+0000","last_peered":"2024-09-21T06:35:05.001150+0000","last_clean":"2024-09-21T06:35:05.001150+0000","last_became_active":"2024-09-21T06:34:39.709744+0000","last_became_peered":"2024-09-21T06:34:39.709744+0000","last_unstale":"2024-09-21T06:35:05.001150+0000","last_undegraded":"2024-09-21T06:35:05.001150+0000","last_fullsized":"2024-09-21T06:35:05.001150+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:45.484088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.272833+0000","last_change":"2024-09-21T06:34:32.634170+0000","last_active":"2024-09-21T06:35:05.272833+0000","last_peered":"2024-09-21T06:35:05.272833+0000","last_clean":"2024-09-21T06:35:05.272833+0000","last_became_active":"2024-09-21T06:34:32.633831+0000","last_became_peered":"2024-09-21T06:34:32.633831+0000","last_unstale":"2024-09-21T06:35:05.272833+0000","last_undegraded":"2024-09-21T06:35:05.272833+0000","last_fullsized":"2024-09-21T06:35:05.272833+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:36.738279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.994732+0000","last_change":"2024-09-21T06:34:30.629422+0000","last_active":"2024-09-21T06:35:04.994732+0000","last_peered":"2024-09-21T06:35:04.994732+0000","last_clean":"2024-09-21T06:35:04.994732+0000","last_became_active":"2024-09-21T06:34:30.629025+0000","last_became_peered":"2024-09-21T06:34:30.629025+0000","last_unstale":"2024-09-21T06:35:04.994732+0000","last_undegraded":"2024-09-21T06:35:04.994732+0000","last_fullsized":"2024-09-21T06:35:04.994732+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:05.068816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000955+0000","last_change":"2024-09-21T06:34:30.634562+0000","last_active":"2024-09-21T06:35:05.000955+0000","last_peered":"2024-09-21T06:35:05.000955+0000","last_clean":"2024-09-21T06:35:05.000955+0000","last_became_active":"2024-09-21T06:34:30.634170+0000","last_became_peered":"2024-09-21T06:34:30.634170+0000","last_unstale":"2024-09-21T06:35:05.000955+0000","last_undegraded":"2024-09-21T06:35:05.000955+0000","last_fullsized":"2024-09-21T06:35:05.000955+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:15.243103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999225+0000","last_change":"2024-09-21T06:34:32.645718+0000","last_active":"2024-09-21T06:35:04.999225+0000","last_peered":"2024-09-21T06:35:04.999225+0000","last_clean":"2024-09-21T06:35:04.999225+0000","last_became_active":"2024-09-21T06:34:32.645072+0000","last_became_peered":"2024-09-21T06:34:32.645072+0000","last_unstale":"2024-09-21T06:35:04.999225+0000","last_undegraded":"2024-09-21T06:35:04.999225+0000","last_fullsized":"2024-09-21T06:35:04.999225+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:54.109246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.996127+0000","last_change":"2024-09-21T06:34:39.716947+0000","last_active":"2024-09-21T06:35:04.996127+0000","last_peered":"2024-09-21T06:35:04.996127+0000","last_clean":"2024-09-21T06:35:04.996127+0000","last_became_active":"2024-09-21T06:34:39.716540+0000","last_became_peered":"2024-09-21T06:34:39.716540+0000","last_unstale":"2024-09-21T06:35:04.996127+0000","last_undegraded":"2024-09-21T06:35:04.996127+0000","last_fullsized":"2024-09-21T06:35:04.996127+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:12.748444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999360+0000","last_change":"2024-09-21T06:34:30.636389+0000","last_active":"2024-09-21T06:35:04.999360+0000","last_peered":"2024-09-21T06:35:04.999360+0000","last_clean":"2024-09-21T06:35:04.999360+0000","last_became_active":"2024-09-21T06:34:30.636008+0000","last_became_peered":"2024-09-21T06:34:30.636008+0000","last_unstale":"2024-09-21T06:35:04.999360+0000","last_undegraded":"2024-09-21T06:35:04.999360+0000","last_fullsized":"2024-09-21T06:35:04.999360+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:45.476267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.003221+0000","last_change":"2024-09-21T06:34:32.635644+0000","last_active":"2024-09-21T06:35:05.003221+0000","last_peered":"2024-09-21T06:35:05.003221+0000","last_clean":"2024-09-21T06:35:05.003221+0000","last_became_active":"2024-09-21T06:34:32.635253+0000","last_became_peered":"2024-09-21T06:34:32.635253+0000","last_unstale":"2024-09-21T06:35:05.003221+0000","last_undegraded":"2024-09-21T06:35:05.003221+0000","last_fullsized":"2024-09-21T06:35:05.003221+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:34.713656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.996076+0000","last_change":"2024-09-21T06:34:39.710382+0000","last_active":"2024-09-21T06:35:04.996076+0000","last_peered":"2024-09-21T06:35:04.996076+0000","last_clean":"2024-09-21T06:35:04.996076+0000","last_became_active":"2024-09-21T06:34:39.709884+0000","last_became_peered":"2024-09-21T06:34:39.709884+0000","last_unstale":"2024-09-21T06:35:04.996076+0000","last_undegraded":"2024-09-21T06:35:04.996076+0000","last_fullsized":"2024-09-21T06:35:04.996076+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:01.602366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000463+0000","last_change":"2024-09-21T06:34:32.640025+0000","last_active":"2024-09-21T06:35:05.000463+0000","last_peered":"2024-09-21T06:35:05.000463+0000","last_clean":"2024-09-21T06:35:05.000463+0000","last_became_active":"2024-09-21T06:34:32.639648+0000","last_became_peered":"2024-09-21T06:34:32.639648+0000","last_unstale":"2024-09-21T06:35:05.000463+0000","last_undegraded":"2024-09-21T06:35:05.000463+0000","last_fullsized":"2024-09-21T06:35:05.000463+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:05.833065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.115450+0000","last_change":"2024-09-21T06:35:05.115344+0000","last_active":"2024-09-21T06:35:05.115450+0000","last_peered":"2024-09-21T06:35:05.115450+0000","last_clean":"2024-09-21T06:35:05.115450+0000","last_became_active":"2024-09-21T06:35:05.114678+0000","last_became_peered":"2024-09-21T06:35:05.114678+0000","last_unstale":"2024-09-21T06:35:05.115450+0000","last_undegraded":"2024-09-21T06:35:05.115450+0000","last_fullsized":"2024-09-21T06:35:05.115450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:24.593145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.115415+0000","last_change":"2024-09-21T06:35:05.115301+0000","last_active":"2024-09-21T06:35:05.115415+0000","last_peered":"2024-09-21T06:35:05.115415+0000","last_clean":"2024-09-21T06:35:05.115415+0000","last_became_active":"2024-09-21T06:35:05.114706+0000","last_became_peered":"2024-09-21T06:35:05.114706+0000","last_unstale":"2024-09-21T06:35:05.115415+0000","last_undegraded":"2024-09-21T06:35:05.115415+0000","last_fullsized":"2024-09-21T06:35:05.115415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:09.513594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000261+0000","last_change":"2024-09-21T06:34:32.634679+0000","last_active":"2024-09-21T06:35:05.000261+0000","last_peered":"2024-09-21T06:35:05.000261+0000","last_clean":"2024-09-21T06:35:05.000261+0000","last_became_active":"2024-09-21T06:34:32.634285+0000","last_became_peered":"2024-09-21T06:34:32.634285+0000","last_unstale":"2024-09-21T06:35:05.000261+0000","last_undegraded":"2024-09-21T06:35:05.000261+0000","last_fullsized":"2024-09-21T06:35:05.000261+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:15.185453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995303+0000","last_change":"2024-09-21T06:34:32.639037+0000","last_active":"2024-09-21T06:35:04.995303+0000","last_peered":"2024-09-21T06:35:04.995303+0000","last_clean":"2024-09-21T06:35:04.995303+0000","last_became_active":"2024-09-21T06:34:32.638689+0000","last_became_peered":"2024-09-21T06:34:32.638689+0000","last_unstale":"2024-09-21T06:35:04.995303+0000","last_undegraded":"2024-09-21T06:35:04.995303+0000","last_fullsized":"2024-09-21T06:35:04.995303+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:13.609191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.115157+0000","last_change":"2024-09-21T06:35:05.115051+0000","last_active":"2024-09-21T06:35:05.115157+0000","last_peered":"2024-09-21T06:35:05.115157+0000","last_clean":"2024-09-21T06:35:05.115157+0000","last_became_active":"2024-09-21T06:35:05.114602+0000","last_became_peered":"2024-09-21T06:35:05.114602+0000","last_unstale":"2024-09-21T06:35:05.115157+0000","last_undegraded":"2024-09-21T06:35:05.115157+0000","last_fullsized":"2024-09-21T06:35:05.115157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:48.778188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.996472+0000","last_change":"2024-09-21T06:34:32.637237+0000","last_active":"2024-09-21T06:35:04.996472+0000","last_peered":"2024-09-21T06:35:04.996472+0000","last_clean":"2024-09-21T06:35:04.996472+0000","last_became_active":"2024-09-21T06:34:32.634105+0000","last_became_peered":"2024-09-21T06:34:32.634105+0000","last_unstale":"2024-09-21T06:35:04.996472+0000","last_undegraded":"2024-09-21T06:35:04.996472+0000","last_fullsized":"2024-09-21T06:35:04.996472+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:52.530461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.273180+0000","last_change":"2024-09-21T06:34:32.624441+0000","last_active":"2024-09-21T06:35:05.273180+0000","last_peered":"2024-09-21T06:35:05.273180+0000","last_clean":"2024-09-21T06:35:05.273180+0000","last_became_active":"2024-09-21T06:34:32.623974+0000","last_became_peered":"2024-09-21T06:34:32.623974+0000","last_unstale":"2024-09-21T06:35:05.273180+0000","last_undegraded":"2024-09-21T06:35:05.273180+0000","last_fullsized":"2024-09-21T06:35:05.273180+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:12.378113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.998387+0000","last_change":"2024-09-21T06:34:32.633879+0000","last_active":"2024-09-21T06:35:04.998387+0000","last_peered":"2024-09-21T06:35:04.998387+0000","last_clean":"2024-09-21T06:35:04.998387+0000","last_became_active":"2024-09-21T06:34:32.631791+0000","last_became_peered":"2024-09-21T06:34:32.631791+0000","last_unstale":"2024-09-21T06:35:04.998387+0000","last_undegraded":"2024-09-21T06:35:04.998387+0000","last_fullsized":"2024-09-21T06:35:04.998387+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:46.246892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.273005+0000","last_change":"2024-09-21T06:34:32.639974+0000","last_active":"2024-09-21T06:35:05.273005+0000","last_peered":"2024-09-21T06:35:05.273005+0000","last_clean":"2024-09-21T06:35:05.273005+0000","last_became_active":"2024-09-21T06:34:32.639631+0000","last_became_peered":"2024-09-21T06:34:32.639631+0000","last_unstale":"2024-09-21T06:35:05.273005+0000","last_undegraded":"2024-09-21T06:35:05.273005+0000","last_fullsized":"2024-09-21T06:35:05.273005+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:50.069378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995887+0000","last_change":"2024-09-21T06:34:32.645403+0000","last_active":"2024-09-21T06:35:04.995887+0000","last_peered":"2024-09-21T06:35:04.995887+0000","last_clean":"2024-09-21T06:35:04.995887+0000","last_became_active":"2024-09-21T06:34:32.644986+0000","last_became_peered":"2024-09-21T06:34:32.644986+0000","last_unstale":"2024-09-21T06:35:04.995887+0000","last_undegraded":"2024-09-21T06:35:04.995887+0000","last_fullsized":"2024-09-21T06:35:04.995887+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:43.640107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995183+0000","last_change":"2024-09-21T06:34:32.639933+0000","last_active":"2024-09-21T06:35:04.995183+0000","last_peered":"2024-09-21T06:35:04.995183+0000","last_clean":"2024-09-21T06:35:04.995183+0000","last_became_active":"2024-09-21T06:34:32.639438+0000","last_became_peered":"2024-09-21T06:34:32.639438+0000","last_unstale":"2024-09-21T06:35:04.995183+0000","last_undegraded":"2024-09-21T06:35:04.995183+0000","last_fullsized":"2024-09-21T06:35:04.995183+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:18.860219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000816+0000","last_change":"2024-09-21T06:34:32.648580+0000","last_active":"2024-09-21T06:35:05.000816+0000","last_peered":"2024-09-21T06:35:05.000816+0000","last_clean":"2024-09-21T06:35:05.000816+0000","last_became_active":"2024-09-21T06:34:32.648016+0000","last_became_peered":"2024-09-21T06:34:32.648016+0000","last_unstale":"2024-09-21T06:35:05.000816+0000","last_undegraded":"2024-09-21T06:35:05.000816+0000","last_fullsized":"2024-09-21T06:35:05.000816+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:52.858069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.996429+0000","last_change":"2024-09-21T06:34:32.633376+0000","last_active":"2024-09-21T06:35:04.996429+0000","last_peered":"2024-09-21T06:35:04.996429+0000","last_clean":"2024-09-21T06:35:04.996429+0000","last_became_active":"2024-09-21T06:34:32.631052+0000","last_became_peered":"2024-09-21T06:34:32.631052+0000","last_unstale":"2024-09-21T06:35:04.996429+0000","last_undegraded":"2024-09-21T06:35:04.996429+0000","last_fullsized":"2024-09-21T06:35:04.996429+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:46.229599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.114206+0000","last_change":"2024-09-21T06:34:32.636966+0000","last_active":"2024-09-21T06:35:05.114206+0000","last_peered":"2024-09-21T06:35:05.114206+0000","last_clean":"2024-09-21T06:35:05.114206+0000","last_became_active":"2024-09-21T06:34:32.636565+0000","last_became_peered":"2024-09-21T06:34:32.636565+0000","last_unstale":"2024-09-21T06:35:05.114206+0000","last_undegraded":"2024-09-21T06:35:05.114206+0000","last_fullsized":"2024-09-21T06:35:05.114206+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:36.233082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.001311+0000","last_change":"2024-09-21T06:34:32.646895+0000","last_active":"2024-09-21T06:35:05.001311+0000","last_peered":"2024-09-21T06:35:05.001311+0000","last_clean":"2024-09-21T06:35:05.001311+0000","last_became_active":"2024-09-21T06:34:32.646505+0000","last_became_peered":"2024-09-21T06:34:32.646505+0000","last_unstale":"2024-09-21T06:35:05.001311+0000","last_undegraded":"2024-09-21T06:35:05.001311+0000","last_fullsized":"2024-09-21T06:35:05.001311+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:43.441337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995353+0000","last_change":"2024-09-21T06:34:32.645402+0000","last_active":"2024-09-21T06:35:04.995353+0000","last_peered":"2024-09-21T06:35:04.995353+0000","last_clean":"2024-09-21T06:35:04.995353+0000","last_became_active":"2024-09-21T06:34:32.645013+0000","last_became_peered":"2024-09-21T06:34:32.645013+0000","last_unstale":"2024-09-21T06:35:04.995353+0000","last_undegraded":"2024-09-21T06:35:04.995353+0000","last_fullsized":"2024-09-21T06:35:04.995353+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:33.475847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999814+0000","last_change":"2024-09-21T06:34:32.643234+0000","last_active":"2024-09-21T06:35:04.999814+0000","last_peered":"2024-09-21T06:35:04.999814+0000","last_clean":"2024-09-21T06:35:04.999814+0000","last_became_active":"2024-09-21T06:34:32.642614+0000","last_became_peered":"2024-09-21T06:34:32.642614+0000","last_unstale":"2024-09-21T06:35:04.999814+0000","last_undegraded":"2024-09-21T06:35:04.999814+0000","last_fullsized":"2024-09-21T06:35:04.999814+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:50.903500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999942+0000","last_change":"2024-09-21T06:34:32.641887+0000","last_active":"2024-09-21T06:35:04.999942+0000","last_peered":"2024-09-21T06:35:04.999942+0000","last_clean":"2024-09-21T06:35:04.999942+0000","last_became_active":"2024-09-21T06:34:32.641490+0000","last_became_peered":"2024-09-21T06:34:32.641490+0000","last_unstale":"2024-09-21T06:35:04.999942+0000","last_undegraded":"2024-09-21T06:35:04.999942+0000","last_fullsized":"2024-09-21T06:35:04.999942+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:34.550970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995289+0000","last_change":"2024-09-21T06:34:32.641865+0000","last_active":"2024-09-21T06:35:04.995289+0000","last_peered":"2024-09-21T06:35:04.995289+0000","last_clean":"2024-09-21T06:35:04.995289+0000","last_became_active":"2024-09-21T06:34:32.641252+0000","last_became_peered":"2024-09-21T06:34:32.641252+0000","last_unstale":"2024-09-21T06:35:04.995289+0000","last_undegraded":"2024-09-21T06:35:04.995289+0000","last_fullsized":"2024-09-21T06:35:04.995289+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:44.459202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997920+0000","last_change":"2024-09-21T06:34:32.641259+0000","last_active":"2024-09-21T06:35:04.997920+0000","last_peered":"2024-09-21T06:35:04.997920+0000","last_clean":"2024-09-21T06:35:04.997920+0000","last_became_active":"2024-09-21T06:34:32.640907+0000","last_became_peered":"2024-09-21T06:34:32.640907+0000","last_unstale":"2024-09-21T06:35:04.997920+0000","last_undegraded":"2024-09-21T06:35:04.997920+0000","last_fullsized":"2024-09-21T06:35:04.997920+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:59.966857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.030054+0000","last_change":"2024-09-21T06:34:32.646095+0000","last_active":"2024-09-21T06:35:05.030054+0000","last_peered":"2024-09-21T06:35:05.030054+0000","last_clean":"2024-09-21T06:35:05.030054+0000","last_became_active":"2024-09-21T06:34:32.645604+0000","last_became_peered":"2024-09-21T06:34:32.645604+0000","last_unstale":"2024-09-21T06:35:05.030054+0000","last_undegraded":"2024-09-21T06:35:05.030054+0000","last_fullsized":"2024-09-21T06:35:05.030054+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:30.872428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.001219+0000","last_change":"2024-09-21T06:34:32.649303+0000","last_active":"2024-09-21T06:35:05.001219+0000","last_peered":"2024-09-21T06:35:05.001219+0000","last_clean":"2024-09-21T06:35:05.001219+0000","last_became_active":"2024-09-21T06:34:32.648965+0000","last_became_peered":"2024-09-21T06:34:32.648965+0000","last_unstale":"2024-09-21T06:35:05.001219+0000","last_undegraded":"2024-09-21T06:35:05.001219+0000","last_fullsized":"2024-09-21T06:35:05.001219+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:51.335949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.996335+0000","last_change":"2024-09-21T06:34:32.637261+0000","last_active":"2024-09-21T06:35:04.996335+0000","last_peered":"2024-09-21T06:35:04.996335+0000","last_clean":"2024-09-21T06:35:04.996335+0000","last_became_active":"2024-09-21T06:34:32.636861+0000","last_became_peered":"2024-09-21T06:34:32.636861+0000","last_unstale":"2024-09-21T06:35:04.996335+0000","last_undegraded":"2024-09-21T06:35:04.996335+0000","last_fullsized":"2024-09-21T06:35:04.996335+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:55.271027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.272766+0000","last_change":"2024-09-21T06:34:32.635156+0000","last_active":"2024-09-21T06:35:05.272766+0000","last_peered":"2024-09-21T06:35:05.272766+0000","last_clean":"2024-09-21T06:35:05.272766+0000","last_became_active":"2024-09-21T06:34:32.634802+0000","last_became_peered":"2024-09-21T06:34:32.634802+0000","last_unstale":"2024-09-21T06:35:05.272766+0000","last_undegraded":"2024-09-21T06:35:05.272766+0000","last_fullsized":"2024-09-21T06:35:05.272766+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:16.139829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029506+0000","last_change":"2024-09-21T06:34:32.637509+0000","last_active":"2024-09-21T06:35:05.029506+0000","last_peered":"2024-09-21T06:35:05.029506+0000","last_clean":"2024-09-21T06:35:05.029506+0000","last_became_active":"2024-09-21T06:34:32.637136+0000","last_became_peered":"2024-09-21T06:34:32.637136+0000","last_unstale":"2024-09-21T06:35:05.029506+0000","last_undegraded":"2024-09-21T06:35:05.029506+0000","last_fullsized":"2024-09-21T06:35:05.029506+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:48.088988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.114942+0000","last_change":"2024-09-21T06:35:05.114819+0000","last_active":"2024-09-21T06:35:05.114942+0000","last_peered":"2024-09-21T06:35:05.114942+0000","last_clean":"2024-09-21T06:35:05.114942+0000","last_became_active":"2024-09-21T06:35:05.114424+0000","last_became_peered":"2024-09-21T06:35:05.114424+0000","last_unstale":"2024-09-21T06:35:05.114942+0000","last_undegraded":"2024-09-21T06:35:05.114942+0000","last_fullsized":"2024-09-21T06:35:05.114942+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:59.323643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.003387+0000","last_change":"2024-09-21T06:34:32.636103+0000","last_active":"2024-09-21T06:35:05.003387+0000","last_peered":"2024-09-21T06:35:05.003387+0000","last_clean":"2024-09-21T06:35:05.003387+0000","last_became_active":"2024-09-21T06:34:32.635728+0000","last_became_peered":"2024-09-21T06:34:32.635728+0000","last_unstale":"2024-09-21T06:35:05.003387+0000","last_undegraded":"2024-09-21T06:35:05.003387+0000","last_fullsized":"2024-09-21T06:35:05.003387+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:16.731402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997848+0000","last_change":"2024-09-21T06:34:32.635654+0000","last_active":"2024-09-21T06:35:04.997848+0000","last_peered":"2024-09-21T06:35:04.997848+0000","last_clean":"2024-09-21T06:35:04.997848+0000","last_became_active":"2024-09-21T06:34:32.635265+0000","last_became_peered":"2024-09-21T06:34:32.635265+0000","last_unstale":"2024-09-21T06:35:04.997848+0000","last_undegraded":"2024-09-21T06:35:04.997848+0000","last_fullsized":"2024-09-21T06:35:04.997848+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:28.169625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000238+0000","last_change":"2024-09-21T06:34:32.644640+0000","last_active":"2024-09-21T06:35:05.000238+0000","last_peered":"2024-09-21T06:35:05.000238+0000","last_clean":"2024-09-21T06:35:05.000238+0000","last_became_active":"2024-09-21T06:34:32.644334+0000","last_became_peered":"2024-09-21T06:34:32.644334+0000","last_unstale":"2024-09-21T06:35:05.000238+0000","last_undegraded":"2024-09-21T06:35:05.000238+0000","last_fullsized":"2024-09-21T06:35:05.000238+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:25.572178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000396+0000","last_change":"2024-09-21T06:34:32.643705+0000","last_active":"2024-09-21T06:35:05.000396+0000","last_peered":"2024-09-21T06:35:05.000396+0000","last_clean":"2024-09-21T06:35:05.000396+0000","last_became_active":"2024-09-21T06:34:32.642705+0000","last_became_peered":"2024-09-21T06:34:32.642705+0000","last_unstale":"2024-09-21T06:35:05.000396+0000","last_undegraded":"2024-09-21T06:35:05.000396+0000","last_fullsized":"2024-09-21T06:35:05.000396+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:22.542559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.001285+0000","last_change":"2024-09-21T06:34:32.646539+0000","last_active":"2024-09-21T06:35:05.001285+0000","last_peered":"2024-09-21T06:35:05.001285+0000","last_clean":"2024-09-21T06:35:05.001285+0000","last_became_active":"2024-09-21T06:34:32.645992+0000","last_became_peered":"2024-09-21T06:34:32.645992+0000","last_unstale":"2024-09-21T06:35:05.001285+0000","last_undegraded":"2024-09-21T06:35:05.001285+0000","last_fullsized":"2024-09-21T06:35:05.001285+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:38.856001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":49417,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":37,"num_read_kb":69,"num_write":71,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":233472,"data_stored":148251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":61,"ondisk_log_size":61,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282016,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":466743,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939675,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1019904,"data_stored":1101828,"data_compressed":59457,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103201,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":471421,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299430,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":467347,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495659,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":1061789,"data_compressed":59457,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659185,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":901120,"data_stored":462647,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855414,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":1057293,"data_compressed":59457,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018939,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":466765,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215167,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":462751,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":511300,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":511266,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":905216,"data_stored":462737,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":48619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":48653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":48813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:35:24.734 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T06:35:24.995 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:25.382 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:25 smithi067 ceph-mon[31232]: pgmap v323: 129 pgs: 129 active+clean; 625 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.6 KiB/s wr, 3 op/s 2024-09-21T06:35:25.382 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:25 smithi067 ceph-mon[31232]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:35:25.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:25 smithi203 ceph-mon[33735]: pgmap v323: 129 pgs: 129 active+clean; 625 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.6 KiB/s wr, 3 op/s 2024-09-21T06:35:25.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:25 smithi203 ceph-mon[33735]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:35:25.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.473+0000 7f653b8c9640 1 -- 172.21.15.67:0/135999438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6534102a90 msgr2=0x7f6534102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:25.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.473+0000 7f653b8c9640 1 --2- 172.21.15.67:0/135999438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6534102a90 0x7f6534102ea0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6530009900 tx=0x7f653002a410 comp rx=0 tx=0).stop 2024-09-21T06:35:25.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.473+0000 7f653b8c9640 1 -- 172.21.15.67:0/135999438 shutdown_connections 2024-09-21T06:35:25.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.473+0000 7f653b8c9640 1 --2- 172.21.15.67:0/135999438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6534108d20 0x7f653410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:25.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.473+0000 7f653b8c9640 1 --2- 172.21.15.67:0/135999438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65341083d0 0x7f65341087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:25.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.473+0000 7f653b8c9640 1 --2- 172.21.15.67:0/135999438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6534102a90 0x7f6534102ea0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:25.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.473+0000 7f653b8c9640 1 -- 172.21.15.67:0/135999438 >> 172.21.15.67:0/135999438 conn(0x7f6534079750 msgr2=0x7f6534079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:25.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.473+0000 7f653b8c9640 1 -- 172.21.15.67:0/135999438 shutdown_connections 2024-09-21T06:35:25.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.473+0000 7f653b8c9640 1 -- 172.21.15.67:0/135999438 wait complete. 2024-09-21T06:35:25.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.473+0000 7f653b8c9640 1 Processor -- start 2024-09-21T06:35:25.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.473+0000 7f653b8c9640 1 -- start start 2024-09-21T06:35:25.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.474+0000 7f653b8c9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6534102a90 0x7f6534104280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:25.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.474+0000 7f653b8c9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65341083d0 0x7f65341047c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:25.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.474+0000 7f653b8c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6534108d20 0x7f6534104d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:25.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.474+0000 7f653b8c9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6534112840 con 0x7f6534102a90 2024-09-21T06:35:25.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.474+0000 7f653b8c9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f65341126c0 con 0x7f6534108d20 2024-09-21T06:35:25.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.474+0000 7f653b8c9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f65341129c0 con 0x7f65341083d0 2024-09-21T06:35:25.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.474+0000 7f6539665640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6534102a90 0x7f6534104280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:25.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.474+0000 7f6539665640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6534102a90 0x7f6534104280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54860/0 (socket says 172.21.15.67:54860) 2024-09-21T06:35:25.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.474+0000 7f6539665640 1 -- 172.21.15.67:0/3462345422 learned_addr learned my addr 172.21.15.67:0/3462345422 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:25.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.474+0000 7f6538e64640 1 --2- 172.21.15.67:0/3462345422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65341083d0 0x7f65341047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:25.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.474+0000 7f6539e66640 1 --2- 172.21.15.67:0/3462345422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6534108d20 0x7f6534104d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:25.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.474+0000 7f6539665640 1 -- 172.21.15.67:0/3462345422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65341083d0 msgr2=0x7f65341047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:25.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.474+0000 7f6539665640 1 --2- 172.21.15.67:0/3462345422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65341083d0 0x7f65341047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:25.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.474+0000 7f6539665640 1 -- 172.21.15.67:0/3462345422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6534108d20 msgr2=0x7f6534104d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:25.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.474+0000 7f6539665640 1 --2- 172.21.15.67:0/3462345422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6534108d20 0x7f6534104d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:25.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.474+0000 7f6539665640 1 -- 172.21.15.67:0/3462345422 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65341a8660 con 0x7f6534102a90 2024-09-21T06:35:25.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.474+0000 7f6538e64640 1 --2- 172.21.15.67:0/3462345422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65341083d0 0x7f65341047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:25.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.474+0000 7f6539665640 1 --2- 172.21.15.67:0/3462345422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6534102a90 0x7f6534104280 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6530009230 tx=0x7f6530004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:25.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.475+0000 7f652a7fc640 1 -- 172.21.15.67:0/3462345422 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653003f030 con 0x7f6534102a90 2024-09-21T06:35:25.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.475+0000 7f653b8c9640 1 -- 172.21.15.67:0/3462345422 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65341a88f0 con 0x7f6534102a90 2024-09-21T06:35:25.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.475+0000 7f652a7fc640 1 -- 172.21.15.67:0/3462345422 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6530003a80 con 0x7f6534102a90 2024-09-21T06:35:25.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.475+0000 7f652a7fc640 1 -- 172.21.15.67:0/3462345422 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530035070 con 0x7f6534102a90 2024-09-21T06:35:25.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.475+0000 7f653b8c9640 1 -- 172.21.15.67:0/3462345422 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65341a8dd0 con 0x7f6534102a90 2024-09-21T06:35:25.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.476+0000 7f653b8c9640 1 -- 172.21.15.67:0/3462345422 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6534103d10 con 0x7f6534102a90 2024-09-21T06:35:25.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.477+0000 7f652a7fc640 1 -- 172.21.15.67:0/3462345422 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f6530042030 con 0x7f6534102a90 2024-09-21T06:35:25.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.478+0000 7f652a7fc640 1 --2- 172.21.15.67:0/3462345422 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f65200b65a0 0x7f65200b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:25.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.478+0000 7f6538e64640 1 --2- 172.21.15.67:0/3462345422 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f65200b65a0 0x7f65200b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:25.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.478+0000 7f652a7fc640 1 -- 172.21.15.67:0/3462345422 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f6530108480 con 0x7f6534102a90 2024-09-21T06:35:25.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.478+0000 7f6538e64640 1 --2- 172.21.15.67:0/3462345422 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f65200b65a0 0x7f65200b8ab0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f65240058c0 tx=0x7f6524005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:25.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.481+0000 7f652a7fc640 1 -- 172.21.15.67:0/3462345422 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f65300ce850 con 0x7f6534102a90 2024-09-21T06:35:25.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:25 smithi100 ceph-mon[33401]: pgmap v323: 129 pgs: 129 active+clean; 625 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.6 KiB/s wr, 3 op/s 2024-09-21T06:35:25.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:25 smithi100 ceph-mon[33401]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:35:25.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.666+0000 7f653b8c9640 1 -- 172.21.15.67:0/3462345422 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6534106350 con 0x7f65200b65a0 2024-09-21T06:35:25.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.672+0000 7f652a7fc640 1 -- 172.21.15.67:0/3462345422 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340526 (secure 0 0 0) 0x7f6534106350 con 0x7f65200b65a0 2024-09-21T06:35:25.678 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:35:25.679 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-09-21T06:35:25.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.676+0000 7f653b8c9640 1 -- 172.21.15.67:0/3462345422 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f65200b65a0 msgr2=0x7f65200b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:25.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.676+0000 7f653b8c9640 1 --2- 172.21.15.67:0/3462345422 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f65200b65a0 0x7f65200b8ab0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f65240058c0 tx=0x7f6524005850 comp rx=0 tx=0).stop 2024-09-21T06:35:25.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.676+0000 7f653b8c9640 1 -- 172.21.15.67:0/3462345422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6534102a90 msgr2=0x7f6534104280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:25.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.676+0000 7f653b8c9640 1 --2- 172.21.15.67:0/3462345422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6534102a90 0x7f6534104280 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6530009230 tx=0x7f6530004270 comp rx=0 tx=0).stop 2024-09-21T06:35:25.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.676+0000 7f653b8c9640 1 -- 172.21.15.67:0/3462345422 shutdown_connections 2024-09-21T06:35:25.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.676+0000 7f653b8c9640 1 --2- 172.21.15.67:0/3462345422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6534108d20 0x7f6534104d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:25.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.676+0000 7f653b8c9640 1 --2- 172.21.15.67:0/3462345422 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f65200b65a0 0x7f65200b8ab0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:25.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.676+0000 7f653b8c9640 1 --2- 172.21.15.67:0/3462345422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65341083d0 0x7f65341047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:25.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.676+0000 7f653b8c9640 1 --2- 172.21.15.67:0/3462345422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6534102a90 0x7f6534104280 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:25.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.676+0000 7f653b8c9640 1 -- 172.21.15.67:0/3462345422 >> 172.21.15.67:0/3462345422 conn(0x7f6534079750 msgr2=0x7f6534101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:25.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.676+0000 7f653b8c9640 1 -- 172.21.15.67:0/3462345422 shutdown_connections 2024-09-21T06:35:25.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:25.676+0000 7f653b8c9640 1 -- 172.21.15.67:0/3462345422 wait complete. 2024-09-21T06:35:26.430 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-09-21T06:35:23.849362+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":640019,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":113,"num_read_kb":133,"num_write":196,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364388,"kb_used_data":10964,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577436,"statfs":{"total":1151940427776,"available":1151567294464,"internally_reserved":0,"allocated":11227136,"data_stored":7503887,"data_compressed":390735,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":45011,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":56,"num_write":24,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003431"},"pg_stats":[{"pgid":"2.1f","version":"75'4","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.114370+0000","last_change":"2024-09-21T06:34:30.633366+0000","last_active":"2024-09-21T06:35:05.114370+0000","last_peered":"2024-09-21T06:35:05.114370+0000","last_clean":"2024-09-21T06:35:05.114370+0000","last_became_active":"2024-09-21T06:34:30.632929+0000","last_became_peered":"2024-09-21T06:34:30.632929+0000","last_unstale":"2024-09-21T06:35:05.114370+0000","last_undegraded":"2024-09-21T06:35:05.114370+0000","last_fullsized":"2024-09-21T06:35:05.114370+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:47.722461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.272950+0000","last_change":"2024-09-21T06:34:32.632989+0000","last_active":"2024-09-21T06:35:05.272950+0000","last_peered":"2024-09-21T06:35:05.272950+0000","last_clean":"2024-09-21T06:35:05.272950+0000","last_became_active":"2024-09-21T06:34:32.632024+0000","last_became_peered":"2024-09-21T06:34:32.632024+0000","last_unstale":"2024-09-21T06:35:05.272950+0000","last_undegraded":"2024-09-21T06:35:05.272950+0000","last_fullsized":"2024-09-21T06:35:05.272950+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:13.669613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.274531+0000","last_change":"2024-09-21T06:35:05.274434+0000","last_active":"2024-09-21T06:35:05.274531+0000","last_peered":"2024-09-21T06:35:05.274531+0000","last_clean":"2024-09-21T06:35:05.274531+0000","last_became_active":"2024-09-21T06:35:05.273833+0000","last_became_peered":"2024-09-21T06:35:05.273833+0000","last_unstale":"2024-09-21T06:35:05.274531+0000","last_undegraded":"2024-09-21T06:35:05.274531+0000","last_fullsized":"2024-09-21T06:35:05.274531+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:48.847861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.273266+0000","last_change":"2024-09-21T06:34:30.946309+0000","last_active":"2024-09-21T06:35:05.273266+0000","last_peered":"2024-09-21T06:35:05.273266+0000","last_clean":"2024-09-21T06:35:05.273266+0000","last_became_active":"2024-09-21T06:34:30.945913+0000","last_became_peered":"2024-09-21T06:34:30.945913+0000","last_unstale":"2024-09-21T06:35:05.273266+0000","last_undegraded":"2024-09-21T06:35:05.273266+0000","last_fullsized":"2024-09-21T06:35:05.273266+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:21.676524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000669+0000","last_change":"2024-09-21T06:34:32.637636+0000","last_active":"2024-09-21T06:35:05.000669+0000","last_peered":"2024-09-21T06:35:05.000669+0000","last_clean":"2024-09-21T06:35:05.000669+0000","last_became_active":"2024-09-21T06:34:32.636534+0000","last_became_peered":"2024-09-21T06:34:32.636534+0000","last_unstale":"2024-09-21T06:35:05.000669+0000","last_undegraded":"2024-09-21T06:35:05.000669+0000","last_fullsized":"2024-09-21T06:35:05.000669+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:05.802118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.274538+0000","last_change":"2024-09-21T06:35:05.274419+0000","last_active":"2024-09-21T06:35:05.274538+0000","last_peered":"2024-09-21T06:35:05.274538+0000","last_clean":"2024-09-21T06:35:05.274538+0000","last_became_active":"2024-09-21T06:35:05.273865+0000","last_became_peered":"2024-09-21T06:35:05.273865+0000","last_unstale":"2024-09-21T06:35:05.274538+0000","last_undegraded":"2024-09-21T06:35:05.274538+0000","last_fullsized":"2024-09-21T06:35:05.274538+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:04.490355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.004199+0000","last_change":"2024-09-21T06:34:30.636856+0000","last_active":"2024-09-21T06:35:05.004199+0000","last_peered":"2024-09-21T06:35:05.004199+0000","last_clean":"2024-09-21T06:35:05.004199+0000","last_became_active":"2024-09-21T06:34:30.636460+0000","last_became_peered":"2024-09-21T06:34:30.636460+0000","last_unstale":"2024-09-21T06:35:05.004199+0000","last_undegraded":"2024-09-21T06:35:05.004199+0000","last_fullsized":"2024-09-21T06:35:05.004199+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:58.917851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995796+0000","last_change":"2024-09-21T06:34:32.650303+0000","last_active":"2024-09-21T06:35:04.995796+0000","last_peered":"2024-09-21T06:35:04.995796+0000","last_clean":"2024-09-21T06:35:04.995796+0000","last_became_active":"2024-09-21T06:34:32.647189+0000","last_became_peered":"2024-09-21T06:34:32.647189+0000","last_unstale":"2024-09-21T06:35:04.995796+0000","last_undegraded":"2024-09-21T06:35:04.995796+0000","last_fullsized":"2024-09-21T06:35:04.995796+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:14.889384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.996942+0000","last_change":"2024-09-21T06:34:39.713139+0000","last_active":"2024-09-21T06:35:04.996942+0000","last_peered":"2024-09-21T06:35:04.996942+0000","last_clean":"2024-09-21T06:35:04.996942+0000","last_became_active":"2024-09-21T06:34:39.712379+0000","last_became_peered":"2024-09-21T06:34:39.712379+0000","last_unstale":"2024-09-21T06:35:04.996942+0000","last_undegraded":"2024-09-21T06:35:04.996942+0000","last_fullsized":"2024-09-21T06:35:04.996942+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:53.100529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.004039+0000","last_change":"2024-09-21T06:34:30.633850+0000","last_active":"2024-09-21T06:35:05.004039+0000","last_peered":"2024-09-21T06:35:05.004039+0000","last_clean":"2024-09-21T06:35:05.004039+0000","last_became_active":"2024-09-21T06:34:30.633468+0000","last_became_peered":"2024-09-21T06:34:30.633468+0000","last_unstale":"2024-09-21T06:35:05.004039+0000","last_undegraded":"2024-09-21T06:35:05.004039+0000","last_fullsized":"2024-09-21T06:35:05.004039+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:59.095803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.005361+0000","last_change":"2024-09-21T06:35:05.005240+0000","last_active":"2024-09-21T06:35:05.005361+0000","last_peered":"2024-09-21T06:35:05.005361+0000","last_clean":"2024-09-21T06:35:05.005361+0000","last_became_active":"2024-09-21T06:35:05.004849+0000","last_became_peered":"2024-09-21T06:35:05.004849+0000","last_unstale":"2024-09-21T06:35:05.005361+0000","last_undegraded":"2024-09-21T06:35:05.005361+0000","last_fullsized":"2024-09-21T06:35:05.005361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:28.849604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.274638+0000","last_change":"2024-09-21T06:35:05.274437+0000","last_active":"2024-09-21T06:35:05.274638+0000","last_peered":"2024-09-21T06:35:05.274638+0000","last_clean":"2024-09-21T06:35:05.274638+0000","last_became_active":"2024-09-21T06:35:05.273865+0000","last_became_peered":"2024-09-21T06:35:05.273865+0000","last_unstale":"2024-09-21T06:35:05.274638+0000","last_undegraded":"2024-09-21T06:35:05.274638+0000","last_fullsized":"2024-09-21T06:35:05.274638+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:43.339504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029725+0000","last_change":"2024-09-21T06:34:32.645630+0000","last_active":"2024-09-21T06:35:05.029725+0000","last_peered":"2024-09-21T06:35:05.029725+0000","last_clean":"2024-09-21T06:35:05.029725+0000","last_became_active":"2024-09-21T06:34:32.645237+0000","last_became_peered":"2024-09-21T06:34:32.645237+0000","last_unstale":"2024-09-21T06:35:05.029725+0000","last_undegraded":"2024-09-21T06:35:05.029725+0000","last_fullsized":"2024-09-21T06:35:05.029725+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:25.064318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.003627+0000","last_change":"2024-09-21T06:34:30.636609+0000","last_active":"2024-09-21T06:35:05.003627+0000","last_peered":"2024-09-21T06:35:05.003627+0000","last_clean":"2024-09-21T06:35:05.003627+0000","last_became_active":"2024-09-21T06:34:30.636217+0000","last_became_peered":"2024-09-21T06:34:30.636217+0000","last_unstale":"2024-09-21T06:35:05.003627+0000","last_undegraded":"2024-09-21T06:35:05.003627+0000","last_fullsized":"2024-09-21T06:35:05.003627+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:32.541307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997116+0000","last_change":"2024-09-21T06:34:39.714177+0000","last_active":"2024-09-21T06:35:04.997116+0000","last_peered":"2024-09-21T06:35:04.997116+0000","last_clean":"2024-09-21T06:35:04.997116+0000","last_became_active":"2024-09-21T06:34:39.713713+0000","last_became_peered":"2024-09-21T06:34:39.713713+0000","last_unstale":"2024-09-21T06:35:04.997116+0000","last_undegraded":"2024-09-21T06:35:04.997116+0000","last_fullsized":"2024-09-21T06:35:04.997116+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:07.042240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.272625+0000","last_change":"2024-09-21T06:34:32.636297+0000","last_active":"2024-09-21T06:35:05.272625+0000","last_peered":"2024-09-21T06:35:05.272625+0000","last_clean":"2024-09-21T06:35:05.272625+0000","last_became_active":"2024-09-21T06:34:32.635930+0000","last_became_peered":"2024-09-21T06:34:32.635930+0000","last_unstale":"2024-09-21T06:35:05.272625+0000","last_undegraded":"2024-09-21T06:35:05.272625+0000","last_fullsized":"2024-09-21T06:35:05.272625+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:43.091682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995838+0000","last_change":"2024-09-21T06:34:30.631405+0000","last_active":"2024-09-21T06:35:04.995838+0000","last_peered":"2024-09-21T06:35:04.995838+0000","last_clean":"2024-09-21T06:35:04.995838+0000","last_became_active":"2024-09-21T06:34:30.630512+0000","last_became_peered":"2024-09-21T06:34:30.630512+0000","last_unstale":"2024-09-21T06:35:04.995838+0000","last_undegraded":"2024-09-21T06:35:04.995838+0000","last_fullsized":"2024-09-21T06:35:04.995838+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:43.412695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"83'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997680+0000","last_change":"2024-09-21T06:34:39.710573+0000","last_active":"2024-09-21T06:35:04.997680+0000","last_peered":"2024-09-21T06:35:04.997680+0000","last_clean":"2024-09-21T06:35:04.997680+0000","last_became_active":"2024-09-21T06:34:39.708969+0000","last_became_peered":"2024-09-21T06:34:39.708969+0000","last_unstale":"2024-09-21T06:35:04.997680+0000","last_undegraded":"2024-09-21T06:35:04.997680+0000","last_fullsized":"2024-09-21T06:35:04.997680+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:34.847587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000559+0000","last_change":"2024-09-21T06:34:32.641085+0000","last_active":"2024-09-21T06:35:05.000559+0000","last_peered":"2024-09-21T06:35:05.000559+0000","last_clean":"2024-09-21T06:35:05.000559+0000","last_became_active":"2024-09-21T06:34:32.640723+0000","last_became_peered":"2024-09-21T06:34:32.640723+0000","last_unstale":"2024-09-21T06:35:05.000559+0000","last_undegraded":"2024-09-21T06:35:05.000559+0000","last_fullsized":"2024-09-21T06:35:05.000559+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:21.410262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.273157+0000","last_change":"2024-09-21T06:34:30.626719+0000","last_active":"2024-09-21T06:35:05.273157+0000","last_peered":"2024-09-21T06:35:05.273157+0000","last_clean":"2024-09-21T06:35:05.273157+0000","last_became_active":"2024-09-21T06:34:30.626312+0000","last_became_peered":"2024-09-21T06:34:30.626312+0000","last_unstale":"2024-09-21T06:35:05.273157+0000","last_undegraded":"2024-09-21T06:35:05.273157+0000","last_fullsized":"2024-09-21T06:35:05.273157+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:53.938914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.114020+0000","last_change":"2024-09-21T06:34:39.711172+0000","last_active":"2024-09-21T06:35:05.114020+0000","last_peered":"2024-09-21T06:35:05.114020+0000","last_clean":"2024-09-21T06:35:05.114020+0000","last_became_active":"2024-09-21T06:34:39.708424+0000","last_became_peered":"2024-09-21T06:34:39.708424+0000","last_unstale":"2024-09-21T06:35:05.114020+0000","last_undegraded":"2024-09-21T06:35:05.114020+0000","last_fullsized":"2024-09-21T06:35:05.114020+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:43.918355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.003179+0000","last_change":"2024-09-21T06:34:32.641463+0000","last_active":"2024-09-21T06:35:05.003179+0000","last_peered":"2024-09-21T06:35:05.003179+0000","last_clean":"2024-09-21T06:35:05.003179+0000","last_became_active":"2024-09-21T06:34:32.641086+0000","last_became_peered":"2024-09-21T06:34:32.641086+0000","last_unstale":"2024-09-21T06:35:05.003179+0000","last_undegraded":"2024-09-21T06:35:05.003179+0000","last_fullsized":"2024-09-21T06:35:05.003179+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:58.991374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995541+0000","last_change":"2024-09-21T06:34:30.946698+0000","last_active":"2024-09-21T06:35:04.995541+0000","last_peered":"2024-09-21T06:35:04.995541+0000","last_clean":"2024-09-21T06:35:04.995541+0000","last_became_active":"2024-09-21T06:34:30.946325+0000","last_became_peered":"2024-09-21T06:34:30.946325+0000","last_unstale":"2024-09-21T06:35:04.995541+0000","last_undegraded":"2024-09-21T06:35:04.995541+0000","last_fullsized":"2024-09-21T06:35:04.995541+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:22.639265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997471+0000","last_change":"2024-09-21T06:34:39.712429+0000","last_active":"2024-09-21T06:35:04.997471+0000","last_peered":"2024-09-21T06:35:04.997471+0000","last_clean":"2024-09-21T06:35:04.997471+0000","last_became_active":"2024-09-21T06:34:39.711823+0000","last_became_peered":"2024-09-21T06:34:39.711823+0000","last_unstale":"2024-09-21T06:35:04.997471+0000","last_undegraded":"2024-09-21T06:35:04.997471+0000","last_fullsized":"2024-09-21T06:35:04.997471+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:03:49.513991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.274438+0000","last_change":"2024-09-21T06:35:05.274326+0000","last_active":"2024-09-21T06:35:05.274438+0000","last_peered":"2024-09-21T06:35:05.274438+0000","last_clean":"2024-09-21T06:35:05.274438+0000","last_became_active":"2024-09-21T06:35:05.273884+0000","last_became_peered":"2024-09-21T06:35:05.273884+0000","last_unstale":"2024-09-21T06:35:05.274438+0000","last_undegraded":"2024-09-21T06:35:05.274438+0000","last_fullsized":"2024-09-21T06:35:05.274438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:03.219765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997399+0000","last_change":"2024-09-21T06:34:30.676794+0000","last_active":"2024-09-21T06:35:04.997399+0000","last_peered":"2024-09-21T06:35:04.997399+0000","last_clean":"2024-09-21T06:35:04.997399+0000","last_became_active":"2024-09-21T06:34:30.676400+0000","last_became_peered":"2024-09-21T06:34:30.676400+0000","last_unstale":"2024-09-21T06:35:04.997399+0000","last_undegraded":"2024-09-21T06:35:04.997399+0000","last_fullsized":"2024-09-21T06:35:04.997399+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:23.681363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995836+0000","last_change":"2024-09-21T06:34:39.708980+0000","last_active":"2024-09-21T06:35:04.995836+0000","last_peered":"2024-09-21T06:35:04.995836+0000","last_clean":"2024-09-21T06:35:04.995836+0000","last_became_active":"2024-09-21T06:34:39.707748+0000","last_became_peered":"2024-09-21T06:34:39.707748+0000","last_unstale":"2024-09-21T06:35:04.995836+0000","last_undegraded":"2024-09-21T06:35:04.995836+0000","last_fullsized":"2024-09-21T06:35:04.995836+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:16.812226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.273262+0000","last_change":"2024-09-21T06:34:32.638337+0000","last_active":"2024-09-21T06:35:05.273262+0000","last_peered":"2024-09-21T06:35:05.273262+0000","last_clean":"2024-09-21T06:35:05.273262+0000","last_became_active":"2024-09-21T06:34:32.638013+0000","last_became_peered":"2024-09-21T06:34:32.638013+0000","last_unstale":"2024-09-21T06:35:05.273262+0000","last_undegraded":"2024-09-21T06:35:05.273262+0000","last_fullsized":"2024-09-21T06:35:05.273262+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:26.321190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"83'5","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:17.661508+0000","last_change":"2024-09-21T06:34:30.631086+0000","last_active":"2024-09-21T06:35:17.661508+0000","last_peered":"2024-09-21T06:35:17.661508+0000","last_clean":"2024-09-21T06:35:17.661508+0000","last_became_active":"2024-09-21T06:34:30.630706+0000","last_became_peered":"2024-09-21T06:34:30.630706+0000","last_unstale":"2024-09-21T06:35:17.661508+0000","last_undegraded":"2024-09-21T06:35:17.661508+0000","last_fullsized":"2024-09-21T06:35:17.661508+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:30.260359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.272633+0000","last_change":"2024-09-21T06:34:39.709097+0000","last_active":"2024-09-21T06:35:05.272633+0000","last_peered":"2024-09-21T06:35:05.272633+0000","last_clean":"2024-09-21T06:35:05.272633+0000","last_became_active":"2024-09-21T06:34:39.708609+0000","last_became_peered":"2024-09-21T06:34:39.708609+0000","last_unstale":"2024-09-21T06:35:05.272633+0000","last_undegraded":"2024-09-21T06:35:05.272633+0000","last_fullsized":"2024-09-21T06:35:05.272633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:48.842174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995363+0000","last_change":"2024-09-21T06:34:32.636816+0000","last_active":"2024-09-21T06:35:04.995363+0000","last_peered":"2024-09-21T06:35:04.995363+0000","last_clean":"2024-09-21T06:35:04.995363+0000","last_became_active":"2024-09-21T06:34:32.636429+0000","last_became_peered":"2024-09-21T06:34:32.636429+0000","last_unstale":"2024-09-21T06:35:04.995363+0000","last_undegraded":"2024-09-21T06:35:04.995363+0000","last_fullsized":"2024-09-21T06:35:04.995363+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:49.010491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.004246+0000","last_change":"2024-09-21T06:34:30.947481+0000","last_active":"2024-09-21T06:35:05.004246+0000","last_peered":"2024-09-21T06:35:05.004246+0000","last_clean":"2024-09-21T06:35:05.004246+0000","last_became_active":"2024-09-21T06:34:30.947040+0000","last_became_peered":"2024-09-21T06:34:30.947040+0000","last_unstale":"2024-09-21T06:35:05.004246+0000","last_undegraded":"2024-09-21T06:35:05.004246+0000","last_fullsized":"2024-09-21T06:35:05.004246+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:52.813476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000546+0000","last_change":"2024-09-21T06:34:39.708674+0000","last_active":"2024-09-21T06:35:05.000546+0000","last_peered":"2024-09-21T06:35:05.000546+0000","last_clean":"2024-09-21T06:35:05.000546+0000","last_became_active":"2024-09-21T06:34:39.708176+0000","last_became_peered":"2024-09-21T06:34:39.708176+0000","last_unstale":"2024-09-21T06:35:05.000546+0000","last_undegraded":"2024-09-21T06:35:05.000546+0000","last_fullsized":"2024-09-21T06:35:05.000546+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:57.070792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000844+0000","last_change":"2024-09-21T06:34:32.636381+0000","last_active":"2024-09-21T06:35:05.000844+0000","last_peered":"2024-09-21T06:35:05.000844+0000","last_clean":"2024-09-21T06:35:05.000844+0000","last_became_active":"2024-09-21T06:34:32.635937+0000","last_became_peered":"2024-09-21T06:34:32.635937+0000","last_unstale":"2024-09-21T06:35:05.000844+0000","last_undegraded":"2024-09-21T06:35:05.000844+0000","last_fullsized":"2024-09-21T06:35:05.000844+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:52.870676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.114033+0000","last_change":"2024-09-21T06:34:30.947160+0000","last_active":"2024-09-21T06:35:05.114033+0000","last_peered":"2024-09-21T06:35:05.114033+0000","last_clean":"2024-09-21T06:35:05.114033+0000","last_became_active":"2024-09-21T06:34:30.946736+0000","last_became_peered":"2024-09-21T06:34:30.946736+0000","last_unstale":"2024-09-21T06:35:05.114033+0000","last_undegraded":"2024-09-21T06:35:05.114033+0000","last_fullsized":"2024-09-21T06:35:05.114033+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:47.425481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.006145+0000","last_change":"2024-09-21T06:35:05.006032+0000","last_active":"2024-09-21T06:35:05.006145+0000","last_peered":"2024-09-21T06:35:05.006145+0000","last_clean":"2024-09-21T06:35:05.006145+0000","last_became_active":"2024-09-21T06:35:05.005429+0000","last_became_peered":"2024-09-21T06:35:05.005429+0000","last_unstale":"2024-09-21T06:35:05.006145+0000","last_undegraded":"2024-09-21T06:35:05.006145+0000","last_fullsized":"2024-09-21T06:35:05.006145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:04.138097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000613+0000","last_change":"2024-09-21T06:34:32.648256+0000","last_active":"2024-09-21T06:35:05.000613+0000","last_peered":"2024-09-21T06:35:05.000613+0000","last_clean":"2024-09-21T06:35:05.000613+0000","last_became_active":"2024-09-21T06:34:32.647881+0000","last_became_peered":"2024-09-21T06:34:32.647881+0000","last_unstale":"2024-09-21T06:35:05.000613+0000","last_undegraded":"2024-09-21T06:35:05.000613+0000","last_fullsized":"2024-09-21T06:35:05.000613+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:14.590834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029849+0000","last_change":"2024-09-21T06:34:30.629918+0000","last_active":"2024-09-21T06:35:05.029849+0000","last_peered":"2024-09-21T06:35:05.029849+0000","last_clean":"2024-09-21T06:35:05.029849+0000","last_became_active":"2024-09-21T06:34:30.628785+0000","last_became_peered":"2024-09-21T06:34:30.628785+0000","last_unstale":"2024-09-21T06:35:05.029849+0000","last_undegraded":"2024-09-21T06:35:05.029849+0000","last_fullsized":"2024-09-21T06:35:05.029849+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:05.333406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995102+0000","last_change":"2024-09-21T06:34:39.710073+0000","last_active":"2024-09-21T06:35:04.995102+0000","last_peered":"2024-09-21T06:35:04.995102+0000","last_clean":"2024-09-21T06:35:04.995102+0000","last_became_active":"2024-09-21T06:34:39.708986+0000","last_became_peered":"2024-09-21T06:34:39.708986+0000","last_unstale":"2024-09-21T06:35:04.995102+0000","last_undegraded":"2024-09-21T06:35:04.995102+0000","last_fullsized":"2024-09-21T06:35:04.995102+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:46.379902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.998454+0000","last_change":"2024-09-21T06:34:32.644224+0000","last_active":"2024-09-21T06:35:04.998454+0000","last_peered":"2024-09-21T06:35:04.998454+0000","last_clean":"2024-09-21T06:35:04.998454+0000","last_became_active":"2024-09-21T06:34:32.643850+0000","last_became_peered":"2024-09-21T06:34:32.643850+0000","last_unstale":"2024-09-21T06:35:04.998454+0000","last_undegraded":"2024-09-21T06:35:04.998454+0000","last_fullsized":"2024-09-21T06:35:04.998454+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:24:33.188537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.994875+0000","last_change":"2024-09-21T06:34:30.946334+0000","last_active":"2024-09-21T06:35:04.994875+0000","last_peered":"2024-09-21T06:35:04.994875+0000","last_clean":"2024-09-21T06:35:04.994875+0000","last_became_active":"2024-09-21T06:34:30.945908+0000","last_became_peered":"2024-09-21T06:34:30.945908+0000","last_unstale":"2024-09-21T06:35:04.994875+0000","last_undegraded":"2024-09-21T06:35:04.994875+0000","last_fullsized":"2024-09-21T06:35:04.994875+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:46.746527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995193+0000","last_change":"2024-09-21T06:34:39.707909+0000","last_active":"2024-09-21T06:35:04.995193+0000","last_peered":"2024-09-21T06:35:04.995193+0000","last_clean":"2024-09-21T06:35:04.995193+0000","last_became_active":"2024-09-21T06:34:39.707492+0000","last_became_peered":"2024-09-21T06:34:39.707492+0000","last_unstale":"2024-09-21T06:35:04.995193+0000","last_undegraded":"2024-09-21T06:35:04.995193+0000","last_fullsized":"2024-09-21T06:35:04.995193+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:56.201212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.994774+0000","last_change":"2024-09-21T06:34:32.635870+0000","last_active":"2024-09-21T06:35:04.994774+0000","last_peered":"2024-09-21T06:35:04.994774+0000","last_clean":"2024-09-21T06:35:04.994774+0000","last_became_active":"2024-09-21T06:34:32.635252+0000","last_became_peered":"2024-09-21T06:34:32.635252+0000","last_unstale":"2024-09-21T06:35:04.994774+0000","last_undegraded":"2024-09-21T06:35:04.994774+0000","last_fullsized":"2024-09-21T06:35:04.994774+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:10.613033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029607+0000","last_change":"2024-09-21T06:34:30.634360+0000","last_active":"2024-09-21T06:35:05.029607+0000","last_peered":"2024-09-21T06:35:05.029607+0000","last_clean":"2024-09-21T06:35:05.029607+0000","last_became_active":"2024-09-21T06:34:30.633845+0000","last_became_peered":"2024-09-21T06:34:30.633845+0000","last_unstale":"2024-09-21T06:35:05.029607+0000","last_undegraded":"2024-09-21T06:35:05.029607+0000","last_fullsized":"2024-09-21T06:35:05.029607+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:34:53.388670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999643+0000","last_change":"2024-09-21T06:34:39.719620+0000","last_active":"2024-09-21T06:35:04.999643+0000","last_peered":"2024-09-21T06:35:04.999643+0000","last_clean":"2024-09-21T06:35:04.999643+0000","last_became_active":"2024-09-21T06:34:39.718786+0000","last_became_peered":"2024-09-21T06:34:39.718786+0000","last_unstale":"2024-09-21T06:35:04.999643+0000","last_undegraded":"2024-09-21T06:35:04.999643+0000","last_fullsized":"2024-09-21T06:35:04.999643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:43.524932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.273070+0000","last_change":"2024-09-21T06:34:32.634970+0000","last_active":"2024-09-21T06:35:05.273070+0000","last_peered":"2024-09-21T06:35:05.273070+0000","last_clean":"2024-09-21T06:35:05.273070+0000","last_became_active":"2024-09-21T06:34:32.634611+0000","last_became_peered":"2024-09-21T06:34:32.634611+0000","last_unstale":"2024-09-21T06:35:05.273070+0000","last_undegraded":"2024-09-21T06:35:05.273070+0000","last_fullsized":"2024-09-21T06:35:05.273070+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:37.855849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997918+0000","last_change":"2024-09-21T06:34:30.628123+0000","last_active":"2024-09-21T06:35:04.997918+0000","last_peered":"2024-09-21T06:35:04.997918+0000","last_clean":"2024-09-21T06:35:04.997918+0000","last_became_active":"2024-09-21T06:34:30.627498+0000","last_became_peered":"2024-09-21T06:34:30.627498+0000","last_unstale":"2024-09-21T06:35:04.997918+0000","last_undegraded":"2024-09-21T06:35:04.997918+0000","last_fullsized":"2024-09-21T06:35:04.997918+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:42.576441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":110,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029689+0000","last_change":"2024-09-21T06:33:16.856623+0000","last_active":"2024-09-21T06:35:05.029689+0000","last_peered":"2024-09-21T06:35:05.029689+0000","last_clean":"2024-09-21T06:35:05.029689+0000","last_became_active":"2024-09-21T06:33:16.850787+0000","last_became_peered":"2024-09-21T06:33:16.850787+0000","last_unstale":"2024-09-21T06:35:05.029689+0000","last_undegraded":"2024-09-21T06:35:05.029689+0000","last_fullsized":"2024-09-21T06:35:05.029689+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:50.283252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:50.283252+0000","last_clean_scrub_stamp":"2024-09-21T06:29:50.283252+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:29:20.711208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029992+0000","last_change":"2024-09-21T06:34:39.706472+0000","last_active":"2024-09-21T06:35:05.029992+0000","last_peered":"2024-09-21T06:35:05.029992+0000","last_clean":"2024-09-21T06:35:05.029992+0000","last_became_active":"2024-09-21T06:34:39.705724+0000","last_became_peered":"2024-09-21T06:34:39.705724+0000","last_unstale":"2024-09-21T06:35:05.029992+0000","last_undegraded":"2024-09-21T06:35:05.029992+0000","last_fullsized":"2024-09-21T06:35:05.029992+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:43.238704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.030085+0000","last_change":"2024-09-21T06:34:32.645944+0000","last_active":"2024-09-21T06:35:05.030085+0000","last_peered":"2024-09-21T06:35:05.030085+0000","last_clean":"2024-09-21T06:35:05.030085+0000","last_became_active":"2024-09-21T06:34:32.645134+0000","last_became_peered":"2024-09-21T06:34:32.645134+0000","last_unstale":"2024-09-21T06:35:05.030085+0000","last_undegraded":"2024-09-21T06:35:05.030085+0000","last_fullsized":"2024-09-21T06:35:05.030085+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:45.254164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.996320+0000","last_change":"2024-09-21T06:34:30.946296+0000","last_active":"2024-09-21T06:35:04.996320+0000","last_peered":"2024-09-21T06:35:04.996320+0000","last_clean":"2024-09-21T06:35:04.996320+0000","last_became_active":"2024-09-21T06:34:30.945777+0000","last_became_peered":"2024-09-21T06:34:30.945777+0000","last_unstale":"2024-09-21T06:35:04.996320+0000","last_undegraded":"2024-09-21T06:35:04.996320+0000","last_fullsized":"2024-09-21T06:35:04.996320+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:32:25.988512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995681+0000","last_change":"2024-09-21T06:34:39.705876+0000","last_active":"2024-09-21T06:35:04.995681+0000","last_peered":"2024-09-21T06:35:04.995681+0000","last_clean":"2024-09-21T06:35:04.995681+0000","last_became_active":"2024-09-21T06:34:39.705132+0000","last_became_peered":"2024-09-21T06:34:39.705132+0000","last_unstale":"2024-09-21T06:35:04.995681+0000","last_undegraded":"2024-09-21T06:35:04.995681+0000","last_fullsized":"2024-09-21T06:35:04.995681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:18.180244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029892+0000","last_change":"2024-09-21T06:34:32.638242+0000","last_active":"2024-09-21T06:35:05.029892+0000","last_peered":"2024-09-21T06:35:05.029892+0000","last_clean":"2024-09-21T06:35:05.029892+0000","last_became_active":"2024-09-21T06:34:32.637816+0000","last_became_peered":"2024-09-21T06:34:32.637816+0000","last_unstale":"2024-09-21T06:35:05.029892+0000","last_undegraded":"2024-09-21T06:35:05.029892+0000","last_fullsized":"2024-09-21T06:35:05.029892+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:40.926147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000042+0000","last_change":"2024-09-21T06:34:30.632173+0000","last_active":"2024-09-21T06:35:05.000042+0000","last_peered":"2024-09-21T06:35:05.000042+0000","last_clean":"2024-09-21T06:35:05.000042+0000","last_became_active":"2024-09-21T06:34:30.631638+0000","last_became_peered":"2024-09-21T06:34:30.631638+0000","last_unstale":"2024-09-21T06:35:05.000042+0000","last_undegraded":"2024-09-21T06:35:05.000042+0000","last_fullsized":"2024-09-21T06:35:05.000042+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:29.763272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000157+0000","last_change":"2024-09-21T06:34:39.714620+0000","last_active":"2024-09-21T06:35:05.000157+0000","last_peered":"2024-09-21T06:35:05.000157+0000","last_clean":"2024-09-21T06:35:05.000157+0000","last_became_active":"2024-09-21T06:34:39.714199+0000","last_became_peered":"2024-09-21T06:34:39.714199+0000","last_unstale":"2024-09-21T06:35:05.000157+0000","last_undegraded":"2024-09-21T06:35:05.000157+0000","last_fullsized":"2024-09-21T06:35:05.000157+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:37.041249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.030100+0000","last_change":"2024-09-21T06:34:32.645001+0000","last_active":"2024-09-21T06:35:05.030100+0000","last_peered":"2024-09-21T06:35:05.030100+0000","last_clean":"2024-09-21T06:35:05.030100+0000","last_became_active":"2024-09-21T06:34:32.644584+0000","last_became_peered":"2024-09-21T06:34:32.644584+0000","last_unstale":"2024-09-21T06:35:05.030100+0000","last_undegraded":"2024-09-21T06:35:05.030100+0000","last_fullsized":"2024-09-21T06:35:05.030100+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:47.098850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995420+0000","last_change":"2024-09-21T06:34:30.632593+0000","last_active":"2024-09-21T06:35:04.995420+0000","last_peered":"2024-09-21T06:35:04.995420+0000","last_clean":"2024-09-21T06:35:04.995420+0000","last_became_active":"2024-09-21T06:34:30.632170+0000","last_became_peered":"2024-09-21T06:34:30.632170+0000","last_unstale":"2024-09-21T06:35:04.995420+0000","last_undegraded":"2024-09-21T06:35:04.995420+0000","last_fullsized":"2024-09-21T06:35:04.995420+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:56.170655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.998214+0000","last_change":"2024-09-21T06:34:39.710172+0000","last_active":"2024-09-21T06:35:04.998214+0000","last_peered":"2024-09-21T06:35:04.998214+0000","last_clean":"2024-09-21T06:35:04.998214+0000","last_became_active":"2024-09-21T06:34:39.708876+0000","last_became_peered":"2024-09-21T06:34:39.708876+0000","last_unstale":"2024-09-21T06:35:04.998214+0000","last_undegraded":"2024-09-21T06:35:04.998214+0000","last_fullsized":"2024-09-21T06:35:04.998214+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:32.371873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000997+0000","last_change":"2024-09-21T06:34:32.651056+0000","last_active":"2024-09-21T06:35:05.000997+0000","last_peered":"2024-09-21T06:35:05.000997+0000","last_clean":"2024-09-21T06:35:05.000997+0000","last_became_active":"2024-09-21T06:34:32.650676+0000","last_became_peered":"2024-09-21T06:34:32.650676+0000","last_unstale":"2024-09-21T06:35:05.000997+0000","last_undegraded":"2024-09-21T06:35:05.000997+0000","last_fullsized":"2024-09-21T06:35:05.000997+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:34.599654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.004248+0000","last_change":"2024-09-21T06:34:30.632307+0000","last_active":"2024-09-21T06:35:05.004248+0000","last_peered":"2024-09-21T06:35:05.004248+0000","last_clean":"2024-09-21T06:35:05.004248+0000","last_became_active":"2024-09-21T06:34:30.631920+0000","last_became_peered":"2024-09-21T06:34:30.631920+0000","last_unstale":"2024-09-21T06:35:05.004248+0000","last_undegraded":"2024-09-21T06:35:05.004248+0000","last_fullsized":"2024-09-21T06:35:05.004248+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:46.940898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.998452+0000","last_change":"2024-09-21T06:34:39.715056+0000","last_active":"2024-09-21T06:35:04.998452+0000","last_peered":"2024-09-21T06:35:04.998452+0000","last_clean":"2024-09-21T06:35:04.998452+0000","last_became_active":"2024-09-21T06:34:39.714563+0000","last_became_peered":"2024-09-21T06:34:39.714563+0000","last_unstale":"2024-09-21T06:35:04.998452+0000","last_undegraded":"2024-09-21T06:35:04.998452+0000","last_fullsized":"2024-09-21T06:35:04.998452+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:48.567057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.998840+0000","last_change":"2024-09-21T06:34:32.646934+0000","last_active":"2024-09-21T06:35:04.998840+0000","last_peered":"2024-09-21T06:35:04.998840+0000","last_clean":"2024-09-21T06:35:04.998840+0000","last_became_active":"2024-09-21T06:34:32.646559+0000","last_became_peered":"2024-09-21T06:34:32.646559+0000","last_unstale":"2024-09-21T06:35:04.998840+0000","last_undegraded":"2024-09-21T06:35:04.998840+0000","last_fullsized":"2024-09-21T06:35:04.998840+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:37.802590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.004016+0000","last_change":"2024-09-21T06:34:30.946769+0000","last_active":"2024-09-21T06:35:05.004016+0000","last_peered":"2024-09-21T06:35:05.004016+0000","last_clean":"2024-09-21T06:35:05.004016+0000","last_became_active":"2024-09-21T06:34:30.946327+0000","last_became_peered":"2024-09-21T06:34:30.946327+0000","last_unstale":"2024-09-21T06:35:05.004016+0000","last_undegraded":"2024-09-21T06:35:05.004016+0000","last_fullsized":"2024-09-21T06:35:05.004016+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:52.055398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995322+0000","last_change":"2024-09-21T06:34:39.713398+0000","last_active":"2024-09-21T06:35:04.995322+0000","last_peered":"2024-09-21T06:35:04.995322+0000","last_clean":"2024-09-21T06:35:04.995322+0000","last_became_active":"2024-09-21T06:34:39.712954+0000","last_became_peered":"2024-09-21T06:34:39.712954+0000","last_unstale":"2024-09-21T06:35:04.995322+0000","last_undegraded":"2024-09-21T06:35:04.995322+0000","last_fullsized":"2024-09-21T06:35:04.995322+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:58.881014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029465+0000","last_change":"2024-09-21T06:34:32.640879+0000","last_active":"2024-09-21T06:35:05.029465+0000","last_peered":"2024-09-21T06:35:05.029465+0000","last_clean":"2024-09-21T06:35:05.029465+0000","last_became_active":"2024-09-21T06:34:32.640526+0000","last_became_peered":"2024-09-21T06:34:32.640526+0000","last_unstale":"2024-09-21T06:35:05.029465+0000","last_undegraded":"2024-09-21T06:35:05.029465+0000","last_fullsized":"2024-09-21T06:35:05.029465+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:12.327195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997068+0000","last_change":"2024-09-21T06:34:30.631449+0000","last_active":"2024-09-21T06:35:04.997068+0000","last_peered":"2024-09-21T06:35:04.997068+0000","last_clean":"2024-09-21T06:35:04.997068+0000","last_became_active":"2024-09-21T06:34:30.631081+0000","last_became_peered":"2024-09-21T06:34:30.631081+0000","last_unstale":"2024-09-21T06:35:04.997068+0000","last_undegraded":"2024-09-21T06:35:04.997068+0000","last_fullsized":"2024-09-21T06:35:04.997068+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:09.202696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.114180+0000","last_change":"2024-09-21T06:34:39.707379+0000","last_active":"2024-09-21T06:35:05.114180+0000","last_peered":"2024-09-21T06:35:05.114180+0000","last_clean":"2024-09-21T06:35:05.114180+0000","last_became_active":"2024-09-21T06:34:39.706979+0000","last_became_peered":"2024-09-21T06:34:39.706979+0000","last_unstale":"2024-09-21T06:35:05.114180+0000","last_undegraded":"2024-09-21T06:35:05.114180+0000","last_fullsized":"2024-09-21T06:35:05.114180+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:24.394233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029782+0000","last_change":"2024-09-21T06:34:32.649458+0000","last_active":"2024-09-21T06:35:05.029782+0000","last_peered":"2024-09-21T06:35:05.029782+0000","last_clean":"2024-09-21T06:35:05.029782+0000","last_became_active":"2024-09-21T06:34:32.649051+0000","last_became_peered":"2024-09-21T06:34:32.649051+0000","last_unstale":"2024-09-21T06:35:05.029782+0000","last_undegraded":"2024-09-21T06:35:05.029782+0000","last_fullsized":"2024-09-21T06:35:05.029782+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:12.631446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999624+0000","last_change":"2024-09-21T06:34:30.634592+0000","last_active":"2024-09-21T06:35:04.999624+0000","last_peered":"2024-09-21T06:35:04.999624+0000","last_clean":"2024-09-21T06:35:04.999624+0000","last_became_active":"2024-09-21T06:34:30.634251+0000","last_became_peered":"2024-09-21T06:34:30.634251+0000","last_unstale":"2024-09-21T06:35:04.999624+0000","last_undegraded":"2024-09-21T06:35:04.999624+0000","last_fullsized":"2024-09-21T06:35:04.999624+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:32.817467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000642+0000","last_change":"2024-09-21T06:34:39.719416+0000","last_active":"2024-09-21T06:35:05.000642+0000","last_peered":"2024-09-21T06:35:05.000642+0000","last_clean":"2024-09-21T06:35:05.000642+0000","last_became_active":"2024-09-21T06:34:39.718910+0000","last_became_peered":"2024-09-21T06:34:39.718910+0000","last_unstale":"2024-09-21T06:35:05.000642+0000","last_undegraded":"2024-09-21T06:35:05.000642+0000","last_fullsized":"2024-09-21T06:35:05.000642+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:41.413250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.994958+0000","last_change":"2024-09-21T06:34:32.640727+0000","last_active":"2024-09-21T06:35:04.994958+0000","last_peered":"2024-09-21T06:35:04.994958+0000","last_clean":"2024-09-21T06:35:04.994958+0000","last_became_active":"2024-09-21T06:34:32.640296+0000","last_became_peered":"2024-09-21T06:34:32.640296+0000","last_unstale":"2024-09-21T06:35:04.994958+0000","last_undegraded":"2024-09-21T06:35:04.994958+0000","last_fullsized":"2024-09-21T06:35:04.994958+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:35.124059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.996682+0000","last_change":"2024-09-21T06:34:30.633332+0000","last_active":"2024-09-21T06:35:04.996682+0000","last_peered":"2024-09-21T06:35:04.996682+0000","last_clean":"2024-09-21T06:35:04.996682+0000","last_became_active":"2024-09-21T06:34:30.632641+0000","last_became_peered":"2024-09-21T06:34:30.632641+0000","last_unstale":"2024-09-21T06:35:04.996682+0000","last_undegraded":"2024-09-21T06:35:04.996682+0000","last_fullsized":"2024-09-21T06:35:04.996682+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:30:23.930313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999766+0000","last_change":"2024-09-21T06:34:39.712504+0000","last_active":"2024-09-21T06:35:04.999766+0000","last_peered":"2024-09-21T06:35:04.999766+0000","last_clean":"2024-09-21T06:35:04.999766+0000","last_became_active":"2024-09-21T06:34:39.711638+0000","last_became_peered":"2024-09-21T06:34:39.711638+0000","last_unstale":"2024-09-21T06:35:04.999766+0000","last_undegraded":"2024-09-21T06:35:04.999766+0000","last_fullsized":"2024-09-21T06:35:04.999766+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:26:09.003367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.004007+0000","last_change":"2024-09-21T06:34:32.639582+0000","last_active":"2024-09-21T06:35:05.004007+0000","last_peered":"2024-09-21T06:35:05.004007+0000","last_clean":"2024-09-21T06:35:05.004007+0000","last_became_active":"2024-09-21T06:34:32.639023+0000","last_became_peered":"2024-09-21T06:34:32.639023+0000","last_unstale":"2024-09-21T06:35:05.004007+0000","last_undegraded":"2024-09-21T06:35:05.004007+0000","last_fullsized":"2024-09-21T06:35:05.004007+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:26.773389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"83'24","reported_seq":90,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:20.052467+0000","last_change":"2024-09-21T06:34:30.626280+0000","last_active":"2024-09-21T06:35:20.052467+0000","last_peered":"2024-09-21T06:35:20.052467+0000","last_clean":"2024-09-21T06:35:20.052467+0000","last_became_active":"2024-09-21T06:34:30.625662+0000","last_became_peered":"2024-09-21T06:34:30.625662+0000","last_unstale":"2024-09-21T06:35:20.052467+0000","last_undegraded":"2024-09-21T06:35:20.052467+0000","last_fullsized":"2024-09-21T06:35:20.052467+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:10.541881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48619,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":31,"num_read_kb":63,"num_write":24,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.113723+0000","last_change":"2024-09-21T06:34:39.708999+0000","last_active":"2024-09-21T06:35:05.113723+0000","last_peered":"2024-09-21T06:35:05.113723+0000","last_clean":"2024-09-21T06:35:05.113723+0000","last_became_active":"2024-09-21T06:34:39.708580+0000","last_became_peered":"2024-09-21T06:34:39.708580+0000","last_unstale":"2024-09-21T06:35:05.113723+0000","last_undegraded":"2024-09-21T06:35:05.113723+0000","last_fullsized":"2024-09-21T06:35:05.113723+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:07.648358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995798+0000","last_change":"2024-09-21T06:34:32.651157+0000","last_active":"2024-09-21T06:35:04.995798+0000","last_peered":"2024-09-21T06:35:04.995798+0000","last_clean":"2024-09-21T06:35:04.995798+0000","last_became_active":"2024-09-21T06:34:32.650804+0000","last_became_peered":"2024-09-21T06:34:32.650804+0000","last_unstale":"2024-09-21T06:35:04.995798+0000","last_undegraded":"2024-09-21T06:35:04.995798+0000","last_fullsized":"2024-09-21T06:35:04.995798+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:49.758496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999901+0000","last_change":"2024-09-21T06:34:30.947142+0000","last_active":"2024-09-21T06:35:04.999901+0000","last_peered":"2024-09-21T06:35:04.999901+0000","last_clean":"2024-09-21T06:35:04.999901+0000","last_became_active":"2024-09-21T06:34:30.946748+0000","last_became_peered":"2024-09-21T06:34:30.946748+0000","last_unstale":"2024-09-21T06:35:04.999901+0000","last_undegraded":"2024-09-21T06:35:04.999901+0000","last_fullsized":"2024-09-21T06:35:04.999901+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:39.222286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997754+0000","last_change":"2024-09-21T06:34:39.709266+0000","last_active":"2024-09-21T06:35:04.997754+0000","last_peered":"2024-09-21T06:35:04.997754+0000","last_clean":"2024-09-21T06:35:04.997754+0000","last_became_active":"2024-09-21T06:34:39.708753+0000","last_became_peered":"2024-09-21T06:34:39.708753+0000","last_unstale":"2024-09-21T06:35:04.997754+0000","last_undegraded":"2024-09-21T06:35:04.997754+0000","last_fullsized":"2024-09-21T06:35:04.997754+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:48.711681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000708+0000","last_change":"2024-09-21T06:34:32.637976+0000","last_active":"2024-09-21T06:35:05.000708+0000","last_peered":"2024-09-21T06:35:05.000708+0000","last_clean":"2024-09-21T06:35:05.000708+0000","last_became_active":"2024-09-21T06:34:32.637267+0000","last_became_peered":"2024-09-21T06:34:32.637267+0000","last_unstale":"2024-09-21T06:35:05.000708+0000","last_undegraded":"2024-09-21T06:35:05.000708+0000","last_fullsized":"2024-09-21T06:35:05.000708+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:31.340583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.030056+0000","last_change":"2024-09-21T06:34:30.631059+0000","last_active":"2024-09-21T06:35:05.030056+0000","last_peered":"2024-09-21T06:35:05.030056+0000","last_clean":"2024-09-21T06:35:05.030056+0000","last_became_active":"2024-09-21T06:34:30.630633+0000","last_became_peered":"2024-09-21T06:34:30.630633+0000","last_unstale":"2024-09-21T06:35:05.030056+0000","last_undegraded":"2024-09-21T06:35:05.030056+0000","last_fullsized":"2024-09-21T06:35:05.030056+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:09.049438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.113457+0000","last_change":"2024-09-21T06:34:39.713831+0000","last_active":"2024-09-21T06:35:05.113457+0000","last_peered":"2024-09-21T06:35:05.113457+0000","last_clean":"2024-09-21T06:35:05.113457+0000","last_became_active":"2024-09-21T06:34:39.713403+0000","last_became_peered":"2024-09-21T06:34:39.713403+0000","last_unstale":"2024-09-21T06:35:05.113457+0000","last_undegraded":"2024-09-21T06:35:05.113457+0000","last_fullsized":"2024-09-21T06:35:05.113457+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:22.201063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.996367+0000","last_change":"2024-09-21T06:34:32.646924+0000","last_active":"2024-09-21T06:35:04.996367+0000","last_peered":"2024-09-21T06:35:04.996367+0000","last_clean":"2024-09-21T06:35:04.996367+0000","last_became_active":"2024-09-21T06:34:32.646531+0000","last_became_peered":"2024-09-21T06:34:32.646531+0000","last_unstale":"2024-09-21T06:35:04.996367+0000","last_undegraded":"2024-09-21T06:35:04.996367+0000","last_fullsized":"2024-09-21T06:35:04.996367+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:48.550193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.001234+0000","last_change":"2024-09-21T06:34:30.631901+0000","last_active":"2024-09-21T06:35:05.001234+0000","last_peered":"2024-09-21T06:35:05.001234+0000","last_clean":"2024-09-21T06:35:05.001234+0000","last_became_active":"2024-09-21T06:34:30.631450+0000","last_became_peered":"2024-09-21T06:34:30.631450+0000","last_unstale":"2024-09-21T06:35:05.001234+0000","last_undegraded":"2024-09-21T06:35:05.001234+0000","last_fullsized":"2024-09-21T06:35:05.001234+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:20.709456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.272748+0000","last_change":"2024-09-21T06:34:39.711951+0000","last_active":"2024-09-21T06:35:05.272748+0000","last_peered":"2024-09-21T06:35:05.272748+0000","last_clean":"2024-09-21T06:35:05.272748+0000","last_became_active":"2024-09-21T06:34:39.710783+0000","last_became_peered":"2024-09-21T06:34:39.710783+0000","last_unstale":"2024-09-21T06:35:05.272748+0000","last_undegraded":"2024-09-21T06:35:05.272748+0000","last_fullsized":"2024-09-21T06:35:05.272748+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:27:58.501155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000355+0000","last_change":"2024-09-21T06:34:39.709639+0000","last_active":"2024-09-21T06:35:05.000355+0000","last_peered":"2024-09-21T06:35:05.000355+0000","last_clean":"2024-09-21T06:35:05.000355+0000","last_became_active":"2024-09-21T06:34:39.709241+0000","last_became_peered":"2024-09-21T06:34:39.709241+0000","last_unstale":"2024-09-21T06:35:05.000355+0000","last_undegraded":"2024-09-21T06:35:05.000355+0000","last_fullsized":"2024-09-21T06:35:05.000355+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.357399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.113674+0000","last_change":"2024-09-21T06:34:32.635365+0000","last_active":"2024-09-21T06:35:05.113674+0000","last_peered":"2024-09-21T06:35:05.113674+0000","last_clean":"2024-09-21T06:35:05.113674+0000","last_became_active":"2024-09-21T06:34:32.635012+0000","last_became_peered":"2024-09-21T06:34:32.635012+0000","last_unstale":"2024-09-21T06:35:05.113674+0000","last_undegraded":"2024-09-21T06:35:05.113674+0000","last_fullsized":"2024-09-21T06:35:05.113674+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:54.671074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.998620+0000","last_change":"2024-09-21T06:34:30.633347+0000","last_active":"2024-09-21T06:35:04.998620+0000","last_peered":"2024-09-21T06:35:04.998620+0000","last_clean":"2024-09-21T06:35:04.998620+0000","last_became_active":"2024-09-21T06:34:30.632513+0000","last_became_peered":"2024-09-21T06:34:30.632513+0000","last_unstale":"2024-09-21T06:35:04.998620+0000","last_undegraded":"2024-09-21T06:35:04.998620+0000","last_fullsized":"2024-09-21T06:35:04.998620+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:12.439109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.001150+0000","last_change":"2024-09-21T06:34:39.710204+0000","last_active":"2024-09-21T06:35:05.001150+0000","last_peered":"2024-09-21T06:35:05.001150+0000","last_clean":"2024-09-21T06:35:05.001150+0000","last_became_active":"2024-09-21T06:34:39.709744+0000","last_became_peered":"2024-09-21T06:34:39.709744+0000","last_unstale":"2024-09-21T06:35:05.001150+0000","last_undegraded":"2024-09-21T06:35:05.001150+0000","last_fullsized":"2024-09-21T06:35:05.001150+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:45.484088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.272833+0000","last_change":"2024-09-21T06:34:32.634170+0000","last_active":"2024-09-21T06:35:05.272833+0000","last_peered":"2024-09-21T06:35:05.272833+0000","last_clean":"2024-09-21T06:35:05.272833+0000","last_became_active":"2024-09-21T06:34:32.633831+0000","last_became_peered":"2024-09-21T06:34:32.633831+0000","last_unstale":"2024-09-21T06:35:05.272833+0000","last_undegraded":"2024-09-21T06:35:05.272833+0000","last_fullsized":"2024-09-21T06:35:05.272833+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:36.738279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.994732+0000","last_change":"2024-09-21T06:34:30.629422+0000","last_active":"2024-09-21T06:35:04.994732+0000","last_peered":"2024-09-21T06:35:04.994732+0000","last_clean":"2024-09-21T06:35:04.994732+0000","last_became_active":"2024-09-21T06:34:30.629025+0000","last_became_peered":"2024-09-21T06:34:30.629025+0000","last_unstale":"2024-09-21T06:35:04.994732+0000","last_undegraded":"2024-09-21T06:35:04.994732+0000","last_fullsized":"2024-09-21T06:35:04.994732+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:05.068816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000955+0000","last_change":"2024-09-21T06:34:30.634562+0000","last_active":"2024-09-21T06:35:05.000955+0000","last_peered":"2024-09-21T06:35:05.000955+0000","last_clean":"2024-09-21T06:35:05.000955+0000","last_became_active":"2024-09-21T06:34:30.634170+0000","last_became_peered":"2024-09-21T06:34:30.634170+0000","last_unstale":"2024-09-21T06:35:05.000955+0000","last_undegraded":"2024-09-21T06:35:05.000955+0000","last_fullsized":"2024-09-21T06:35:05.000955+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:15.243103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999225+0000","last_change":"2024-09-21T06:34:32.645718+0000","last_active":"2024-09-21T06:35:04.999225+0000","last_peered":"2024-09-21T06:35:04.999225+0000","last_clean":"2024-09-21T06:35:04.999225+0000","last_became_active":"2024-09-21T06:34:32.645072+0000","last_became_peered":"2024-09-21T06:34:32.645072+0000","last_unstale":"2024-09-21T06:35:04.999225+0000","last_undegraded":"2024-09-21T06:35:04.999225+0000","last_fullsized":"2024-09-21T06:35:04.999225+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:54.109246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.996127+0000","last_change":"2024-09-21T06:34:39.716947+0000","last_active":"2024-09-21T06:35:04.996127+0000","last_peered":"2024-09-21T06:35:04.996127+0000","last_clean":"2024-09-21T06:35:04.996127+0000","last_became_active":"2024-09-21T06:34:39.716540+0000","last_became_peered":"2024-09-21T06:34:39.716540+0000","last_unstale":"2024-09-21T06:35:04.996127+0000","last_undegraded":"2024-09-21T06:35:04.996127+0000","last_fullsized":"2024-09-21T06:35:04.996127+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:12.748444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999360+0000","last_change":"2024-09-21T06:34:30.636389+0000","last_active":"2024-09-21T06:35:04.999360+0000","last_peered":"2024-09-21T06:35:04.999360+0000","last_clean":"2024-09-21T06:35:04.999360+0000","last_became_active":"2024-09-21T06:34:30.636008+0000","last_became_peered":"2024-09-21T06:34:30.636008+0000","last_unstale":"2024-09-21T06:35:04.999360+0000","last_undegraded":"2024-09-21T06:35:04.999360+0000","last_fullsized":"2024-09-21T06:35:04.999360+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:45.476267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.003221+0000","last_change":"2024-09-21T06:34:32.635644+0000","last_active":"2024-09-21T06:35:05.003221+0000","last_peered":"2024-09-21T06:35:05.003221+0000","last_clean":"2024-09-21T06:35:05.003221+0000","last_became_active":"2024-09-21T06:34:32.635253+0000","last_became_peered":"2024-09-21T06:34:32.635253+0000","last_unstale":"2024-09-21T06:35:05.003221+0000","last_undegraded":"2024-09-21T06:35:05.003221+0000","last_fullsized":"2024-09-21T06:35:05.003221+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:34.713656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.996076+0000","last_change":"2024-09-21T06:34:39.710382+0000","last_active":"2024-09-21T06:35:04.996076+0000","last_peered":"2024-09-21T06:35:04.996076+0000","last_clean":"2024-09-21T06:35:04.996076+0000","last_became_active":"2024-09-21T06:34:39.709884+0000","last_became_peered":"2024-09-21T06:34:39.709884+0000","last_unstale":"2024-09-21T06:35:04.996076+0000","last_undegraded":"2024-09-21T06:35:04.996076+0000","last_fullsized":"2024-09-21T06:35:04.996076+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:01.602366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000463+0000","last_change":"2024-09-21T06:34:32.640025+0000","last_active":"2024-09-21T06:35:05.000463+0000","last_peered":"2024-09-21T06:35:05.000463+0000","last_clean":"2024-09-21T06:35:05.000463+0000","last_became_active":"2024-09-21T06:34:32.639648+0000","last_became_peered":"2024-09-21T06:34:32.639648+0000","last_unstale":"2024-09-21T06:35:05.000463+0000","last_undegraded":"2024-09-21T06:35:05.000463+0000","last_fullsized":"2024-09-21T06:35:05.000463+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:05.833065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.115450+0000","last_change":"2024-09-21T06:35:05.115344+0000","last_active":"2024-09-21T06:35:05.115450+0000","last_peered":"2024-09-21T06:35:05.115450+0000","last_clean":"2024-09-21T06:35:05.115450+0000","last_became_active":"2024-09-21T06:35:05.114678+0000","last_became_peered":"2024-09-21T06:35:05.114678+0000","last_unstale":"2024-09-21T06:35:05.115450+0000","last_undegraded":"2024-09-21T06:35:05.115450+0000","last_fullsized":"2024-09-21T06:35:05.115450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:24.593145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.115415+0000","last_change":"2024-09-21T06:35:05.115301+0000","last_active":"2024-09-21T06:35:05.115415+0000","last_peered":"2024-09-21T06:35:05.115415+0000","last_clean":"2024-09-21T06:35:05.115415+0000","last_became_active":"2024-09-21T06:35:05.114706+0000","last_became_peered":"2024-09-21T06:35:05.114706+0000","last_unstale":"2024-09-21T06:35:05.115415+0000","last_undegraded":"2024-09-21T06:35:05.115415+0000","last_fullsized":"2024-09-21T06:35:05.115415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:09.513594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000261+0000","last_change":"2024-09-21T06:34:32.634679+0000","last_active":"2024-09-21T06:35:05.000261+0000","last_peered":"2024-09-21T06:35:05.000261+0000","last_clean":"2024-09-21T06:35:05.000261+0000","last_became_active":"2024-09-21T06:34:32.634285+0000","last_became_peered":"2024-09-21T06:34:32.634285+0000","last_unstale":"2024-09-21T06:35:05.000261+0000","last_undegraded":"2024-09-21T06:35:05.000261+0000","last_fullsized":"2024-09-21T06:35:05.000261+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:15.185453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995303+0000","last_change":"2024-09-21T06:34:32.639037+0000","last_active":"2024-09-21T06:35:04.995303+0000","last_peered":"2024-09-21T06:35:04.995303+0000","last_clean":"2024-09-21T06:35:04.995303+0000","last_became_active":"2024-09-21T06:34:32.638689+0000","last_became_peered":"2024-09-21T06:34:32.638689+0000","last_unstale":"2024-09-21T06:35:04.995303+0000","last_undegraded":"2024-09-21T06:35:04.995303+0000","last_fullsized":"2024-09-21T06:35:04.995303+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:13.609191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.115157+0000","last_change":"2024-09-21T06:35:05.115051+0000","last_active":"2024-09-21T06:35:05.115157+0000","last_peered":"2024-09-21T06:35:05.115157+0000","last_clean":"2024-09-21T06:35:05.115157+0000","last_became_active":"2024-09-21T06:35:05.114602+0000","last_became_peered":"2024-09-21T06:35:05.114602+0000","last_unstale":"2024-09-21T06:35:05.115157+0000","last_undegraded":"2024-09-21T06:35:05.115157+0000","last_fullsized":"2024-09-21T06:35:05.115157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:48.778188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.996472+0000","last_change":"2024-09-21T06:34:32.637237+0000","last_active":"2024-09-21T06:35:04.996472+0000","last_peered":"2024-09-21T06:35:04.996472+0000","last_clean":"2024-09-21T06:35:04.996472+0000","last_became_active":"2024-09-21T06:34:32.634105+0000","last_became_peered":"2024-09-21T06:34:32.634105+0000","last_unstale":"2024-09-21T06:35:04.996472+0000","last_undegraded":"2024-09-21T06:35:04.996472+0000","last_fullsized":"2024-09-21T06:35:04.996472+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:52.530461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.273180+0000","last_change":"2024-09-21T06:34:32.624441+0000","last_active":"2024-09-21T06:35:05.273180+0000","last_peered":"2024-09-21T06:35:05.273180+0000","last_clean":"2024-09-21T06:35:05.273180+0000","last_became_active":"2024-09-21T06:34:32.623974+0000","last_became_peered":"2024-09-21T06:34:32.623974+0000","last_unstale":"2024-09-21T06:35:05.273180+0000","last_undegraded":"2024-09-21T06:35:05.273180+0000","last_fullsized":"2024-09-21T06:35:05.273180+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:12.378113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.998387+0000","last_change":"2024-09-21T06:34:32.633879+0000","last_active":"2024-09-21T06:35:04.998387+0000","last_peered":"2024-09-21T06:35:04.998387+0000","last_clean":"2024-09-21T06:35:04.998387+0000","last_became_active":"2024-09-21T06:34:32.631791+0000","last_became_peered":"2024-09-21T06:34:32.631791+0000","last_unstale":"2024-09-21T06:35:04.998387+0000","last_undegraded":"2024-09-21T06:35:04.998387+0000","last_fullsized":"2024-09-21T06:35:04.998387+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:46.246892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.273005+0000","last_change":"2024-09-21T06:34:32.639974+0000","last_active":"2024-09-21T06:35:05.273005+0000","last_peered":"2024-09-21T06:35:05.273005+0000","last_clean":"2024-09-21T06:35:05.273005+0000","last_became_active":"2024-09-21T06:34:32.639631+0000","last_became_peered":"2024-09-21T06:34:32.639631+0000","last_unstale":"2024-09-21T06:35:05.273005+0000","last_undegraded":"2024-09-21T06:35:05.273005+0000","last_fullsized":"2024-09-21T06:35:05.273005+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:50.069378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995887+0000","last_change":"2024-09-21T06:34:32.645403+0000","last_active":"2024-09-21T06:35:04.995887+0000","last_peered":"2024-09-21T06:35:04.995887+0000","last_clean":"2024-09-21T06:35:04.995887+0000","last_became_active":"2024-09-21T06:34:32.644986+0000","last_became_peered":"2024-09-21T06:34:32.644986+0000","last_unstale":"2024-09-21T06:35:04.995887+0000","last_undegraded":"2024-09-21T06:35:04.995887+0000","last_fullsized":"2024-09-21T06:35:04.995887+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:43.640107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995183+0000","last_change":"2024-09-21T06:34:32.639933+0000","last_active":"2024-09-21T06:35:04.995183+0000","last_peered":"2024-09-21T06:35:04.995183+0000","last_clean":"2024-09-21T06:35:04.995183+0000","last_became_active":"2024-09-21T06:34:32.639438+0000","last_became_peered":"2024-09-21T06:34:32.639438+0000","last_unstale":"2024-09-21T06:35:04.995183+0000","last_undegraded":"2024-09-21T06:35:04.995183+0000","last_fullsized":"2024-09-21T06:35:04.995183+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:18.860219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000816+0000","last_change":"2024-09-21T06:34:32.648580+0000","last_active":"2024-09-21T06:35:05.000816+0000","last_peered":"2024-09-21T06:35:05.000816+0000","last_clean":"2024-09-21T06:35:05.000816+0000","last_became_active":"2024-09-21T06:34:32.648016+0000","last_became_peered":"2024-09-21T06:34:32.648016+0000","last_unstale":"2024-09-21T06:35:05.000816+0000","last_undegraded":"2024-09-21T06:35:05.000816+0000","last_fullsized":"2024-09-21T06:35:05.000816+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:52.858069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.996429+0000","last_change":"2024-09-21T06:34:32.633376+0000","last_active":"2024-09-21T06:35:04.996429+0000","last_peered":"2024-09-21T06:35:04.996429+0000","last_clean":"2024-09-21T06:35:04.996429+0000","last_became_active":"2024-09-21T06:34:32.631052+0000","last_became_peered":"2024-09-21T06:34:32.631052+0000","last_unstale":"2024-09-21T06:35:04.996429+0000","last_undegraded":"2024-09-21T06:35:04.996429+0000","last_fullsized":"2024-09-21T06:35:04.996429+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:46.229599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.114206+0000","last_change":"2024-09-21T06:34:32.636966+0000","last_active":"2024-09-21T06:35:05.114206+0000","last_peered":"2024-09-21T06:35:05.114206+0000","last_clean":"2024-09-21T06:35:05.114206+0000","last_became_active":"2024-09-21T06:34:32.636565+0000","last_became_peered":"2024-09-21T06:34:32.636565+0000","last_unstale":"2024-09-21T06:35:05.114206+0000","last_undegraded":"2024-09-21T06:35:05.114206+0000","last_fullsized":"2024-09-21T06:35:05.114206+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:36.233082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.001311+0000","last_change":"2024-09-21T06:34:32.646895+0000","last_active":"2024-09-21T06:35:05.001311+0000","last_peered":"2024-09-21T06:35:05.001311+0000","last_clean":"2024-09-21T06:35:05.001311+0000","last_became_active":"2024-09-21T06:34:32.646505+0000","last_became_peered":"2024-09-21T06:34:32.646505+0000","last_unstale":"2024-09-21T06:35:05.001311+0000","last_undegraded":"2024-09-21T06:35:05.001311+0000","last_fullsized":"2024-09-21T06:35:05.001311+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:43.441337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995353+0000","last_change":"2024-09-21T06:34:32.645402+0000","last_active":"2024-09-21T06:35:04.995353+0000","last_peered":"2024-09-21T06:35:04.995353+0000","last_clean":"2024-09-21T06:35:04.995353+0000","last_became_active":"2024-09-21T06:34:32.645013+0000","last_became_peered":"2024-09-21T06:34:32.645013+0000","last_unstale":"2024-09-21T06:35:04.995353+0000","last_undegraded":"2024-09-21T06:35:04.995353+0000","last_fullsized":"2024-09-21T06:35:04.995353+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:33.475847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999814+0000","last_change":"2024-09-21T06:34:32.643234+0000","last_active":"2024-09-21T06:35:04.999814+0000","last_peered":"2024-09-21T06:35:04.999814+0000","last_clean":"2024-09-21T06:35:04.999814+0000","last_became_active":"2024-09-21T06:34:32.642614+0000","last_became_peered":"2024-09-21T06:34:32.642614+0000","last_unstale":"2024-09-21T06:35:04.999814+0000","last_undegraded":"2024-09-21T06:35:04.999814+0000","last_fullsized":"2024-09-21T06:35:04.999814+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:50.903500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999942+0000","last_change":"2024-09-21T06:34:32.641887+0000","last_active":"2024-09-21T06:35:04.999942+0000","last_peered":"2024-09-21T06:35:04.999942+0000","last_clean":"2024-09-21T06:35:04.999942+0000","last_became_active":"2024-09-21T06:34:32.641490+0000","last_became_peered":"2024-09-21T06:34:32.641490+0000","last_unstale":"2024-09-21T06:35:04.999942+0000","last_undegraded":"2024-09-21T06:35:04.999942+0000","last_fullsized":"2024-09-21T06:35:04.999942+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:34.550970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995289+0000","last_change":"2024-09-21T06:34:32.641865+0000","last_active":"2024-09-21T06:35:04.995289+0000","last_peered":"2024-09-21T06:35:04.995289+0000","last_clean":"2024-09-21T06:35:04.995289+0000","last_became_active":"2024-09-21T06:34:32.641252+0000","last_became_peered":"2024-09-21T06:34:32.641252+0000","last_unstale":"2024-09-21T06:35:04.995289+0000","last_undegraded":"2024-09-21T06:35:04.995289+0000","last_fullsized":"2024-09-21T06:35:04.995289+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:44.459202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997920+0000","last_change":"2024-09-21T06:34:32.641259+0000","last_active":"2024-09-21T06:35:04.997920+0000","last_peered":"2024-09-21T06:35:04.997920+0000","last_clean":"2024-09-21T06:35:04.997920+0000","last_became_active":"2024-09-21T06:34:32.640907+0000","last_became_peered":"2024-09-21T06:34:32.640907+0000","last_unstale":"2024-09-21T06:35:04.997920+0000","last_undegraded":"2024-09-21T06:35:04.997920+0000","last_fullsized":"2024-09-21T06:35:04.997920+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:59.966857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.030054+0000","last_change":"2024-09-21T06:34:32.646095+0000","last_active":"2024-09-21T06:35:05.030054+0000","last_peered":"2024-09-21T06:35:05.030054+0000","last_clean":"2024-09-21T06:35:05.030054+0000","last_became_active":"2024-09-21T06:34:32.645604+0000","last_became_peered":"2024-09-21T06:34:32.645604+0000","last_unstale":"2024-09-21T06:35:05.030054+0000","last_undegraded":"2024-09-21T06:35:05.030054+0000","last_fullsized":"2024-09-21T06:35:05.030054+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:30.872428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.001219+0000","last_change":"2024-09-21T06:34:32.649303+0000","last_active":"2024-09-21T06:35:05.001219+0000","last_peered":"2024-09-21T06:35:05.001219+0000","last_clean":"2024-09-21T06:35:05.001219+0000","last_became_active":"2024-09-21T06:34:32.648965+0000","last_became_peered":"2024-09-21T06:34:32.648965+0000","last_unstale":"2024-09-21T06:35:05.001219+0000","last_undegraded":"2024-09-21T06:35:05.001219+0000","last_fullsized":"2024-09-21T06:35:05.001219+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:51.335949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.996335+0000","last_change":"2024-09-21T06:34:32.637261+0000","last_active":"2024-09-21T06:35:04.996335+0000","last_peered":"2024-09-21T06:35:04.996335+0000","last_clean":"2024-09-21T06:35:04.996335+0000","last_became_active":"2024-09-21T06:34:32.636861+0000","last_became_peered":"2024-09-21T06:34:32.636861+0000","last_unstale":"2024-09-21T06:35:04.996335+0000","last_undegraded":"2024-09-21T06:35:04.996335+0000","last_fullsized":"2024-09-21T06:35:04.996335+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:55.271027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.272766+0000","last_change":"2024-09-21T06:34:32.635156+0000","last_active":"2024-09-21T06:35:05.272766+0000","last_peered":"2024-09-21T06:35:05.272766+0000","last_clean":"2024-09-21T06:35:05.272766+0000","last_became_active":"2024-09-21T06:34:32.634802+0000","last_became_peered":"2024-09-21T06:34:32.634802+0000","last_unstale":"2024-09-21T06:35:05.272766+0000","last_undegraded":"2024-09-21T06:35:05.272766+0000","last_fullsized":"2024-09-21T06:35:05.272766+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:16.139829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029506+0000","last_change":"2024-09-21T06:34:32.637509+0000","last_active":"2024-09-21T06:35:05.029506+0000","last_peered":"2024-09-21T06:35:05.029506+0000","last_clean":"2024-09-21T06:35:05.029506+0000","last_became_active":"2024-09-21T06:34:32.637136+0000","last_became_peered":"2024-09-21T06:34:32.637136+0000","last_unstale":"2024-09-21T06:35:05.029506+0000","last_undegraded":"2024-09-21T06:35:05.029506+0000","last_fullsized":"2024-09-21T06:35:05.029506+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:48.088988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.114942+0000","last_change":"2024-09-21T06:35:05.114819+0000","last_active":"2024-09-21T06:35:05.114942+0000","last_peered":"2024-09-21T06:35:05.114942+0000","last_clean":"2024-09-21T06:35:05.114942+0000","last_became_active":"2024-09-21T06:35:05.114424+0000","last_became_peered":"2024-09-21T06:35:05.114424+0000","last_unstale":"2024-09-21T06:35:05.114942+0000","last_undegraded":"2024-09-21T06:35:05.114942+0000","last_fullsized":"2024-09-21T06:35:05.114942+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:59.323643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.003387+0000","last_change":"2024-09-21T06:34:32.636103+0000","last_active":"2024-09-21T06:35:05.003387+0000","last_peered":"2024-09-21T06:35:05.003387+0000","last_clean":"2024-09-21T06:35:05.003387+0000","last_became_active":"2024-09-21T06:34:32.635728+0000","last_became_peered":"2024-09-21T06:34:32.635728+0000","last_unstale":"2024-09-21T06:35:05.003387+0000","last_undegraded":"2024-09-21T06:35:05.003387+0000","last_fullsized":"2024-09-21T06:35:05.003387+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:16.731402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997848+0000","last_change":"2024-09-21T06:34:32.635654+0000","last_active":"2024-09-21T06:35:04.997848+0000","last_peered":"2024-09-21T06:35:04.997848+0000","last_clean":"2024-09-21T06:35:04.997848+0000","last_became_active":"2024-09-21T06:34:32.635265+0000","last_became_peered":"2024-09-21T06:34:32.635265+0000","last_unstale":"2024-09-21T06:35:04.997848+0000","last_undegraded":"2024-09-21T06:35:04.997848+0000","last_fullsized":"2024-09-21T06:35:04.997848+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:28.169625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000238+0000","last_change":"2024-09-21T06:34:32.644640+0000","last_active":"2024-09-21T06:35:05.000238+0000","last_peered":"2024-09-21T06:35:05.000238+0000","last_clean":"2024-09-21T06:35:05.000238+0000","last_became_active":"2024-09-21T06:34:32.644334+0000","last_became_peered":"2024-09-21T06:34:32.644334+0000","last_unstale":"2024-09-21T06:35:05.000238+0000","last_undegraded":"2024-09-21T06:35:05.000238+0000","last_fullsized":"2024-09-21T06:35:05.000238+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:25.572178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000396+0000","last_change":"2024-09-21T06:34:32.643705+0000","last_active":"2024-09-21T06:35:05.000396+0000","last_peered":"2024-09-21T06:35:05.000396+0000","last_clean":"2024-09-21T06:35:05.000396+0000","last_became_active":"2024-09-21T06:34:32.642705+0000","last_became_peered":"2024-09-21T06:34:32.642705+0000","last_unstale":"2024-09-21T06:35:05.000396+0000","last_undegraded":"2024-09-21T06:35:05.000396+0000","last_fullsized":"2024-09-21T06:35:05.000396+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:22.542559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.001285+0000","last_change":"2024-09-21T06:34:32.646539+0000","last_active":"2024-09-21T06:35:05.001285+0000","last_peered":"2024-09-21T06:35:05.001285+0000","last_clean":"2024-09-21T06:35:05.001285+0000","last_became_active":"2024-09-21T06:34:32.645992+0000","last_became_peered":"2024-09-21T06:34:32.645992+0000","last_unstale":"2024-09-21T06:35:05.001285+0000","last_undegraded":"2024-09-21T06:35:05.001285+0000","last_fullsized":"2024-09-21T06:35:05.001285+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:38.856001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":49417,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":37,"num_read_kb":69,"num_write":71,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":233472,"data_stored":148251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":61,"ondisk_log_size":61,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282016,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":466743,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939675,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1019904,"data_stored":1101828,"data_compressed":59457,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103201,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":471421,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299430,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":467347,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495659,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":1061789,"data_compressed":59457,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659185,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":901120,"data_stored":462647,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855414,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":1057293,"data_compressed":59457,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018939,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":466765,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215167,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":462751,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":511300,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":511266,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":905216,"data_stored":462737,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":48619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":48653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":48813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:35:26.435 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T06:35:26.436 INFO:tasks.cephadm:Setting up client nodes... 2024-09-21T06:35:26.436 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-21T06:35:26.705 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:27.118 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:27 smithi067 ceph-mon[31232]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:35:27.118 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:27 smithi067 ceph-mon[31232]: pgmap v324: 129 pgs: 129 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.9 KiB/s wr, 5 op/s 2024-09-21T06:35:27.118 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:27 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:35:27.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.218+0000 7f81fbd56640 1 -- 172.21.15.67:0/2875631472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f4104080 msgr2=0x7f81f410aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:27.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.218+0000 7f81fbd56640 1 --2- 172.21.15.67:0/2875631472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f4104080 0x7f81f410aa20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f81ec009ca0 tx=0x7f81ec02a950 comp rx=0 tx=0).stop 2024-09-21T06:35:27.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.218+0000 7f81fbd56640 1 -- 172.21.15.67:0/2875631472 shutdown_connections 2024-09-21T06:35:27.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.218+0000 7f81fbd56640 1 --2- 172.21.15.67:0/2875631472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f4104080 0x7f81f410aa20 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:27.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.218+0000 7f81fbd56640 1 --2- 172.21.15.67:0/2875631472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81f41036e0 0x7f81f4103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:27.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.218+0000 7f81fbd56640 1 --2- 172.21.15.67:0/2875631472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81f4102650 0x7f81f4102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:27.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.218+0000 7f81fbd56640 1 -- 172.21.15.67:0/2875631472 >> 172.21.15.67:0/2875631472 conn(0x7f81f40fdfa0 msgr2=0x7f81f4100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:27.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.218+0000 7f81fbd56640 1 -- 172.21.15.67:0/2875631472 shutdown_connections 2024-09-21T06:35:27.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.218+0000 7f81fbd56640 1 -- 172.21.15.67:0/2875631472 wait complete. 2024-09-21T06:35:27.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.219+0000 7f81fbd56640 1 Processor -- start 2024-09-21T06:35:27.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.219+0000 7f81fbd56640 1 -- start start 2024-09-21T06:35:27.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.219+0000 7f81fbd56640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81f4102650 0x7f81f4199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:27.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.219+0000 7f81fbd56640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81f41036e0 0x7f81f419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:27.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.219+0000 7f81fbd56640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f4104080 0x7f81f419b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:27.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.219+0000 7f81fbd56640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f81f410e180 con 0x7f81f4104080 2024-09-21T06:35:27.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.219+0000 7f81fbd56640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f81f410e000 con 0x7f81f4102650 2024-09-21T06:35:27.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.219+0000 7f81fbd56640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f81f410e300 con 0x7f81f41036e0 2024-09-21T06:35:27.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.219+0000 7f81fa2f3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f4104080 0x7f81f419b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:27.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.219+0000 7f81fa2f3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f4104080 0x7f81f419b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54880/0 (socket says 172.21.15.67:54880) 2024-09-21T06:35:27.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.219+0000 7f81fa2f3640 1 -- 172.21.15.67:0/3442586215 learned_addr learned my addr 172.21.15.67:0/3442586215 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:27.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.220+0000 7f81f9af2640 1 --2- 172.21.15.67:0/3442586215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81f4102650 0x7f81f4199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:27.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.220+0000 7f81f92f1640 1 --2- 172.21.15.67:0/3442586215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81f41036e0 0x7f81f419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:27.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.220+0000 7f81fa2f3640 1 -- 172.21.15.67:0/3442586215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81f41036e0 msgr2=0x7f81f419a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:27.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.220+0000 7f81fa2f3640 1 --2- 172.21.15.67:0/3442586215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81f41036e0 0x7f81f419a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:27.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.220+0000 7f81fa2f3640 1 -- 172.21.15.67:0/3442586215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81f4102650 msgr2=0x7f81f4199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:27.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.220+0000 7f81fa2f3640 1 --2- 172.21.15.67:0/3442586215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81f4102650 0x7f81f4199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:27.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.220+0000 7f81fa2f3640 1 -- 172.21.15.67:0/3442586215 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f419b890 con 0x7f81f4104080 2024-09-21T06:35:27.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.220+0000 7f81f92f1640 1 --2- 172.21.15.67:0/3442586215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81f41036e0 0x7f81f419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:27.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.220+0000 7f81fa2f3640 1 --2- 172.21.15.67:0/3442586215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f4104080 0x7f81f419b070 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f81ec02af40 tx=0x7f81ec003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:27.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.220+0000 7f81eaffd640 1 -- 172.21.15.67:0/3442586215 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec034040 con 0x7f81f4104080 2024-09-21T06:35:27.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.220+0000 7f81eaffd640 1 -- 172.21.15.67:0/3442586215 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f81ec030850 con 0x7f81f4104080 2024-09-21T06:35:27.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.220+0000 7f81eaffd640 1 -- 172.21.15.67:0/3442586215 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec004760 con 0x7f81f4104080 2024-09-21T06:35:27.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.220+0000 7f81fbd56640 1 -- 172.21.15.67:0/3442586215 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81f419bb20 con 0x7f81f4104080 2024-09-21T06:35:27.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.220+0000 7f81fbd56640 1 -- 172.21.15.67:0/3442586215 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81f41a20b0 con 0x7f81f4104080 2024-09-21T06:35:27.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.221+0000 7f81fbd56640 1 -- 172.21.15.67:0/3442586215 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81f41a22e0 con 0x7f81f4104080 2024-09-21T06:35:27.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.223+0000 7f81eaffd640 1 -- 172.21.15.67:0/3442586215 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f81ec037020 con 0x7f81f4104080 2024-09-21T06:35:27.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.224+0000 7f81eaffd640 1 --2- 172.21.15.67:0/3442586215 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f81e00b65a0 0x7f81e00b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:27.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.224+0000 7f81f9af2640 1 --2- 172.21.15.67:0/3442586215 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f81e00b65a0 0x7f81e00b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:27.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.224+0000 7f81eaffd640 1 -- 172.21.15.67:0/3442586215 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f81ec1088e0 con 0x7f81f4104080 2024-09-21T06:35:27.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.224+0000 7f81f9af2640 1 --2- 172.21.15.67:0/3442586215 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f81e00b65a0 0x7f81e00b8ab0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f81f000bbe0 tx=0x7f81f000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:27.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.227+0000 7f81eaffd640 1 -- 172.21.15.67:0/3442586215 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f81ec083710 con 0x7f81f4104080 2024-09-21T06:35:27.354 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:27 smithi203 ceph-mon[33735]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:35:27.354 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:27 smithi203 ceph-mon[33735]: pgmap v324: 129 pgs: 129 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.9 KiB/s wr, 5 op/s 2024-09-21T06:35:27.354 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:27 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:35:27.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.493+0000 7f81fbd56640 1 -- 172.21.15.67:0/3442586215 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f81f41a24b0 con 0x7f81f4104080 2024-09-21T06:35:27.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.496+0000 7f81eaffd640 1 -- 172.21.15.67:0/3442586215 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f81ec0cecb0 con 0x7f81f4104080 2024-09-21T06:35:27.498 INFO:teuthology.orchestra.run.smithi067.stdout:[client.0] 2024-09-21T06:35:27.498 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQAvae5mkciDHRAA+FoPgt8nmo2hGAjVI//ZVA== 2024-09-21T06:35:27.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.498+0000 7f81fbd56640 1 -- 172.21.15.67:0/3442586215 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f81e00b65a0 msgr2=0x7f81e00b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.498+0000 7f81fbd56640 1 --2- 172.21.15.67:0/3442586215 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f81e00b65a0 0x7f81e00b8ab0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f81f000bbe0 tx=0x7f81f000b390 comp rx=0 tx=0).stop 2024-09-21T06:35:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.498+0000 7f81fbd56640 1 -- 172.21.15.67:0/3442586215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f4104080 msgr2=0x7f81f419b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.498+0000 7f81fbd56640 1 --2- 172.21.15.67:0/3442586215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f4104080 0x7f81f419b070 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f81ec02af40 tx=0x7f81ec003f10 comp rx=0 tx=0).stop 2024-09-21T06:35:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.498+0000 7f81fbd56640 1 -- 172.21.15.67:0/3442586215 shutdown_connections 2024-09-21T06:35:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.498+0000 7f81fbd56640 1 --2- 172.21.15.67:0/3442586215 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f81e00b65a0 0x7f81e00b8ab0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.498+0000 7f81fbd56640 1 --2- 172.21.15.67:0/3442586215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f4104080 0x7f81f419b070 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.498+0000 7f81fbd56640 1 --2- 172.21.15.67:0/3442586215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81f41036e0 0x7f81f419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.498+0000 7f81fbd56640 1 --2- 172.21.15.67:0/3442586215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81f4102650 0x7f81f4199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.498+0000 7f81fbd56640 1 -- 172.21.15.67:0/3442586215 >> 172.21.15.67:0/3442586215 conn(0x7f81f40fdfa0 msgr2=0x7f81f40ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.498+0000 7f81fbd56640 1 -- 172.21.15.67:0/3442586215 shutdown_connections 2024-09-21T06:35:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:27.498+0000 7f81fbd56640 1 -- 172.21.15.67:0/3442586215 wait complete. 2024-09-21T06:35:27.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:27 smithi100 ceph-mon[33401]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:35:27.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:27 smithi100 ceph-mon[33401]: pgmap v324: 129 pgs: 129 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.9 KiB/s wr, 5 op/s 2024-09-21T06:35:27.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:27 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:35:28.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:28 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3442586215' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T06:35:28.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:28 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3442586215' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T06:35:28.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:28 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:35:28.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:28 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:35:28.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:28 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:35:28.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:28 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:35:28.412 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-21T06:35:28.412 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-21T06:35:28.412 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T06:35:28.450 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph config log 1 --format=json 2024-09-21T06:35:28.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:28 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3442586215' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T06:35:28.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:28 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3442586215' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T06:35:28.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:28 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:35:28.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:28 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:35:28.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:28 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:35:28.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:28 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:35:28.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:28 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3442586215' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T06:35:28.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:28 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3442586215' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T06:35:28.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:28 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:35:28.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:28 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:35:28.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:28 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:35:28.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:28 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:35:28.762 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:29.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.265+0000 7fd88d171640 1 -- 172.21.15.67:0/3243971338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd888108420 msgr2=0x7fd888108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:29.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.265+0000 7fd88d171640 1 --2- 172.21.15.67:0/3243971338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd888108420 0x7fd888108830 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd87c009900 tx=0x7fd87c02a410 comp rx=0 tx=0).stop 2024-09-21T06:35:29.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.265+0000 7fd88d171640 1 -- 172.21.15.67:0/3243971338 shutdown_connections 2024-09-21T06:35:29.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.265+0000 7fd88d171640 1 --2- 172.21.15.67:0/3243971338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd888108d70 0x7fd88810f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.265+0000 7fd88d171640 1 --2- 172.21.15.67:0/3243971338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd888108420 0x7fd888108830 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.265+0000 7fd88d171640 1 --2- 172.21.15.67:0/3243971338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd888102ae0 0x7fd888102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.265+0000 7fd88d171640 1 -- 172.21.15.67:0/3243971338 >> 172.21.15.67:0/3243971338 conn(0x7fd888079770 msgr2=0x7fd888079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:29.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.265+0000 7fd88d171640 1 -- 172.21.15.67:0/3243971338 shutdown_connections 2024-09-21T06:35:29.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.265+0000 7fd88d171640 1 -- 172.21.15.67:0/3243971338 wait complete. 2024-09-21T06:35:29.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.266+0000 7fd88d171640 1 Processor -- start 2024-09-21T06:35:29.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.266+0000 7fd88d171640 1 -- start start 2024-09-21T06:35:29.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.266+0000 7fd88d171640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd888102ae0 0x7fd88819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:29.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.266+0000 7fd88d171640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd888108420 0x7fd88819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:29.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.266+0000 7fd88d171640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd888108d70 0x7fd88819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:29.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.266+0000 7fd88d171640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881128e0 con 0x7fd888108420 2024-09-21T06:35:29.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.266+0000 7fd88d171640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd888112760 con 0x7fd888108d70 2024-09-21T06:35:29.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.266+0000 7fd88d171640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd888112a60 con 0x7fd888102ae0 2024-09-21T06:35:29.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.266+0000 7fd88659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd888108420 0x7fd88819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:29.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.266+0000 7fd88659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd888108420 0x7fd88819aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49776/0 (socket says 172.21.15.67:49776) 2024-09-21T06:35:29.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.267+0000 7fd88659c640 1 -- 172.21.15.67:0/2042810755 learned_addr learned my addr 172.21.15.67:0/2042810755 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:29.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.267+0000 7fd886d9d640 1 --2- 172.21.15.67:0/2042810755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd888102ae0 0x7fd88819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:29.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.267+0000 7fd88759e640 1 --2- 172.21.15.67:0/2042810755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd888108d70 0x7fd88819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:29.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.267+0000 7fd88659c640 1 -- 172.21.15.67:0/2042810755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd888102ae0 msgr2=0x7fd88819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:29.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.267+0000 7fd88659c640 1 --2- 172.21.15.67:0/2042810755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd888102ae0 0x7fd88819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.267+0000 7fd88659c640 1 -- 172.21.15.67:0/2042810755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd888108d70 msgr2=0x7fd88819fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:29.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.267+0000 7fd88659c640 1 --2- 172.21.15.67:0/2042810755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd888108d70 0x7fd88819fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.267+0000 7fd88659c640 1 -- 172.21.15.67:0/2042810755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8881a03d0 con 0x7fd888108420 2024-09-21T06:35:29.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.267+0000 7fd886d9d640 1 --2- 172.21.15.67:0/2042810755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd888102ae0 0x7fd88819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:29.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.267+0000 7fd88659c640 1 --2- 172.21.15.67:0/2042810755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd888108420 0x7fd88819aa00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd87c004770 tx=0x7fd87c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:29.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.267+0000 7fd86ffff640 1 -- 172.21.15.67:0/2042810755 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87c03f030 con 0x7fd888108420 2024-09-21T06:35:29.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.267+0000 7fd88d171640 1 -- 172.21.15.67:0/2042810755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8881a0660 con 0x7fd888108420 2024-09-21T06:35:29.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.267+0000 7fd88d171640 1 -- 172.21.15.67:0/2042810755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8881a6e20 con 0x7fd888108420 2024-09-21T06:35:29.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.267+0000 7fd86ffff640 1 -- 172.21.15.67:0/2042810755 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd87c004440 con 0x7fd888108420 2024-09-21T06:35:29.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.268+0000 7fd86ffff640 1 -- 172.21.15.67:0/2042810755 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87c049060 con 0x7fd888108420 2024-09-21T06:35:29.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.269+0000 7fd88d171640 1 -- 172.21.15.67:0/2042810755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8740052c0 con 0x7fd888108420 2024-09-21T06:35:29.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.270+0000 7fd86ffff640 1 -- 172.21.15.67:0/2042810755 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fd87c042030 con 0x7fd888108420 2024-09-21T06:35:29.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.271+0000 7fd86ffff640 1 --2- 172.21.15.67:0/2042810755 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd8700b65a0 0x7fd8700b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:29.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.271+0000 7fd886d9d640 1 --2- 172.21.15.67:0/2042810755 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd8700b65a0 0x7fd8700b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:29.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.271+0000 7fd86ffff640 1 -- 172.21.15.67:0/2042810755 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7fd87c107d00 con 0x7fd888108420 2024-09-21T06:35:29.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.272+0000 7fd886d9d640 1 --2- 172.21.15.67:0/2042810755 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd8700b65a0 0x7fd8700b8ab0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd878009c20 tx=0x7fd878005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:29.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.274+0000 7fd86ffff640 1 -- 172.21.15.67:0/2042810755 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd87c0ce0d0 con 0x7fd888108420 2024-09-21T06:35:29.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.524+0000 7fd88d171640 1 -- 172.21.15.67:0/2042810755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fd874005880 con 0x7fd888108420 2024-09-21T06:35:29.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.525+0000 7fd86ffff640 1 -- 172.21.15.67:0/2042810755 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v52) ==== 86+0+133 (secure 0 0 0) 0x7fd87c0d24b0 con 0x7fd888108420 2024-09-21T06:35:29.527 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:35:29.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.527+0000 7fd88d171640 1 -- 172.21.15.67:0/2042810755 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd8700b65a0 msgr2=0x7fd8700b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:29.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.527+0000 7fd88d171640 1 --2- 172.21.15.67:0/2042810755 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd8700b65a0 0x7fd8700b8ab0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd878009c20 tx=0x7fd878005f90 comp rx=0 tx=0).stop 2024-09-21T06:35:29.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.527+0000 7fd88d171640 1 -- 172.21.15.67:0/2042810755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd888108420 msgr2=0x7fd88819aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:29.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.527+0000 7fd88d171640 1 --2- 172.21.15.67:0/2042810755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd888108420 0x7fd88819aa00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd87c004770 tx=0x7fd87c004040 comp rx=0 tx=0).stop 2024-09-21T06:35:29.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.527+0000 7fd88d171640 1 -- 172.21.15.67:0/2042810755 shutdown_connections 2024-09-21T06:35:29.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.527+0000 7fd88d171640 1 --2- 172.21.15.67:0/2042810755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd888108d70 0x7fd88819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.527+0000 7fd88d171640 1 --2- 172.21.15.67:0/2042810755 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd8700b65a0 0x7fd8700b8ab0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.527+0000 7fd88d171640 1 --2- 172.21.15.67:0/2042810755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd888108420 0x7fd88819aa00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.527+0000 7fd88d171640 1 --2- 172.21.15.67:0/2042810755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd888102ae0 0x7fd88819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:29.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.527+0000 7fd88d171640 1 -- 172.21.15.67:0/2042810755 >> 172.21.15.67:0/2042810755 conn(0x7fd888079770 msgr2=0x7fd888101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:29.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.527+0000 7fd88d171640 1 -- 172.21.15.67:0/2042810755 shutdown_connections 2024-09-21T06:35:29.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:29.527+0000 7fd88d171640 1 -- 172.21.15.67:0/2042810755 wait complete. 2024-09-21T06:35:29.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:29 smithi067 ceph-mon[31232]: pgmap v325: 129 pgs: 129 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 3.8 KiB/s wr, 4 op/s 2024-09-21T06:35:29.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:29 smithi100 ceph-mon[33401]: pgmap v325: 129 pgs: 129 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 3.8 KiB/s wr, 4 op/s 2024-09-21T06:35:29.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:29 smithi203 ceph-mon[33735]: pgmap v325: 129 pgs: 129 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 3.8 KiB/s wr, 4 op/s 2024-09-21T06:35:30.317 INFO:teuthology.orchestra.run.smithi067.stdout:[{"version":52,"timestamp":"2024-09-21T06:34:22.182117+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-09-21T06:35:30.317 INFO:tasks.ceph_manager:config epoch is 52 2024-09-21T06:35:30.317 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T06:35:30.317 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-21T06:35:30.317 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph mgr dump --format=json 2024-09-21T06:35:30.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:30 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2042810755' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T06:35:30.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:30 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:35:30.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:30 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:35:30.581 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:30.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:30 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2042810755' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T06:35:30.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:30 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:35:30.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:30 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:35:30.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:30 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2042810755' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T06:35:30.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:30 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:35:30.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:30 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:35:31.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.057+0000 7f15c8a76640 1 -- 172.21.15.67:0/3553825571 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15c4104a30 msgr2=0x7f15c4104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:31.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.057+0000 7f15c8a76640 1 --2- 172.21.15.67:0/3553825571 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15c4104a30 0x7f15c4104e40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009ca0 tx=0x7f15b402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.057+0000 7f15c8a76640 1 -- 172.21.15.67:0/3553825571 shutdown_connections 2024-09-21T06:35:31.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.057+0000 7f15c8a76640 1 --2- 172.21.15.67:0/3553825571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15c4105380 0x7f15c410d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.057+0000 7f15c8a76640 1 --2- 172.21.15.67:0/3553825571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15c410de60 0x7f15c4110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.057+0000 7f15c8a76640 1 --2- 172.21.15.67:0/3553825571 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15c4104a30 0x7f15c4104e40 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.057+0000 7f15c8a76640 1 -- 172.21.15.67:0/3553825571 >> 172.21.15.67:0/3553825571 conn(0x7f15c4079760 msgr2=0x7f15c4079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:31.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.057+0000 7f15c8a76640 1 -- 172.21.15.67:0/3553825571 shutdown_connections 2024-09-21T06:35:31.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.057+0000 7f15c8a76640 1 -- 172.21.15.67:0/3553825571 wait complete. 2024-09-21T06:35:31.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.058+0000 7f15c8a76640 1 Processor -- start 2024-09-21T06:35:31.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.058+0000 7f15c8a76640 1 -- start start 2024-09-21T06:35:31.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.058+0000 7f15c8a76640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15c4104a30 0x7f15c419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.058+0000 7f15c8a76640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15c4105380 0x7f15c419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.058+0000 7f15c8a76640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15c410de60 0x7f15c419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.058+0000 7f15c8a76640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f15c41139c0 con 0x7f15c4104a30 2024-09-21T06:35:31.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.058+0000 7f15c8a76640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f15c4113840 con 0x7f15c4105380 2024-09-21T06:35:31.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.058+0000 7f15c8a76640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f15c4113b40 con 0x7f15c410de60 2024-09-21T06:35:31.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.058+0000 7f15c259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15c4104a30 0x7f15c419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.059+0000 7f15c259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15c4104a30 0x7f15c419a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49800/0 (socket says 172.21.15.67:49800) 2024-09-21T06:35:31.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.059+0000 7f15c259c640 1 -- 172.21.15.67:0/3242252389 learned_addr learned my addr 172.21.15.67:0/3242252389 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:31.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.059+0000 7f15c1d9b640 1 --2- 172.21.15.67:0/3242252389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15c4105380 0x7f15c419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.059+0000 7f15c2d9d640 1 --2- 172.21.15.67:0/3242252389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15c410de60 0x7f15c419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.059+0000 7f15c259c640 1 -- 172.21.15.67:0/3242252389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15c410de60 msgr2=0x7f15c419fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:31.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.059+0000 7f15c259c640 1 --2- 172.21.15.67:0/3242252389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15c410de60 0x7f15c419fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.059+0000 7f15c259c640 1 -- 172.21.15.67:0/3242252389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15c4105380 msgr2=0x7f15c419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:31.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.059+0000 7f15c259c640 1 --2- 172.21.15.67:0/3242252389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15c4105380 0x7f15c419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.059+0000 7f15c259c640 1 -- 172.21.15.67:0/3242252389 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c41a0380 con 0x7f15c4104a30 2024-09-21T06:35:31.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.059+0000 7f15c259c640 1 --2- 172.21.15.67:0/3242252389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15c4104a30 0x7f15c419a410 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f15b4032a10 tx=0x7f15b4032a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:31.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.059+0000 7f15b37fe640 1 -- 172.21.15.67:0/3242252389 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b4032e60 con 0x7f15c4104a30 2024-09-21T06:35:31.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.059+0000 7f15b37fe640 1 -- 172.21.15.67:0/3242252389 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15b40044d0 con 0x7f15c4104a30 2024-09-21T06:35:31.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.059+0000 7f15c8a76640 1 -- 172.21.15.67:0/3242252389 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15c41a0610 con 0x7f15c4104a30 2024-09-21T06:35:31.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.060+0000 7f15b37fe640 1 -- 172.21.15.67:0/3242252389 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b4030c00 con 0x7f15c4104a30 2024-09-21T06:35:31.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.060+0000 7f15c8a76640 1 -- 172.21.15.67:0/3242252389 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15c41a6dd0 con 0x7f15c4104a30 2024-09-21T06:35:31.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.061+0000 7f15c8a76640 1 -- 172.21.15.67:0/3242252389 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15c406c1f0 con 0x7f15c4104a30 2024-09-21T06:35:31.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.062+0000 7f15b37fe640 1 -- 172.21.15.67:0/3242252389 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f15b4004040 con 0x7f15c4104a30 2024-09-21T06:35:31.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.063+0000 7f15b37fe640 1 --2- 172.21.15.67:0/3242252389 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f15ac0b6620 0x7f15ac0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:31.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.063+0000 7f15c1d9b640 1 --2- 172.21.15.67:0/3242252389 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f15ac0b6620 0x7f15ac0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:31.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.063+0000 7f15b37fe640 1 -- 172.21.15.67:0/3242252389 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f15b4108ba0 con 0x7f15c4104a30 2024-09-21T06:35:31.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.064+0000 7f15c1d9b640 1 --2- 172.21.15.67:0/3242252389 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f15ac0b6620 0x7f15ac0b8b30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f15c4102410 tx=0x7f15b800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:31.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.066+0000 7f15b37fe640 1 -- 172.21.15.67:0/3242252389 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f15b40cf000 con 0x7f15c4104a30 2024-09-21T06:35:31.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.304+0000 7f15c8a76640 1 -- 172.21.15.67:0/3242252389 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f15c4104e40 con 0x7f15c4104a30 2024-09-21T06:35:31.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.308+0000 7f15b37fe640 1 -- 172.21.15.67:0/3242252389 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+296437 (secure 0 0 0) 0x7f15c4104e40 con 0x7f15c4104a30 2024-09-21T06:35:31.314 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:35:31.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.311+0000 7f15c8a76640 1 -- 172.21.15.67:0/3242252389 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f15ac0b6620 msgr2=0x7f15ac0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:31.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.311+0000 7f15c8a76640 1 --2- 172.21.15.67:0/3242252389 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f15ac0b6620 0x7f15ac0b8b30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f15c4102410 tx=0x7f15b800b410 comp rx=0 tx=0).stop 2024-09-21T06:35:31.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.311+0000 7f15c8a76640 1 -- 172.21.15.67:0/3242252389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15c4104a30 msgr2=0x7f15c419a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.311+0000 7f15c8a76640 1 --2- 172.21.15.67:0/3242252389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15c4104a30 0x7f15c419a410 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f15b4032a10 tx=0x7f15b4032a40 comp rx=0 tx=0).stop 2024-09-21T06:35:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.312+0000 7f15c8a76640 1 -- 172.21.15.67:0/3242252389 shutdown_connections 2024-09-21T06:35:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.312+0000 7f15c8a76640 1 --2- 172.21.15.67:0/3242252389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15c4105380 0x7f15c419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.312+0000 7f15c8a76640 1 --2- 172.21.15.67:0/3242252389 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f15ac0b6620 0x7f15ac0b8b30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.312+0000 7f15c8a76640 1 --2- 172.21.15.67:0/3242252389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15c410de60 0x7f15c419fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.312+0000 7f15c8a76640 1 --2- 172.21.15.67:0/3242252389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15c4104a30 0x7f15c419a410 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.312+0000 7f15c8a76640 1 -- 172.21.15.67:0/3242252389 >> 172.21.15.67:0/3242252389 conn(0x7f15c4079760 msgr2=0x7f15c4100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.312+0000 7f15c8a76640 1 -- 172.21.15.67:0/3242252389 shutdown_connections 2024-09-21T06:35:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:31.312+0000 7f15c8a76640 1 -- 172.21.15.67:0/3242252389 wait complete. 2024-09-21T06:35:31.717 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:31 smithi067 ceph-mon[31232]: pgmap v326: 129 pgs: 129 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 3.6 KiB/s wr, 3 op/s 2024-09-21T06:35:31.717 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:31 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:35:31.718 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:31 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:35:31.718 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:31 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:35:31.718 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:31 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3242252389' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T06:35:31.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:31 smithi100 ceph-mon[33401]: pgmap v326: 129 pgs: 129 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 3.6 KiB/s wr, 3 op/s 2024-09-21T06:35:31.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:31 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:35:31.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:31 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:35:31.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:31 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:35:31.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:31 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3242252389' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T06:35:31.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:31 smithi203 ceph-mon[33735]: pgmap v326: 129 pgs: 129 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 3.6 KiB/s wr, 3 op/s 2024-09-21T06:35:31.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:31 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:35:31.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:31 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:35:31.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:31 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:35:31.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:31 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3242252389' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T06:35:32.030 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1134893458},{"type":"v1","addr":"172.21.15.67:6801","nonce":1134893458}]},"active_addr":"172.21.15.67:6801/1134893458","active_change":"2024-09-21T06:27:03.740607+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24121,"name":"z","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24149,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.67:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":1315043503}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":2762272475}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":2644215278}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":993849907}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":2469773898}]}]} 2024-09-21T06:35:32.036 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-21T06:35:32.036 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-21T06:35:32.037 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:35:32.295 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:32.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.743+0000 7efcabc20640 1 -- 172.21.15.67:0/1593016863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efca41094a0 msgr2=0x7efca41098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.743+0000 7efcabc20640 1 --2- 172.21.15.67:0/1593016863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efca41094a0 0x7efca41098b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7efc94009900 tx=0x7efc9402a410 comp rx=0 tx=0).stop 2024-09-21T06:35:32.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.743+0000 7efcabc20640 1 -- 172.21.15.67:0/1593016863 shutdown_connections 2024-09-21T06:35:32.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.743+0000 7efcabc20640 1 --2- 172.21.15.67:0/1593016863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efca4109df0 0x7efca4110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.743+0000 7efcabc20640 1 --2- 172.21.15.67:0/1593016863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efca41094a0 0x7efca41098b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.743+0000 7efcabc20640 1 --2- 172.21.15.67:0/1593016863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efca4103b60 0x7efca4103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.743+0000 7efcabc20640 1 -- 172.21.15.67:0/1593016863 >> 172.21.15.67:0/1593016863 conn(0x7efca407b850 msgr2=0x7efca407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:32.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.743+0000 7efcabc20640 1 -- 172.21.15.67:0/1593016863 shutdown_connections 2024-09-21T06:35:32.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.743+0000 7efcabc20640 1 -- 172.21.15.67:0/1593016863 wait complete. 2024-09-21T06:35:32.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.744+0000 7efcabc20640 1 Processor -- start 2024-09-21T06:35:32.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.744+0000 7efcabc20640 1 -- start start 2024-09-21T06:35:32.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.744+0000 7efcabc20640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efca4103b60 0x7efca419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.744+0000 7efcabc20640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efca41094a0 0x7efca419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.744+0000 7efcabc20640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efca4109df0 0x7efca419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.744+0000 7efcabc20640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efca4113960 con 0x7efca4103b60 2024-09-21T06:35:32.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.744+0000 7efcabc20640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efca41137e0 con 0x7efca41094a0 2024-09-21T06:35:32.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.744+0000 7efcabc20640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7efca4113ae0 con 0x7efca4109df0 2024-09-21T06:35:32.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.744+0000 7efca99bc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efca4103b60 0x7efca419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.744+0000 7efca99bc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efca4103b60 0x7efca419a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49808/0 (socket says 172.21.15.67:49808) 2024-09-21T06:35:32.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.744+0000 7efca99bc640 1 -- 172.21.15.67:0/1402754237 learned_addr learned my addr 172.21.15.67:0/1402754237 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:32.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.744+0000 7efca91bb640 1 --2- 172.21.15.67:0/1402754237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efca41094a0 0x7efca419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.745+0000 7efcaa1bd640 1 --2- 172.21.15.67:0/1402754237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efca4109df0 0x7efca419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.745+0000 7efca99bc640 1 -- 172.21.15.67:0/1402754237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efca4109df0 msgr2=0x7efca419fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.745+0000 7efca99bc640 1 --2- 172.21.15.67:0/1402754237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efca4109df0 0x7efca419fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.745+0000 7efca99bc640 1 -- 172.21.15.67:0/1402754237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efca41094a0 msgr2=0x7efca419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.745+0000 7efca99bc640 1 --2- 172.21.15.67:0/1402754237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efca41094a0 0x7efca419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.745+0000 7efca99bc640 1 -- 172.21.15.67:0/1402754237 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca41a02e0 con 0x7efca4103b60 2024-09-21T06:35:32.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.745+0000 7efcaa1bd640 1 --2- 172.21.15.67:0/1402754237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efca4109df0 0x7efca419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:32.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.745+0000 7efca99bc640 1 --2- 172.21.15.67:0/1402754237 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efca4103b60 0x7efca419a430 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efca000c920 tx=0x7efca000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:32.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.745+0000 7efc9affd640 1 -- 172.21.15.67:0/1402754237 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca0007c70 con 0x7efca4103b60 2024-09-21T06:35:32.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.745+0000 7efcabc20640 1 -- 172.21.15.67:0/1402754237 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca41a05d0 con 0x7efca4103b60 2024-09-21T06:35:32.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.745+0000 7efc9affd640 1 -- 172.21.15.67:0/1402754237 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efca0007e10 con 0x7efca4103b60 2024-09-21T06:35:32.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.745+0000 7efc9affd640 1 -- 172.21.15.67:0/1402754237 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca0018710 con 0x7efca4103b60 2024-09-21T06:35:32.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.745+0000 7efcabc20640 1 -- 172.21.15.67:0/1402754237 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca41a6df0 con 0x7efca4103b60 2024-09-21T06:35:32.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.746+0000 7efcabc20640 1 -- 172.21.15.67:0/1402754237 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efca4103f70 con 0x7efca4103b60 2024-09-21T06:35:32.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.748+0000 7efc9affd640 1 -- 172.21.15.67:0/1402754237 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7efca000f450 con 0x7efca4103b60 2024-09-21T06:35:32.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.748+0000 7efc9affd640 1 --2- 172.21.15.67:0/1402754237 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7efc900b65d0 0x7efc900b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:32.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.749+0000 7efca91bb640 1 --2- 172.21.15.67:0/1402754237 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7efc900b65d0 0x7efc900b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:32.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.749+0000 7efc9affd640 1 -- 172.21.15.67:0/1402754237 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7efca00e7bf0 con 0x7efca4103b60 2024-09-21T06:35:32.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.749+0000 7efca91bb640 1 --2- 172.21.15.67:0/1402754237 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7efc900b65d0 0x7efc900b8ae0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efc94005940 tx=0x7efc940058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:32.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.752+0000 7efc9affd640 1 -- 172.21.15.67:0/1402754237 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efca0014030 con 0x7efca4103b60 2024-09-21T06:35:32.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.940+0000 7efcabc20640 1 -- 172.21.15.67:0/1402754237 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efca410b630 con 0x7efca4103b60 2024-09-21T06:35:32.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.942+0000 7efc9affd640 1 -- 172.21.15.67:0/1402754237 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+24709 (secure 0 0 0) 0x7efca00ae050 con 0x7efca4103b60 2024-09-21T06:35:32.944 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:35:32.945 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":83,"fsid":"2a791768-77e2-11ef-baf5-efdc52797490","created":"2024-09-21T06:26:15.222283+0000","modified":"2024-09-21T06:35:04.990843+0000","last_up_change":"2024-09-21T06:33:40.886310+0000","last_in_change":"2024-09-21T06:33:17.339486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:29:50.212825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:34:29.082881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:34:31.301094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:34:37.681356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"18778364-82ed-41c0-92da-ce44cdad7358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6803","nonce":2310759946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6805","nonce":2310759946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6809","nonce":2310759946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6807","nonce":2310759946}]},"public_addr":"172.21.15.67:6803/2310759946","cluster_addr":"172.21.15.67:6805/2310759946","heartbeat_back_addr":"172.21.15.67:6809/2310759946","heartbeat_front_addr":"172.21.15.67:6807/2310759946","state":["exists","up"]},{"osd":1,"uuid":"84e8d0c2-1c45-4673-ad49-f61df667d7f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6801","nonce":161807364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6803","nonce":161807364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6807","nonce":161807364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6805","nonce":161807364}]},"public_addr":"172.21.15.100:6801/161807364","cluster_addr":"172.21.15.100:6803/161807364","heartbeat_back_addr":"172.21.15.100:6807/161807364","heartbeat_front_addr":"172.21.15.100:6805/161807364","state":["exists","up"]},{"osd":2,"uuid":"e9a4d919-a779-4a1b-9b5b-d03a65f28ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6801","nonce":2754787169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6803","nonce":2754787169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6807","nonce":2754787169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6805","nonce":2754787169}]},"public_addr":"172.21.15.203:6801/2754787169","cluster_addr":"172.21.15.203:6803/2754787169","heartbeat_back_addr":"172.21.15.203:6807/2754787169","heartbeat_front_addr":"172.21.15.203:6805/2754787169","state":["exists","up"]},{"osd":3,"uuid":"3f7d9abd-587c-47ad-bf43-11f7dcdf71c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6811","nonce":4007517951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6813","nonce":4007517951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6817","nonce":4007517951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6815","nonce":4007517951}]},"public_addr":"172.21.15.67:6811/4007517951","cluster_addr":"172.21.15.67:6813/4007517951","heartbeat_back_addr":"172.21.15.67:6817/4007517951","heartbeat_front_addr":"172.21.15.67:6815/4007517951","state":["exists","up"]},{"osd":4,"uuid":"94f3a075-6d8a-4907-8418-a8513b35779a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6809","nonce":3773646501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6811","nonce":3773646501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6815","nonce":3773646501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6813","nonce":3773646501}]},"public_addr":"172.21.15.100:6809/3773646501","cluster_addr":"172.21.15.100:6811/3773646501","heartbeat_back_addr":"172.21.15.100:6815/3773646501","heartbeat_front_addr":"172.21.15.100:6813/3773646501","state":["exists","up"]},{"osd":5,"uuid":"5d842edf-06ad-49ae-ac71-863e5657270c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6809","nonce":2096171166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6811","nonce":2096171166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6815","nonce":2096171166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6813","nonce":2096171166}]},"public_addr":"172.21.15.203:6809/2096171166","cluster_addr":"172.21.15.203:6811/2096171166","heartbeat_back_addr":"172.21.15.203:6815/2096171166","heartbeat_front_addr":"172.21.15.203:6813/2096171166","state":["exists","up"]},{"osd":6,"uuid":"85b9beed-c93d-48ba-bc7a-c4cd816fbdff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6819","nonce":3229936280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6821","nonce":3229936280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6825","nonce":3229936280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6823","nonce":3229936280}]},"public_addr":"172.21.15.67:6819/3229936280","cluster_addr":"172.21.15.67:6821/3229936280","heartbeat_back_addr":"172.21.15.67:6825/3229936280","heartbeat_front_addr":"172.21.15.67:6823/3229936280","state":["exists","up"]},{"osd":7,"uuid":"86de30c2-b8ee-4c9d-9c17-8a56517934ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6817","nonce":1622917909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6819","nonce":1622917909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6823","nonce":1622917909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6821","nonce":1622917909}]},"public_addr":"172.21.15.100:6817/1622917909","cluster_addr":"172.21.15.100:6819/1622917909","heartbeat_back_addr":"172.21.15.100:6823/1622917909","heartbeat_front_addr":"172.21.15.100:6821/1622917909","state":["exists","up"]},{"osd":8,"uuid":"0e96cdad-a135-4115-9e2d-9a28306700e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6817","nonce":3452445943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6819","nonce":3452445943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6823","nonce":3452445943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6821","nonce":3452445943}]},"public_addr":"172.21.15.203:6817/3452445943","cluster_addr":"172.21.15.203:6819/3452445943","heartbeat_back_addr":"172.21.15.203:6823/3452445943","heartbeat_front_addr":"172.21.15.203:6821/3452445943","state":["exists","up"]},{"osd":9,"uuid":"95704ed6-6286-4094-9c6d-23919c7005ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6827","nonce":2998307533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6829","nonce":2998307533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6833","nonce":2998307533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6831","nonce":2998307533}]},"public_addr":"172.21.15.67:6827/2998307533","cluster_addr":"172.21.15.67:6829/2998307533","heartbeat_back_addr":"172.21.15.67:6833/2998307533","heartbeat_front_addr":"172.21.15.67:6831/2998307533","state":["exists","up"]},{"osd":10,"uuid":"bd2a7d33-b935-46c8-95c3-d76f28242ffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6825","nonce":2819057142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6827","nonce":2819057142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6831","nonce":2819057142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6829","nonce":2819057142}]},"public_addr":"172.21.15.100:6825/2819057142","cluster_addr":"172.21.15.100:6827/2819057142","heartbeat_back_addr":"172.21.15.100:6831/2819057142","heartbeat_front_addr":"172.21.15.100:6829/2819057142","state":["exists","up"]},{"osd":11,"uuid":"ea591507-10e2-4c08-8d29-d38ba9be96ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6825","nonce":566502600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6827","nonce":566502600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6831","nonce":566502600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6829","nonce":566502600}]},"public_addr":"172.21.15.203:6825/566502600","cluster_addr":"172.21.15.203:6827/566502600","heartbeat_back_addr":"172.21.15.203:6831/566502600","heartbeat_front_addr":"172.21.15.203:6829/566502600","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:28:58.945733+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:29:23.597118+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:29:48.303563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:30:14.139139+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:30:39.021508+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:04.981241+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:30.853841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:56.452967+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:32:22.239036+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:32:47.998803+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:33:13.850759+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:33:39.282150+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6800/1181476774":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/3100321138":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/2309364298":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/3458717076":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/4110206171":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6801/1181476774":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/2910038982":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6801/2622729103":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6800/2622729103":"2024-09-22T06:26:37.041377+0000","172.21.15.67:0/3346784528":"2024-09-22T06:26:37.041377+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:35:32.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.944+0000 7efcabc20640 1 -- 172.21.15.67:0/1402754237 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7efc900b65d0 msgr2=0x7efc900b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.944+0000 7efcabc20640 1 --2- 172.21.15.67:0/1402754237 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7efc900b65d0 0x7efc900b8ae0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efc94005940 tx=0x7efc940058d0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.944+0000 7efcabc20640 1 -- 172.21.15.67:0/1402754237 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efca4103b60 msgr2=0x7efca419a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:32.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.944+0000 7efcabc20640 1 --2- 172.21.15.67:0/1402754237 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efca4103b60 0x7efca419a430 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efca000c920 tx=0x7efca000cde0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.944+0000 7efcabc20640 1 -- 172.21.15.67:0/1402754237 shutdown_connections 2024-09-21T06:35:32.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.944+0000 7efcabc20640 1 --2- 172.21.15.67:0/1402754237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efca41094a0 0x7efca419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.944+0000 7efcabc20640 1 --2- 172.21.15.67:0/1402754237 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7efc900b65d0 0x7efc900b8ae0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.944+0000 7efcabc20640 1 --2- 172.21.15.67:0/1402754237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efca4109df0 0x7efca419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.944+0000 7efcabc20640 1 --2- 172.21.15.67:0/1402754237 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efca4103b60 0x7efca419a430 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:32.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.944+0000 7efcabc20640 1 -- 172.21.15.67:0/1402754237 >> 172.21.15.67:0/1402754237 conn(0x7efca407b850 msgr2=0x7efca41055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:32.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.945+0000 7efcabc20640 1 -- 172.21.15.67:0/1402754237 shutdown_connections 2024-09-21T06:35:32.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:32.945+0000 7efcabc20640 1 -- 172.21.15.67:0/1402754237 wait complete. 2024-09-21T06:35:33.649 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:33 smithi067 ceph-mon[31232]: pgmap v327: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-09-21T06:35:33.649 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:33 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1402754237' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:35:33.682 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-21T06:35:33.682 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:35:33.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:33 smithi100 ceph-mon[33401]: pgmap v327: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-09-21T06:35:33.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:33 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1402754237' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:35:33.943 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:33.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:33 smithi203 ceph-mon[33735]: pgmap v327: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-09-21T06:35:33.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:33 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1402754237' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:35:34.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.435+0000 7f569fe41640 1 -- 172.21.15.67:0/2420890208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f569810de10 msgr2=0x7f5698110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:34.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.435+0000 7f569fe41640 1 --2- 172.21.15.67:0/2420890208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f569810de10 0x7f5698110300 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5688009ca0 tx=0x7f568802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:35:34.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.435+0000 7f569fe41640 1 -- 172.21.15.67:0/2420890208 shutdown_connections 2024-09-21T06:35:34.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.435+0000 7f569fe41640 1 --2- 172.21.15.67:0/2420890208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f569810de10 0x7f5698110300 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:34.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.435+0000 7f569fe41640 1 --2- 172.21.15.67:0/2420890208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5698105300 0x7f569810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:34.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.435+0000 7f569fe41640 1 --2- 172.21.15.67:0/2420890208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56981049b0 0x7f5698104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:34.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.435+0000 7f569fe41640 1 -- 172.21.15.67:0/2420890208 >> 172.21.15.67:0/2420890208 conn(0x7f5698079730 msgr2=0x7f5698079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:34.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.435+0000 7f569fe41640 1 -- 172.21.15.67:0/2420890208 shutdown_connections 2024-09-21T06:35:34.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.435+0000 7f569fe41640 1 -- 172.21.15.67:0/2420890208 wait complete. 2024-09-21T06:35:34.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.436+0000 7f569fe41640 1 Processor -- start 2024-09-21T06:35:34.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.436+0000 7f569fe41640 1 -- start start 2024-09-21T06:35:34.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.436+0000 7f569fe41640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56981049b0 0x7f56981026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:34.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.436+0000 7f569fe41640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5698105300 0x7f5698102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:34.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.436+0000 7f569fe41640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f569810de10 0x7f5698103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:34.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.436+0000 7f569fe41640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5698113990 con 0x7f56981049b0 2024-09-21T06:35:34.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.436+0000 7f569fe41640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5698113810 con 0x7f569810de10 2024-09-21T06:35:34.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.436+0000 7f569fe41640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5698113b10 con 0x7f5698105300 2024-09-21T06:35:34.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.436+0000 7f569dbdd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56981049b0 0x7f56981026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:34.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.436+0000 7f569dbdd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56981049b0 0x7f56981026f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49824/0 (socket says 172.21.15.67:49824) 2024-09-21T06:35:34.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.436+0000 7f569dbdd640 1 -- 172.21.15.67:0/2798133536 learned_addr learned my addr 172.21.15.67:0/2798133536 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:34.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.437+0000 7f569d3dc640 1 --2- 172.21.15.67:0/2798133536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5698105300 0x7f5698102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:34.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.437+0000 7f569e3de640 1 --2- 172.21.15.67:0/2798133536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f569810de10 0x7f5698103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:34.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.437+0000 7f569dbdd640 1 -- 172.21.15.67:0/2798133536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5698105300 msgr2=0x7f5698102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:34.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.437+0000 7f569dbdd640 1 --2- 172.21.15.67:0/2798133536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5698105300 0x7f5698102c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:34.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.437+0000 7f569dbdd640 1 -- 172.21.15.67:0/2798133536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f569810de10 msgr2=0x7f5698103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:34.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.437+0000 7f569dbdd640 1 --2- 172.21.15.67:0/2798133536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f569810de10 0x7f5698103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:34.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.437+0000 7f569dbdd640 1 -- 172.21.15.67:0/2798133536 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56981a8640 con 0x7f56981049b0 2024-09-21T06:35:34.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.437+0000 7f569d3dc640 1 --2- 172.21.15.67:0/2798133536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5698105300 0x7f5698102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:34.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.437+0000 7f569dbdd640 1 --2- 172.21.15.67:0/2798133536 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56981049b0 0x7f56981026f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f569000d770 tx=0x7f569000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:34.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.437+0000 7f5696ffd640 1 -- 172.21.15.67:0/2798133536 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690017070 con 0x7f56981049b0 2024-09-21T06:35:34.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.437+0000 7f569fe41640 1 -- 172.21.15.67:0/2798133536 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56981a8930 con 0x7f56981049b0 2024-09-21T06:35:34.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.437+0000 7f5696ffd640 1 -- 172.21.15.67:0/2798133536 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5690009c30 con 0x7f56981049b0 2024-09-21T06:35:34.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.437+0000 7f5696ffd640 1 -- 172.21.15.67:0/2798133536 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690003c50 con 0x7f56981049b0 2024-09-21T06:35:34.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.437+0000 7f569fe41640 1 -- 172.21.15.67:0/2798133536 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56981a8ec0 con 0x7f56981049b0 2024-09-21T06:35:34.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.438+0000 7f569fe41640 1 -- 172.21.15.67:0/2798133536 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5698104dc0 con 0x7f56981049b0 2024-09-21T06:35:34.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.441+0000 7f5696ffd640 1 -- 172.21.15.67:0/2798133536 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f5690003df0 con 0x7f56981049b0 2024-09-21T06:35:34.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.441+0000 7f5696ffd640 1 --2- 172.21.15.67:0/2798133536 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f56800b65a0 0x7f56800b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:34.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.441+0000 7f569d3dc640 1 --2- 172.21.15.67:0/2798133536 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f56800b65a0 0x7f56800b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:34.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.442+0000 7f5696ffd640 1 -- 172.21.15.67:0/2798133536 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f56900e76e0 con 0x7f56981049b0 2024-09-21T06:35:34.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.442+0000 7f569d3dc640 1 --2- 172.21.15.67:0/2798133536 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f56800b65a0 0x7f56800b8ab0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f568c0058c0 tx=0x7f568c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:34.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.445+0000 7f5696ffd640 1 -- 172.21.15.67:0/2798133536 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f56900adab0 con 0x7f56981049b0 2024-09-21T06:35:34.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.635+0000 7f569fe41640 1 -- 172.21.15.67:0/2798133536 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56981039f0 con 0x7f56981049b0 2024-09-21T06:35:34.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.637+0000 7f5696ffd640 1 -- 172.21.15.67:0/2798133536 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+24709 (secure 0 0 0) 0x7f56981039f0 con 0x7f56981049b0 2024-09-21T06:35:34.639 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:35:34.639 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":83,"fsid":"2a791768-77e2-11ef-baf5-efdc52797490","created":"2024-09-21T06:26:15.222283+0000","modified":"2024-09-21T06:35:04.990843+0000","last_up_change":"2024-09-21T06:33:40.886310+0000","last_in_change":"2024-09-21T06:33:17.339486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:29:50.212825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:34:29.082881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:34:31.301094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:34:37.681356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"18778364-82ed-41c0-92da-ce44cdad7358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6803","nonce":2310759946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6805","nonce":2310759946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6809","nonce":2310759946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6807","nonce":2310759946}]},"public_addr":"172.21.15.67:6803/2310759946","cluster_addr":"172.21.15.67:6805/2310759946","heartbeat_back_addr":"172.21.15.67:6809/2310759946","heartbeat_front_addr":"172.21.15.67:6807/2310759946","state":["exists","up"]},{"osd":1,"uuid":"84e8d0c2-1c45-4673-ad49-f61df667d7f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6801","nonce":161807364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6803","nonce":161807364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6807","nonce":161807364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6805","nonce":161807364}]},"public_addr":"172.21.15.100:6801/161807364","cluster_addr":"172.21.15.100:6803/161807364","heartbeat_back_addr":"172.21.15.100:6807/161807364","heartbeat_front_addr":"172.21.15.100:6805/161807364","state":["exists","up"]},{"osd":2,"uuid":"e9a4d919-a779-4a1b-9b5b-d03a65f28ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6801","nonce":2754787169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6803","nonce":2754787169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6807","nonce":2754787169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6805","nonce":2754787169}]},"public_addr":"172.21.15.203:6801/2754787169","cluster_addr":"172.21.15.203:6803/2754787169","heartbeat_back_addr":"172.21.15.203:6807/2754787169","heartbeat_front_addr":"172.21.15.203:6805/2754787169","state":["exists","up"]},{"osd":3,"uuid":"3f7d9abd-587c-47ad-bf43-11f7dcdf71c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6811","nonce":4007517951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6813","nonce":4007517951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6817","nonce":4007517951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6815","nonce":4007517951}]},"public_addr":"172.21.15.67:6811/4007517951","cluster_addr":"172.21.15.67:6813/4007517951","heartbeat_back_addr":"172.21.15.67:6817/4007517951","heartbeat_front_addr":"172.21.15.67:6815/4007517951","state":["exists","up"]},{"osd":4,"uuid":"94f3a075-6d8a-4907-8418-a8513b35779a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6809","nonce":3773646501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6811","nonce":3773646501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6815","nonce":3773646501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6813","nonce":3773646501}]},"public_addr":"172.21.15.100:6809/3773646501","cluster_addr":"172.21.15.100:6811/3773646501","heartbeat_back_addr":"172.21.15.100:6815/3773646501","heartbeat_front_addr":"172.21.15.100:6813/3773646501","state":["exists","up"]},{"osd":5,"uuid":"5d842edf-06ad-49ae-ac71-863e5657270c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6809","nonce":2096171166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6811","nonce":2096171166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6815","nonce":2096171166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6813","nonce":2096171166}]},"public_addr":"172.21.15.203:6809/2096171166","cluster_addr":"172.21.15.203:6811/2096171166","heartbeat_back_addr":"172.21.15.203:6815/2096171166","heartbeat_front_addr":"172.21.15.203:6813/2096171166","state":["exists","up"]},{"osd":6,"uuid":"85b9beed-c93d-48ba-bc7a-c4cd816fbdff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6819","nonce":3229936280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6821","nonce":3229936280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6825","nonce":3229936280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6823","nonce":3229936280}]},"public_addr":"172.21.15.67:6819/3229936280","cluster_addr":"172.21.15.67:6821/3229936280","heartbeat_back_addr":"172.21.15.67:6825/3229936280","heartbeat_front_addr":"172.21.15.67:6823/3229936280","state":["exists","up"]},{"osd":7,"uuid":"86de30c2-b8ee-4c9d-9c17-8a56517934ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6817","nonce":1622917909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6819","nonce":1622917909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6823","nonce":1622917909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6821","nonce":1622917909}]},"public_addr":"172.21.15.100:6817/1622917909","cluster_addr":"172.21.15.100:6819/1622917909","heartbeat_back_addr":"172.21.15.100:6823/1622917909","heartbeat_front_addr":"172.21.15.100:6821/1622917909","state":["exists","up"]},{"osd":8,"uuid":"0e96cdad-a135-4115-9e2d-9a28306700e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6817","nonce":3452445943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6819","nonce":3452445943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6823","nonce":3452445943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6821","nonce":3452445943}]},"public_addr":"172.21.15.203:6817/3452445943","cluster_addr":"172.21.15.203:6819/3452445943","heartbeat_back_addr":"172.21.15.203:6823/3452445943","heartbeat_front_addr":"172.21.15.203:6821/3452445943","state":["exists","up"]},{"osd":9,"uuid":"95704ed6-6286-4094-9c6d-23919c7005ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6827","nonce":2998307533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6829","nonce":2998307533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6833","nonce":2998307533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6831","nonce":2998307533}]},"public_addr":"172.21.15.67:6827/2998307533","cluster_addr":"172.21.15.67:6829/2998307533","heartbeat_back_addr":"172.21.15.67:6833/2998307533","heartbeat_front_addr":"172.21.15.67:6831/2998307533","state":["exists","up"]},{"osd":10,"uuid":"bd2a7d33-b935-46c8-95c3-d76f28242ffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6825","nonce":2819057142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6827","nonce":2819057142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6831","nonce":2819057142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6829","nonce":2819057142}]},"public_addr":"172.21.15.100:6825/2819057142","cluster_addr":"172.21.15.100:6827/2819057142","heartbeat_back_addr":"172.21.15.100:6831/2819057142","heartbeat_front_addr":"172.21.15.100:6829/2819057142","state":["exists","up"]},{"osd":11,"uuid":"ea591507-10e2-4c08-8d29-d38ba9be96ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6825","nonce":566502600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6827","nonce":566502600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6831","nonce":566502600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6829","nonce":566502600}]},"public_addr":"172.21.15.203:6825/566502600","cluster_addr":"172.21.15.203:6827/566502600","heartbeat_back_addr":"172.21.15.203:6831/566502600","heartbeat_front_addr":"172.21.15.203:6829/566502600","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:28:58.945733+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:29:23.597118+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:29:48.303563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:30:14.139139+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:30:39.021508+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:04.981241+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:30.853841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:56.452967+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:32:22.239036+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:32:47.998803+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:33:13.850759+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:33:39.282150+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6800/1181476774":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/3100321138":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/2309364298":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/3458717076":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/4110206171":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6801/1181476774":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/2910038982":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6801/2622729103":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6800/2622729103":"2024-09-22T06:26:37.041377+0000","172.21.15.67:0/3346784528":"2024-09-22T06:26:37.041377+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:35:34.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.639+0000 7f569fe41640 1 -- 172.21.15.67:0/2798133536 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f56800b65a0 msgr2=0x7f56800b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:34.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.639+0000 7f569fe41640 1 --2- 172.21.15.67:0/2798133536 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f56800b65a0 0x7f56800b8ab0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f568c0058c0 tx=0x7f568c005850 comp rx=0 tx=0).stop 2024-09-21T06:35:34.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.639+0000 7f569fe41640 1 -- 172.21.15.67:0/2798133536 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56981049b0 msgr2=0x7f56981026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:34.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.639+0000 7f569fe41640 1 --2- 172.21.15.67:0/2798133536 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56981049b0 0x7f56981026f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f569000d770 tx=0x7f569000dc30 comp rx=0 tx=0).stop 2024-09-21T06:35:34.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.639+0000 7f569fe41640 1 -- 172.21.15.67:0/2798133536 shutdown_connections 2024-09-21T06:35:34.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.639+0000 7f569fe41640 1 --2- 172.21.15.67:0/2798133536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f569810de10 0x7f5698103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:34.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.639+0000 7f569fe41640 1 --2- 172.21.15.67:0/2798133536 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f56800b65a0 0x7f56800b8ab0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:34.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.639+0000 7f569fe41640 1 --2- 172.21.15.67:0/2798133536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5698105300 0x7f5698102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:34.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.639+0000 7f569fe41640 1 --2- 172.21.15.67:0/2798133536 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56981049b0 0x7f56981026f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:34.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.639+0000 7f569fe41640 1 -- 172.21.15.67:0/2798133536 >> 172.21.15.67:0/2798133536 conn(0x7f5698079730 msgr2=0x7f5698100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:34.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.639+0000 7f569fe41640 1 -- 172.21.15.67:0/2798133536 shutdown_connections 2024-09-21T06:35:34.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:34.639+0000 7f569fe41640 1 -- 172.21.15.67:0/2798133536 wait complete. 2024-09-21T06:35:35.296 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph tell osd.0 flush_pg_stats 2024-09-21T06:35:35.296 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph tell osd.1 flush_pg_stats 2024-09-21T06:35:35.297 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph tell osd.2 flush_pg_stats 2024-09-21T06:35:35.297 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph tell osd.3 flush_pg_stats 2024-09-21T06:35:35.297 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph tell osd.4 flush_pg_stats 2024-09-21T06:35:35.298 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph tell osd.5 flush_pg_stats 2024-09-21T06:35:35.298 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph tell osd.6 flush_pg_stats 2024-09-21T06:35:35.298 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph tell osd.7 flush_pg_stats 2024-09-21T06:35:35.299 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph tell osd.8 flush_pg_stats 2024-09-21T06:35:35.299 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph tell osd.9 flush_pg_stats 2024-09-21T06:35:35.300 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph tell osd.10 flush_pg_stats 2024-09-21T06:35:35.300 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph tell osd.11 flush_pg_stats 2024-09-21T06:35:35.544 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:35 smithi067 ceph-mon[31232]: pgmap v328: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.0 KiB/s wr, 2 op/s 2024-09-21T06:35:35.544 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:35 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2798133536' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:35:35.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:35 smithi100 ceph-mon[33401]: pgmap v328: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.0 KiB/s wr, 2 op/s 2024-09-21T06:35:35.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:35 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2798133536' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:35:35.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:35 smithi203 ceph-mon[33735]: pgmap v328: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.0 KiB/s wr, 2 op/s 2024-09-21T06:35:35.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:35 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2798133536' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:35:36.059 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:36.063 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:36.063 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:36.066 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:36.066 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:36.072 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:36.072 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:36.081 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:36.081 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:36.081 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:36.082 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:36.086 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:37.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.578+0000 7f2d08296640 1 -- 172.21.15.67:0/4152882899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d00108d00 msgr2=0x7f2d0010f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:37.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.578+0000 7f2d08296640 1 --2- 172.21.15.67:0/4152882899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d00108d00 0x7f2d0010f530 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0009900 tx=0x7f2cf002a410 comp rx=0 tx=0).stop 2024-09-21T06:35:37.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.578+0000 7f2d08296640 1 -- 172.21.15.67:0/4152882899 shutdown_connections 2024-09-21T06:35:37.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.578+0000 7f2d08296640 1 --2- 172.21.15.67:0/4152882899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d001083b0 0x7f2d001087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:37.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.578+0000 7f2d08296640 1 --2- 172.21.15.67:0/4152882899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d00108d00 0x7f2d0010f530 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:37.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.578+0000 7f2d08296640 1 --2- 172.21.15.67:0/4152882899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d00102a70 0x7f2d00102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:37.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.578+0000 7f2d08296640 1 -- 172.21.15.67:0/4152882899 >> 172.21.15.67:0/4152882899 conn(0x7f2d00079710 msgr2=0x7f2d00079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:37.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.578+0000 7f2d08296640 1 -- 172.21.15.67:0/4152882899 shutdown_connections 2024-09-21T06:35:37.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.578+0000 7f2d08296640 1 -- 172.21.15.67:0/4152882899 wait complete. 2024-09-21T06:35:37.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.579+0000 7f2d08296640 1 Processor -- start 2024-09-21T06:35:37.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.579+0000 7f2d08296640 1 -- start start 2024-09-21T06:35:37.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.579+0000 7f2d08296640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d00102a70 0x7f2d0019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:37.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.579+0000 7f2d08296640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d001083b0 0x7f2d0019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:37.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.579+0000 7f2d08296640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d00108d00 0x7f2d0019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:37.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.579+0000 7f2d08296640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2d00112870 con 0x7f2d001083b0 2024-09-21T06:35:37.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.579+0000 7f2d08296640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2d001126f0 con 0x7f2d00108d00 2024-09-21T06:35:37.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.579+0000 7f2d08296640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2d001129f0 con 0x7f2d00102a70 2024-09-21T06:35:37.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.580+0000 7f2d05831640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d001083b0 0x7f2d0019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:37.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.580+0000 7f2d05831640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d001083b0 0x7f2d0019a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33148/0 (socket says 172.21.15.67:33148) 2024-09-21T06:35:37.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.580+0000 7f2d05831640 1 -- 172.21.15.67:0/4015525488 learned_addr learned my addr 172.21.15.67:0/4015525488 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:37.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.580+0000 7f2d06833640 1 --2- 172.21.15.67:0/4015525488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d00108d00 0x7f2d0019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:37.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.580+0000 7f2d06032640 1 --2- 172.21.15.67:0/4015525488 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d00102a70 0x7f2d0019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:37.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.580+0000 7f2d05831640 1 -- 172.21.15.67:0/4015525488 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d00102a70 msgr2=0x7f2d0019a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:37.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.580+0000 7f2d05831640 1 --2- 172.21.15.67:0/4015525488 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d00102a70 0x7f2d0019a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:37.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.580+0000 7f2d05831640 1 -- 172.21.15.67:0/4015525488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d00108d00 msgr2=0x7f2d0019fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:37.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.580+0000 7f2d05831640 1 --2- 172.21.15.67:0/4015525488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d00108d00 0x7f2d0019fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:37.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.580+0000 7f2d05831640 1 -- 172.21.15.67:0/4015525488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d001a0380 con 0x7f2d001083b0 2024-09-21T06:35:37.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.580+0000 7f2d05831640 1 --2- 172.21.15.67:0/4015525488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d001083b0 0x7f2d0019a9b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400e9f0 tx=0x7f2cf400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:37.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.580+0000 7f2cff7fe640 1 -- 172.21.15.67:0/4015525488 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf400ccc0 con 0x7f2d001083b0 2024-09-21T06:35:37.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.580+0000 7f2d08296640 1 -- 172.21.15.67:0/4015525488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d001a0670 con 0x7f2d001083b0 2024-09-21T06:35:37.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.580+0000 7f2cff7fe640 1 -- 172.21.15.67:0/4015525488 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2cf40108c0 con 0x7f2d001083b0 2024-09-21T06:35:37.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.580+0000 7f2cff7fe640 1 -- 172.21.15.67:0/4015525488 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4018660 con 0x7f2d001083b0 2024-09-21T06:35:37.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.580+0000 7f2d08296640 1 -- 172.21.15.67:0/4015525488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d001a6dd0 con 0x7f2d001083b0 2024-09-21T06:35:37.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.581+0000 7f2d08296640 1 -- 172.21.15.67:0/4015525488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2d00063b50 con 0x7f2d001083b0 2024-09-21T06:35:37.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.583+0000 7f2cff7fe640 1 -- 172.21.15.67:0/4015525488 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f2cf4010430 con 0x7f2d001083b0 2024-09-21T06:35:37.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.584+0000 7f2cff7fe640 1 --2- 172.21.15.67:0/4015525488 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f2ce80b65a0 0x7f2ce80b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:37.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.584+0000 7f2d06032640 1 --2- 172.21.15.67:0/4015525488 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f2ce80b65a0 0x7f2ce80b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:37.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.584+0000 7f2cff7fe640 1 -- 172.21.15.67:0/4015525488 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f2cf40e8430 con 0x7f2d001083b0 2024-09-21T06:35:37.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.584+0000 7f2cff7fe640 1 --2- 172.21.15.67:0/4015525488 >> [v2:172.21.15.203:6808/2096171166,v1:172.21.15.203:6809/2096171166] conn(0x7f2ce80c16b0 0x7f2ce80c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:37.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.584+0000 7f2cff7fe640 1 -- 172.21.15.67:0/4015525488 --> [v2:172.21.15.203:6808/2096171166,v1:172.21.15.203:6809/2096171166] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2cf4028ea0 con 0x7f2ce80c16b0 2024-09-21T06:35:37.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.584+0000 7f2cff7fe640 1 -- 172.21.15.67:0/4015525488 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f2cf40e8880 con 0x7f2d001083b0 2024-09-21T06:35:37.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.585+0000 7f2d06032640 1 --2- 172.21.15.67:0/4015525488 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f2ce80b65a0 0x7f2ce80b8ab0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8005fd0 tx=0x7f2cf8009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:37.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.585+0000 7f2d06833640 1 --2- 172.21.15.67:0/4015525488 >> [v2:172.21.15.203:6808/2096171166,v1:172.21.15.203:6809/2096171166] conn(0x7f2ce80c16b0 0x7f2ce80c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:37.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.585+0000 7f2d06833640 1 --2- 172.21.15.67:0/4015525488 >> [v2:172.21.15.203:6808/2096171166,v1:172.21.15.203:6809/2096171166] conn(0x7f2ce80c16b0 0x7f2ce80c3ba0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:37.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.586+0000 7f2cff7fe640 1 -- 172.21.15.67:0/4015525488 <== osd.5 v2:172.21.15.203:6808/2096171166 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2cf4028ea0 con 0x7f2ce80c16b0 2024-09-21T06:35:37.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.600+0000 7f2d08296640 1 -- 172.21.15.67:0/4015525488 --> [v2:172.21.15.203:6808/2096171166,v1:172.21.15.203:6809/2096171166] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2d00063f50 con 0x7f2ce80c16b0 2024-09-21T06:35:37.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.601+0000 7f2cff7fe640 1 -- 172.21.15.67:0/4015525488 <== osd.5 v2:172.21.15.203:6808/2096171166 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2d00063f50 con 0x7f2ce80c16b0 2024-09-21T06:35:37.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.601+0000 7f2d08296640 1 -- 172.21.15.67:0/4015525488 >> [v2:172.21.15.203:6808/2096171166,v1:172.21.15.203:6809/2096171166] conn(0x7f2ce80c16b0 msgr2=0x7f2ce80c3ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:37.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.601+0000 7f2d08296640 1 --2- 172.21.15.67:0/4015525488 >> [v2:172.21.15.203:6808/2096171166,v1:172.21.15.203:6809/2096171166] conn(0x7f2ce80c16b0 0x7f2ce80c3ba0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:37.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.601+0000 7f2d08296640 1 -- 172.21.15.67:0/4015525488 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f2ce80b65a0 msgr2=0x7f2ce80b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:37.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.601+0000 7f2d08296640 1 --2- 172.21.15.67:0/4015525488 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f2ce80b65a0 0x7f2ce80b8ab0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8005fd0 tx=0x7f2cf8009870 comp rx=0 tx=0).stop 2024-09-21T06:35:37.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.601+0000 7f2d08296640 1 -- 172.21.15.67:0/4015525488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d001083b0 msgr2=0x7f2d0019a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:37.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.601+0000 7f2d08296640 1 --2- 172.21.15.67:0/4015525488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d001083b0 0x7f2d0019a9b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400e9f0 tx=0x7f2cf400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:35:37.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.601+0000 7f2d06833640 1 -- 172.21.15.67:0/4015525488 reap_dead start 2024-09-21T06:35:37.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.601+0000 7f2d08296640 1 -- 172.21.15.67:0/4015525488 shutdown_connections 2024-09-21T06:35:37.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.601+0000 7f2d08296640 1 -- 172.21.15.67:0/4015525488 >> 172.21.15.67:0/4015525488 conn(0x7f2d00079710 msgr2=0x7f2d00101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:37.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.601+0000 7f2d08296640 1 -- 172.21.15.67:0/4015525488 shutdown_connections 2024-09-21T06:35:37.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.601+0000 7f2d08296640 1 -- 172.21.15.67:0/4015525488 wait complete. 2024-09-21T06:35:37.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:37 smithi067 ceph-mon[31232]: pgmap v329: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-09-21T06:35:37.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:37 smithi100 ceph-mon[33401]: pgmap v329: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-09-21T06:35:37.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:37 smithi203 ceph-mon[33735]: pgmap v329: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-09-21T06:35:38.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.132+0000 7f0ed64de640 1 -- 172.21.15.67:0/537601394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ed0107b80 msgr2=0x7f0ed0107f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:38.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.132+0000 7f0ed64de640 1 --2- 172.21.15.67:0/537601394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ed0107b80 0x7f0ed0107f90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4009900 tx=0x7f0ec402a410 comp rx=0 tx=0).stop 2024-09-21T06:35:38.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.133+0000 7f0ed64de640 1 -- 172.21.15.67:0/537601394 shutdown_connections 2024-09-21T06:35:38.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.133+0000 7f0ed64de640 1 --2- 172.21.15.67:0/537601394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ed01084d0 0x7f0ed010ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:38.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.133+0000 7f0ed64de640 1 --2- 172.21.15.67:0/537601394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ed0107b80 0x7f0ed0107f90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:38.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.133+0000 7f0ed64de640 1 --2- 172.21.15.67:0/537601394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ed0102240 0x7f0ed0102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:38.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.133+0000 7f0ed64de640 1 -- 172.21.15.67:0/537601394 >> 172.21.15.67:0/537601394 conn(0x7f0ed0079f90 msgr2=0x7f0ed007a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:38.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.133+0000 7f0ed64de640 1 -- 172.21.15.67:0/537601394 shutdown_connections 2024-09-21T06:35:38.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.133+0000 7f0ed64de640 1 -- 172.21.15.67:0/537601394 wait complete. 2024-09-21T06:35:38.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.133+0000 7f0ed64de640 1 Processor -- start 2024-09-21T06:35:38.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.133+0000 7f0ed64de640 1 -- start start 2024-09-21T06:35:38.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.134+0000 7f0ed64de640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ed0102240 0x7f0ed019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:38.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.134+0000 7f0ed64de640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ed0107b80 0x7f0ed019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:38.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.134+0000 7f0ed64de640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ed01084d0 0x7f0ed019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:38.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.134+0000 7f0ed64de640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed01117c0 con 0x7f0ed0102240 2024-09-21T06:35:38.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.134+0000 7f0ed64de640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed0111640 con 0x7f0ed01084d0 2024-09-21T06:35:38.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.134+0000 7f0ed64de640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed0111940 con 0x7f0ed0107b80 2024-09-21T06:35:38.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.134+0000 7f0ecffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ed0102240 0x7f0ed019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:38.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.134+0000 7f0ecffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ed0102240 0x7f0ed019a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33162/0 (socket says 172.21.15.67:33162) 2024-09-21T06:35:38.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.134+0000 7f0ecffff640 1 -- 172.21.15.67:0/31478595 learned_addr learned my addr 172.21.15.67:0/31478595 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:38.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.134+0000 7f0ed4a7b640 1 --2- 172.21.15.67:0/31478595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ed01084d0 0x7f0ed019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:38.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.134+0000 7f0ecf7fe640 1 --2- 172.21.15.67:0/31478595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ed0107b80 0x7f0ed019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:38.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.134+0000 7f0ecffff640 1 -- 172.21.15.67:0/31478595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ed0107b80 msgr2=0x7f0ed019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:38.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.134+0000 7f0ecffff640 1 --2- 172.21.15.67:0/31478595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ed0107b80 0x7f0ed019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:38.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.134+0000 7f0ecffff640 1 -- 172.21.15.67:0/31478595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ed01084d0 msgr2=0x7f0ed019fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:38.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.134+0000 7f0ecffff640 1 --2- 172.21.15.67:0/31478595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ed01084d0 0x7f0ed019fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:38.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.134+0000 7f0ecffff640 1 -- 172.21.15.67:0/31478595 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed01a03c0 con 0x7f0ed0102240 2024-09-21T06:35:38.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.135+0000 7f0ecffff640 1 --2- 172.21.15.67:0/31478595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ed0102240 0x7f0ed019a450 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0ec000ddc0 tx=0x7f0ec000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:38.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.135+0000 7f0ecd7fa640 1 -- 172.21.15.67:0/31478595 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec000be60 con 0x7f0ed0102240 2024-09-21T06:35:38.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.135+0000 7f0ed64de640 1 -- 172.21.15.67:0/31478595 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ed01a06b0 con 0x7f0ed0102240 2024-09-21T06:35:38.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.135+0000 7f0ecd7fa640 1 -- 172.21.15.67:0/31478595 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ec00054b0 con 0x7f0ed0102240 2024-09-21T06:35:38.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.135+0000 7f0ecd7fa640 1 -- 172.21.15.67:0/31478595 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec0004d30 con 0x7f0ed0102240 2024-09-21T06:35:38.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.135+0000 7f0ed64de640 1 -- 172.21.15.67:0/31478595 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ed01a6e10 con 0x7f0ed0102240 2024-09-21T06:35:38.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.136+0000 7f0ed64de640 1 -- 172.21.15.67:0/31478595 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0ed0063b50 con 0x7f0ed0102240 2024-09-21T06:35:38.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.138+0000 7f0ecd7fa640 1 -- 172.21.15.67:0/31478595 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f0ec001d020 con 0x7f0ed0102240 2024-09-21T06:35:38.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.139+0000 7f0ecd7fa640 1 --2- 172.21.15.67:0/31478595 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f0eb80b65a0 0x7f0eb80b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:38.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.139+0000 7f0ecf7fe640 1 --2- 172.21.15.67:0/31478595 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f0eb80b65a0 0x7f0eb80b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:38.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.139+0000 7f0ecd7fa640 1 -- 172.21.15.67:0/31478595 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f0ec00e7e50 con 0x7f0ed0102240 2024-09-21T06:35:38.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.139+0000 7f0ecd7fa640 1 --2- 172.21.15.67:0/31478595 >> [v2:172.21.15.67:6802/2310759946,v1:172.21.15.67:6803/2310759946] conn(0x7f0eb80c16b0 0x7f0eb80c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:38.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.139+0000 7f0ecd7fa640 1 -- 172.21.15.67:0/31478595 --> [v2:172.21.15.67:6802/2310759946,v1:172.21.15.67:6803/2310759946] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0eb80c4270 con 0x7f0eb80c16b0 2024-09-21T06:35:38.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.139+0000 7f0ecd7fa640 1 -- 172.21.15.67:0/31478595 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f0ec00e82a0 con 0x7f0ed0102240 2024-09-21T06:35:38.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.139+0000 7f0ed4a7b640 1 --2- 172.21.15.67:0/31478595 >> [v2:172.21.15.67:6802/2310759946,v1:172.21.15.67:6803/2310759946] conn(0x7f0eb80c16b0 0x7f0eb80c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:38.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.139+0000 7f0ecf7fe640 1 --2- 172.21.15.67:0/31478595 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f0eb80b65a0 0x7f0eb80b8ab0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4005940 tx=0x7f0ec40058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:38.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.139+0000 7f0ed4a7b640 1 --2- 172.21.15.67:0/31478595 >> [v2:172.21.15.67:6802/2310759946,v1:172.21.15.67:6803/2310759946] conn(0x7f0eb80c16b0 0x7f0eb80c3ba0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:38.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.140+0000 7f0ecd7fa640 1 -- 172.21.15.67:0/31478595 <== osd.0 v2:172.21.15.67:6802/2310759946 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0eb80c4270 con 0x7f0eb80c16b0 2024-09-21T06:35:38.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.154+0000 7f0ed64de640 1 -- 172.21.15.67:0/31478595 --> [v2:172.21.15.67:6802/2310759946,v1:172.21.15.67:6803/2310759946] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0ed0063fd0 con 0x7f0eb80c16b0 2024-09-21T06:35:38.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.154+0000 7f0ecd7fa640 1 -- 172.21.15.67:0/31478595 <== osd.0 v2:172.21.15.67:6802/2310759946 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0ed0063fd0 con 0x7f0eb80c16b0 2024-09-21T06:35:38.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.154+0000 7f0ed64de640 1 -- 172.21.15.67:0/31478595 >> [v2:172.21.15.67:6802/2310759946,v1:172.21.15.67:6803/2310759946] conn(0x7f0eb80c16b0 msgr2=0x7f0eb80c3ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:38.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.154+0000 7f0ed64de640 1 --2- 172.21.15.67:0/31478595 >> [v2:172.21.15.67:6802/2310759946,v1:172.21.15.67:6803/2310759946] conn(0x7f0eb80c16b0 0x7f0eb80c3ba0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:38.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.154+0000 7f0ed64de640 1 -- 172.21.15.67:0/31478595 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f0eb80b65a0 msgr2=0x7f0eb80b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:38.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.154+0000 7f0ed64de640 1 --2- 172.21.15.67:0/31478595 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f0eb80b65a0 0x7f0eb80b8ab0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4005940 tx=0x7f0ec40058d0 comp rx=0 tx=0).stop 2024-09-21T06:35:38.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.154+0000 7f0ed64de640 1 -- 172.21.15.67:0/31478595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ed0102240 msgr2=0x7f0ed019a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:38.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.154+0000 7f0ed64de640 1 --2- 172.21.15.67:0/31478595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ed0102240 0x7f0ed019a450 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0ec000ddc0 tx=0x7f0ec000a620 comp rx=0 tx=0).stop 2024-09-21T06:35:38.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.155+0000 7f0ed4a7b640 1 -- 172.21.15.67:0/31478595 reap_dead start 2024-09-21T06:35:38.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.155+0000 7f0ed64de640 1 -- 172.21.15.67:0/31478595 shutdown_connections 2024-09-21T06:35:38.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.155+0000 7f0ed64de640 1 -- 172.21.15.67:0/31478595 >> 172.21.15.67:0/31478595 conn(0x7f0ed0079f90 msgr2=0x7f0ed01014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:38.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.155+0000 7f0ed64de640 1 -- 172.21.15.67:0/31478595 shutdown_connections 2024-09-21T06:35:38.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.155+0000 7f0ed64de640 1 -- 172.21.15.67:0/31478595 wait complete. 2024-09-21T06:35:38.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.771+0000 7f9dac1ab640 1 -- 172.21.15.67:0/2225399730 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9da4109df0 msgr2=0x7f9da4110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:38.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.771+0000 7f9dac1ab640 1 --2- 172.21.15.67:0/2225399730 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9da4109df0 0x7f9da4110620 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c009ca0 tx=0x7f9d9c02a8e0 comp rx=0 tx=0).stop 2024-09-21T06:35:38.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.771+0000 7f9dac1ab640 1 -- 172.21.15.67:0/2225399730 shutdown_connections 2024-09-21T06:35:38.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.771+0000 7f9dac1ab640 1 --2- 172.21.15.67:0/2225399730 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9da4109df0 0x7f9da4110620 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:38.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.771+0000 7f9dac1ab640 1 --2- 172.21.15.67:0/2225399730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9da41094a0 0x7f9da41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:38.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.771+0000 7f9dac1ab640 1 --2- 172.21.15.67:0/2225399730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9da4103b60 0x7f9da4103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:38.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.771+0000 7f9dac1ab640 1 -- 172.21.15.67:0/2225399730 >> 172.21.15.67:0/2225399730 conn(0x7f9da407b850 msgr2=0x7f9da407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:38.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.771+0000 7f9dac1ab640 1 -- 172.21.15.67:0/2225399730 shutdown_connections 2024-09-21T06:35:38.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.771+0000 7f9dac1ab640 1 -- 172.21.15.67:0/2225399730 wait complete. 2024-09-21T06:35:38.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.772+0000 7f9dac1ab640 1 Processor -- start 2024-09-21T06:35:38.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.772+0000 7f9dac1ab640 1 -- start start 2024-09-21T06:35:38.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.772+0000 7f9dac1ab640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9da4103b60 0x7f9da419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:38.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.772+0000 7f9dac1ab640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9da41094a0 0x7f9da419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:38.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.772+0000 7f9dac1ab640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9da4109df0 0x7f9da419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:38.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.772+0000 7f9dac1ab640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9da41139a0 con 0x7f9da4109df0 2024-09-21T06:35:38.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.772+0000 7f9dac1ab640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9da4113820 con 0x7f9da4103b60 2024-09-21T06:35:38.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.772+0000 7f9dac1ab640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9da4113b20 con 0x7f9da41094a0 2024-09-21T06:35:38.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.772+0000 7f9daa748640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9da4109df0 0x7f9da419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:38.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.772+0000 7f9daa748640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9da4109df0 0x7f9da419fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33176/0 (socket says 172.21.15.67:33176) 2024-09-21T06:35:38.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.772+0000 7f9daa748640 1 -- 172.21.15.67:0/1573190653 learned_addr learned my addr 172.21.15.67:0/1573190653 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:38.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.772+0000 7f9da9f47640 1 --2- 172.21.15.67:0/1573190653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9da4103b60 0x7f9da419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:38.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.773+0000 7f9da9746640 1 --2- 172.21.15.67:0/1573190653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9da41094a0 0x7f9da419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:38.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.773+0000 7f9daa748640 1 -- 172.21.15.67:0/1573190653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9da41094a0 msgr2=0x7f9da419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:38.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.773+0000 7f9daa748640 1 --2- 172.21.15.67:0/1573190653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9da41094a0 0x7f9da419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:38.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.773+0000 7f9daa748640 1 -- 172.21.15.67:0/1573190653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9da4103b60 msgr2=0x7f9da419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:38.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.773+0000 7f9daa748640 1 --2- 172.21.15.67:0/1573190653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9da4103b60 0x7f9da419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:38.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.773+0000 7f9daa748640 1 -- 172.21.15.67:0/1573190653 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da41a02e0 con 0x7f9da4109df0 2024-09-21T06:35:38.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.773+0000 7f9daa748640 1 --2- 172.21.15.67:0/1573190653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9da4109df0 0x7f9da419fc70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c006010 tx=0x7f9d9c003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:38.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.773+0000 7f9d9affd640 1 -- 172.21.15.67:0/1573190653 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c004240 con 0x7f9da4109df0 2024-09-21T06:35:38.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.773+0000 7f9dac1ab640 1 -- 172.21.15.67:0/1573190653 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da41a0570 con 0x7f9da4109df0 2024-09-21T06:35:38.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.773+0000 7f9d9affd640 1 -- 172.21.15.67:0/1573190653 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d9c0305e0 con 0x7f9da4109df0 2024-09-21T06:35:38.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.773+0000 7f9d9affd640 1 -- 172.21.15.67:0/1573190653 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c03b660 con 0x7f9da4109df0 2024-09-21T06:35:38.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.773+0000 7f9dac1ab640 1 -- 172.21.15.67:0/1573190653 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da41a6df0 con 0x7f9da4109df0 2024-09-21T06:35:38.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.774+0000 7f9dac1ab640 1 -- 172.21.15.67:0/1573190653 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9da4063b50 con 0x7f9da4109df0 2024-09-21T06:35:38.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.776+0000 7f9d9affd640 1 -- 172.21.15.67:0/1573190653 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f9d9c030790 con 0x7f9da4109df0 2024-09-21T06:35:38.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.777+0000 7f9d9affd640 1 --2- 172.21.15.67:0/1573190653 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9d900b65d0 0x7f9d900b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:38.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.777+0000 7f9da9f47640 1 --2- 172.21.15.67:0/1573190653 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9d900b65d0 0x7f9d900b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:38.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.777+0000 7f9d9affd640 1 -- 172.21.15.67:0/1573190653 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f9d9c108e50 con 0x7f9da4109df0 2024-09-21T06:35:38.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.777+0000 7f9d9affd640 1 --2- 172.21.15.67:0/1573190653 >> [v2:172.21.15.100:6816/1622917909,v1:172.21.15.100:6817/1622917909] conn(0x7f9d900c16e0 0x7f9d900c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:38.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.777+0000 7f9d9affd640 1 -- 172.21.15.67:0/1573190653 --> [v2:172.21.15.100:6816/1622917909,v1:172.21.15.100:6817/1622917909] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9d9c009360 con 0x7f9d900c16e0 2024-09-21T06:35:38.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.777+0000 7f9d9affd640 1 -- 172.21.15.67:0/1573190653 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f9d9c1092a0 con 0x7f9da4109df0 2024-09-21T06:35:38.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.777+0000 7f9da9f47640 1 --2- 172.21.15.67:0/1573190653 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9d900b65d0 0x7f9d900b8ae0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400abe0 tx=0x7f9d9400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:38.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.777+0000 7f9da9746640 1 --2- 172.21.15.67:0/1573190653 >> [v2:172.21.15.100:6816/1622917909,v1:172.21.15.100:6817/1622917909] conn(0x7f9d900c16e0 0x7f9d900c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:38.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.778+0000 7f9da9746640 1 --2- 172.21.15.67:0/1573190653 >> [v2:172.21.15.100:6816/1622917909,v1:172.21.15.100:6817/1622917909] conn(0x7f9d900c16e0 0x7f9d900c3bd0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:38.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.779+0000 7f9d9affd640 1 -- 172.21.15.67:0/1573190653 <== osd.7 v2:172.21.15.100:6816/1622917909 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9d9c009360 con 0x7f9d900c16e0 2024-09-21T06:35:38.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.792+0000 7f9dac1ab640 1 -- 172.21.15.67:0/1573190653 --> [v2:172.21.15.100:6816/1622917909,v1:172.21.15.100:6817/1622917909] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9da4063f50 con 0x7f9d900c16e0 2024-09-21T06:35:38.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.793+0000 7f9d9affd640 1 -- 172.21.15.67:0/1573190653 <== osd.7 v2:172.21.15.100:6816/1622917909 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9da4063f50 con 0x7f9d900c16e0 2024-09-21T06:35:38.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.794+0000 7f9dac1ab640 1 -- 172.21.15.67:0/1573190653 >> [v2:172.21.15.100:6816/1622917909,v1:172.21.15.100:6817/1622917909] conn(0x7f9d900c16e0 msgr2=0x7f9d900c3bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:38.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.794+0000 7f9dac1ab640 1 --2- 172.21.15.67:0/1573190653 >> [v2:172.21.15.100:6816/1622917909,v1:172.21.15.100:6817/1622917909] conn(0x7f9d900c16e0 0x7f9d900c3bd0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:38.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.794+0000 7f9dac1ab640 1 -- 172.21.15.67:0/1573190653 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9d900b65d0 msgr2=0x7f9d900b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:38.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.794+0000 7f9dac1ab640 1 --2- 172.21.15.67:0/1573190653 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9d900b65d0 0x7f9d900b8ae0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400abe0 tx=0x7f9d9400a2d0 comp rx=0 tx=0).stop 2024-09-21T06:35:38.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.794+0000 7f9dac1ab640 1 -- 172.21.15.67:0/1573190653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9da4109df0 msgr2=0x7f9da419fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:38.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.794+0000 7f9dac1ab640 1 --2- 172.21.15.67:0/1573190653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9da4109df0 0x7f9da419fc70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c006010 tx=0x7f9d9c003c90 comp rx=0 tx=0).stop 2024-09-21T06:35:38.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.794+0000 7f9daa748640 1 -- 172.21.15.67:0/1573190653 reap_dead start 2024-09-21T06:35:38.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.794+0000 7f9dac1ab640 1 -- 172.21.15.67:0/1573190653 shutdown_connections 2024-09-21T06:35:38.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.794+0000 7f9dac1ab640 1 -- 172.21.15.67:0/1573190653 >> 172.21.15.67:0/1573190653 conn(0x7f9da407b850 msgr2=0x7f9da41075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:38.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.794+0000 7f9dac1ab640 1 -- 172.21.15.67:0/1573190653 shutdown_connections 2024-09-21T06:35:38.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:38.794+0000 7f9dac1ab640 1 -- 172.21.15.67:0/1573190653 wait complete. 2024-09-21T06:35:38.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.131+0000 7ff8b3ea4640 1 -- 172.21.15.67:0/4082653248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8ac104a30 msgr2=0x7ff8ac104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:38.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.131+0000 7ff8b3ea4640 1 --2- 172.21.15.67:0/4082653248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8ac104a30 0x7ff8ac104e40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8009900 tx=0x7ff8a802a410 comp rx=0 tx=0).stop 2024-09-21T06:35:38.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.131+0000 7ff8b3ea4640 1 -- 172.21.15.67:0/4082653248 shutdown_connections 2024-09-21T06:35:38.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.131+0000 7ff8b3ea4640 1 --2- 172.21.15.67:0/4082653248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8ac105380 0x7ff8ac10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:38.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.131+0000 7ff8b3ea4640 1 --2- 172.21.15.67:0/4082653248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8ac10de60 0x7ff8ac110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:38.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.131+0000 7ff8b3ea4640 1 --2- 172.21.15.67:0/4082653248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8ac104a30 0x7ff8ac104e40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:38.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.131+0000 7ff8b3ea4640 1 -- 172.21.15.67:0/4082653248 >> 172.21.15.67:0/4082653248 conn(0x7ff8ac079760 msgr2=0x7ff8ac079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:38.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.131+0000 7ff8b3ea4640 1 -- 172.21.15.67:0/4082653248 shutdown_connections 2024-09-21T06:35:38.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.131+0000 7ff8b3ea4640 1 -- 172.21.15.67:0/4082653248 wait complete. 2024-09-21T06:35:38.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.132+0000 7ff8b3ea4640 1 Processor -- start 2024-09-21T06:35:38.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.132+0000 7ff8b3ea4640 1 -- start start 2024-09-21T06:35:38.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.132+0000 7ff8b3ea4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8ac104a30 0x7ff8ac19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:38.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.132+0000 7ff8b3ea4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8ac105380 0x7ff8ac19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:38.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.132+0000 7ff8b3ea4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8ac10de60 0x7ff8ac19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:38.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.132+0000 7ff8b3ea4640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac113970 con 0x7ff8ac104a30 2024-09-21T06:35:38.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.132+0000 7ff8b3ea4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac1137f0 con 0x7ff8ac10de60 2024-09-21T06:35:38.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.132+0000 7ff8b3ea4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac113af0 con 0x7ff8ac105380 2024-09-21T06:35:38.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.132+0000 7ff8b1c40640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8ac104a30 0x7ff8ac19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:38.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.132+0000 7ff8b1c40640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8ac104a30 0x7ff8ac19a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49846/0 (socket says 172.21.15.67:49846) 2024-09-21T06:35:38.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.132+0000 7ff8b1c40640 1 -- 172.21.15.67:0/3288998049 learned_addr learned my addr 172.21.15.67:0/3288998049 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:38.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.133+0000 7ff8b2441640 1 --2- 172.21.15.67:0/3288998049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8ac10de60 0x7ff8ac19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:38.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.133+0000 7ff8b143f640 1 --2- 172.21.15.67:0/3288998049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8ac105380 0x7ff8ac19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:38.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.133+0000 7ff8b1c40640 1 -- 172.21.15.67:0/3288998049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8ac105380 msgr2=0x7ff8ac19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:38.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.133+0000 7ff8b1c40640 1 --2- 172.21.15.67:0/3288998049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8ac105380 0x7ff8ac19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:38.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.133+0000 7ff8b1c40640 1 -- 172.21.15.67:0/3288998049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8ac10de60 msgr2=0x7ff8ac19fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:38.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.133+0000 7ff8b1c40640 1 --2- 172.21.15.67:0/3288998049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8ac10de60 0x7ff8ac19fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:38.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.133+0000 7ff8b1c40640 1 -- 172.21.15.67:0/3288998049 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ac1a0370 con 0x7ff8ac104a30 2024-09-21T06:35:38.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.133+0000 7ff8b1c40640 1 --2- 172.21.15.67:0/3288998049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8ac104a30 0x7ff8ac19a400 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8005940 tx=0x7ff8a8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:38.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.133+0000 7ff8a2ffd640 1 -- 172.21.15.67:0/3288998049 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a803f030 con 0x7ff8ac104a30 2024-09-21T06:35:38.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.133+0000 7ff8a2ffd640 1 -- 172.21.15.67:0/3288998049 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff8a8004440 con 0x7ff8ac104a30 2024-09-21T06:35:38.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.133+0000 7ff8b3ea4640 1 -- 172.21.15.67:0/3288998049 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8ac1a0600 con 0x7ff8ac104a30 2024-09-21T06:35:38.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.133+0000 7ff8a2ffd640 1 -- 172.21.15.67:0/3288998049 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8035070 con 0x7ff8ac104a30 2024-09-21T06:35:38.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.133+0000 7ff8b3ea4640 1 -- 172.21.15.67:0/3288998049 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8ac1a6dc0 con 0x7ff8ac104a30 2024-09-21T06:35:38.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.134+0000 7ff8b3ea4640 1 -- 172.21.15.67:0/3288998049 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff8ac063b50 con 0x7ff8ac104a30 2024-09-21T06:35:38.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.136+0000 7ff8a2ffd640 1 -- 172.21.15.67:0/3288998049 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7ff8a8042030 con 0x7ff8ac104a30 2024-09-21T06:35:38.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.137+0000 7ff8a2ffd640 1 --2- 172.21.15.67:0/3288998049 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7ff8980b6620 0x7ff8980b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:38.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.137+0000 7ff8b143f640 1 --2- 172.21.15.67:0/3288998049 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7ff8980b6620 0x7ff8980b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:38.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.137+0000 7ff8a2ffd640 1 -- 172.21.15.67:0/3288998049 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7ff8a8107fb0 con 0x7ff8ac104a30 2024-09-21T06:35:38.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.137+0000 7ff8a2ffd640 1 --2- 172.21.15.67:0/3288998049 >> [v2:172.21.15.100:6800/161807364,v1:172.21.15.100:6801/161807364] conn(0x7ff8980c1730 0x7ff8980c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:38.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.137+0000 7ff8a2ffd640 1 -- 172.21.15.67:0/3288998049 --> [v2:172.21.15.100:6800/161807364,v1:172.21.15.100:6801/161807364] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff8980c42f0 con 0x7ff8980c1730 2024-09-21T06:35:38.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.137+0000 7ff8a2ffd640 1 -- 172.21.15.67:0/3288998049 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7ff8a8108400 con 0x7ff8ac104a30 2024-09-21T06:35:38.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.137+0000 7ff8b143f640 1 --2- 172.21.15.67:0/3288998049 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7ff8980b6620 0x7ff8980b8b30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac079f70 tx=0x7ff89c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:38.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.137+0000 7ff8b2441640 1 --2- 172.21.15.67:0/3288998049 >> [v2:172.21.15.100:6800/161807364,v1:172.21.15.100:6801/161807364] conn(0x7ff8980c1730 0x7ff8980c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:38.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.138+0000 7ff8b2441640 1 --2- 172.21.15.67:0/3288998049 >> [v2:172.21.15.100:6800/161807364,v1:172.21.15.100:6801/161807364] conn(0x7ff8980c1730 0x7ff8980c3c20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:38.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.139+0000 7ff8a2ffd640 1 -- 172.21.15.67:0/3288998049 <== osd.1 v2:172.21.15.100:6800/161807364 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff8980c42f0 con 0x7ff8980c1730 2024-09-21T06:35:38.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.153+0000 7ff8b3ea4640 1 -- 172.21.15.67:0/3288998049 --> [v2:172.21.15.100:6800/161807364,v1:172.21.15.100:6801/161807364] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff8ac063f50 con 0x7ff8980c1730 2024-09-21T06:35:38.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.153+0000 7ff8a2ffd640 1 -- 172.21.15.67:0/3288998049 <== osd.1 v2:172.21.15.100:6800/161807364 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff8ac063f50 con 0x7ff8980c1730 2024-09-21T06:35:38.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.154+0000 7ff8b3ea4640 1 -- 172.21.15.67:0/3288998049 >> [v2:172.21.15.100:6800/161807364,v1:172.21.15.100:6801/161807364] conn(0x7ff8980c1730 msgr2=0x7ff8980c3c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:38.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.154+0000 7ff8b3ea4640 1 --2- 172.21.15.67:0/3288998049 >> [v2:172.21.15.100:6800/161807364,v1:172.21.15.100:6801/161807364] conn(0x7ff8980c1730 0x7ff8980c3c20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:38.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.154+0000 7ff8b3ea4640 1 -- 172.21.15.67:0/3288998049 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7ff8980b6620 msgr2=0x7ff8980b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:38.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.154+0000 7ff8b3ea4640 1 --2- 172.21.15.67:0/3288998049 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7ff8980b6620 0x7ff8980b8b30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac079f70 tx=0x7ff89c00b390 comp rx=0 tx=0).stop 2024-09-21T06:35:38.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.154+0000 7ff8b3ea4640 1 -- 172.21.15.67:0/3288998049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8ac104a30 msgr2=0x7ff8ac19a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:38.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.154+0000 7ff8b3ea4640 1 --2- 172.21.15.67:0/3288998049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8ac104a30 0x7ff8ac19a400 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8005940 tx=0x7ff8a8004040 comp rx=0 tx=0).stop 2024-09-21T06:35:38.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.154+0000 7ff8b2441640 1 -- 172.21.15.67:0/3288998049 reap_dead start 2024-09-21T06:35:38.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.154+0000 7ff8b3ea4640 1 -- 172.21.15.67:0/3288998049 shutdown_connections 2024-09-21T06:35:38.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.154+0000 7ff8b3ea4640 1 -- 172.21.15.67:0/3288998049 >> 172.21.15.67:0/3288998049 conn(0x7ff8ac079760 msgr2=0x7ff8ac100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:38.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.154+0000 7ff8b3ea4640 1 -- 172.21.15.67:0/3288998049 shutdown_connections 2024-09-21T06:35:38.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:37.154+0000 7ff8b3ea4640 1 -- 172.21.15.67:0/3288998049 wait complete. 2024-09-21T06:35:39.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.518+0000 7f12b9a83640 1 -- 172.21.15.67:0/1441638276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12b4105330 msgr2=0x7f12b410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:39.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.518+0000 7f12b9a83640 1 --2- 172.21.15.67:0/1441638276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12b4105330 0x7f12b410d880 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009900 tx=0x7f12a802a3f0 comp rx=0 tx=0).stop 2024-09-21T06:35:39.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.518+0000 7f12b9a83640 1 -- 172.21.15.67:0/1441638276 shutdown_connections 2024-09-21T06:35:39.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.518+0000 7f12b9a83640 1 --2- 172.21.15.67:0/1441638276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12b410de40 0x7f12b4110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:39.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.518+0000 7f12b9a83640 1 --2- 172.21.15.67:0/1441638276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12b4105330 0x7f12b410d880 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:39.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.518+0000 7f12b9a83640 1 --2- 172.21.15.67:0/1441638276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12b41049e0 0x7f12b4104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:39.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.518+0000 7f12b9a83640 1 -- 172.21.15.67:0/1441638276 >> 172.21.15.67:0/1441638276 conn(0x7f12b4079750 msgr2=0x7f12b4079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:39.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.518+0000 7f12b9a83640 1 -- 172.21.15.67:0/1441638276 shutdown_connections 2024-09-21T06:35:39.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.518+0000 7f12b9a83640 1 -- 172.21.15.67:0/1441638276 wait complete. 2024-09-21T06:35:39.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.519+0000 7f12b9a83640 1 Processor -- start 2024-09-21T06:35:39.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.519+0000 7f12b9a83640 1 -- start start 2024-09-21T06:35:39.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.519+0000 7f12b9a83640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12b41049e0 0x7f12b419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:39.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.519+0000 7f12b9a83640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12b4105330 0x7f12b419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:39.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.519+0000 7f12b9a83640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12b410de40 0x7f12b419fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:39.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.519+0000 7f12b9a83640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f12b4113950 con 0x7f12b4105330 2024-09-21T06:35:39.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.519+0000 7f12b9a83640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f12b41137d0 con 0x7f12b410de40 2024-09-21T06:35:39.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.519+0000 7f12b9a83640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f12b4113ad0 con 0x7f12b41049e0 2024-09-21T06:35:39.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.519+0000 7f12b2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12b4105330 0x7f12b419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:39.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.519+0000 7f12b2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12b4105330 0x7f12b419a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33188/0 (socket says 172.21.15.67:33188) 2024-09-21T06:35:39.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.519+0000 7f12b2ffd640 1 -- 172.21.15.67:0/3632667029 learned_addr learned my addr 172.21.15.67:0/3632667029 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:39.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.519+0000 7f12b37fe640 1 --2- 172.21.15.67:0/3632667029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12b41049e0 0x7f12b419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:39.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.519+0000 7f12b3fff640 1 --2- 172.21.15.67:0/3632667029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12b410de40 0x7f12b419fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:39.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.520+0000 7f12b2ffd640 1 -- 172.21.15.67:0/3632667029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12b41049e0 msgr2=0x7f12b419a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:39.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.520+0000 7f12b2ffd640 1 --2- 172.21.15.67:0/3632667029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12b41049e0 0x7f12b419a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:39.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.520+0000 7f12b2ffd640 1 -- 172.21.15.67:0/3632667029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12b410de40 msgr2=0x7f12b419fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:39.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.520+0000 7f12b2ffd640 1 --2- 172.21.15.67:0/3632667029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12b410de40 0x7f12b419fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:39.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.520+0000 7f12b2ffd640 1 -- 172.21.15.67:0/3632667029 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b41a0370 con 0x7f12b4105330 2024-09-21T06:35:39.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.520+0000 7f12b2ffd640 1 --2- 172.21.15.67:0/3632667029 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12b4105330 0x7f12b419a9a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f12a8005940 tx=0x7f12a8005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:39.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.520+0000 7f12b0ff9640 1 -- 172.21.15.67:0/3632667029 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a803f030 con 0x7f12b4105330 2024-09-21T06:35:39.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.520+0000 7f12b0ff9640 1 -- 172.21.15.67:0/3632667029 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f12a80043d0 con 0x7f12b4105330 2024-09-21T06:35:39.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.520+0000 7f12b0ff9640 1 -- 172.21.15.67:0/3632667029 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a8035070 con 0x7f12b4105330 2024-09-21T06:35:39.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.520+0000 7f12b9a83640 1 -- 172.21.15.67:0/3632667029 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b41a0600 con 0x7f12b4105330 2024-09-21T06:35:39.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.520+0000 7f12b9a83640 1 -- 172.21.15.67:0/3632667029 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b41a6dc0 con 0x7f12b4105330 2024-09-21T06:35:39.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.521+0000 7f12b9a83640 1 -- 172.21.15.67:0/3632667029 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f12b4063b50 con 0x7f12b4105330 2024-09-21T06:35:39.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.523+0000 7f12b0ff9640 1 -- 172.21.15.67:0/3632667029 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f12a8042030 con 0x7f12b4105330 2024-09-21T06:35:39.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.523+0000 7f12b0ff9640 1 --2- 172.21.15.67:0/3632667029 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f129c0b65a0 0x7f129c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:39.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.524+0000 7f12b37fe640 1 --2- 172.21.15.67:0/3632667029 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f129c0b65a0 0x7f129c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:39.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.524+0000 7f12b0ff9640 1 -- 172.21.15.67:0/3632667029 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f12a8107f60 con 0x7f12b4105330 2024-09-21T06:35:39.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.524+0000 7f12b0ff9640 1 --2- 172.21.15.67:0/3632667029 >> [v2:172.21.15.203:6800/2754787169,v1:172.21.15.203:6801/2754787169] conn(0x7f129c0c16b0 0x7f129c0c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:39.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.524+0000 7f12b0ff9640 1 -- 172.21.15.67:0/3632667029 --> [v2:172.21.15.203:6800/2754787169,v1:172.21.15.203:6801/2754787169] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f129c0c4270 con 0x7f129c0c16b0 2024-09-21T06:35:39.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.524+0000 7f12b0ff9640 1 -- 172.21.15.67:0/3632667029 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f12a81083b0 con 0x7f12b4105330 2024-09-21T06:35:39.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.524+0000 7f12b37fe640 1 --2- 172.21.15.67:0/3632667029 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f129c0b65a0 0x7f129c0b8ab0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f12b406c980 tx=0x7f12a4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:39.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.524+0000 7f12b3fff640 1 --2- 172.21.15.67:0/3632667029 >> [v2:172.21.15.203:6800/2754787169,v1:172.21.15.203:6801/2754787169] conn(0x7f129c0c16b0 0x7f129c0c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:39.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.525+0000 7f12b3fff640 1 --2- 172.21.15.67:0/3632667029 >> [v2:172.21.15.203:6800/2754787169,v1:172.21.15.203:6801/2754787169] conn(0x7f129c0c16b0 0x7f129c0c3ba0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:39.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.526+0000 7f12b0ff9640 1 -- 172.21.15.67:0/3632667029 <== osd.2 v2:172.21.15.203:6800/2754787169 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f129c0c4270 con 0x7f129c0c16b0 2024-09-21T06:35:39.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.539+0000 7f12b9a83640 1 -- 172.21.15.67:0/3632667029 --> [v2:172.21.15.203:6800/2754787169,v1:172.21.15.203:6801/2754787169] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f12b4063fd0 con 0x7f129c0c16b0 2024-09-21T06:35:39.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.540+0000 7f12b0ff9640 1 -- 172.21.15.67:0/3632667029 <== osd.2 v2:172.21.15.203:6800/2754787169 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f12b4063fd0 con 0x7f129c0c16b0 2024-09-21T06:35:39.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.540+0000 7f12b9a83640 1 -- 172.21.15.67:0/3632667029 >> [v2:172.21.15.203:6800/2754787169,v1:172.21.15.203:6801/2754787169] conn(0x7f129c0c16b0 msgr2=0x7f129c0c3ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:39.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.540+0000 7f12b9a83640 1 --2- 172.21.15.67:0/3632667029 >> [v2:172.21.15.203:6800/2754787169,v1:172.21.15.203:6801/2754787169] conn(0x7f129c0c16b0 0x7f129c0c3ba0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:39.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.540+0000 7f12b9a83640 1 -- 172.21.15.67:0/3632667029 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f129c0b65a0 msgr2=0x7f129c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:39.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.540+0000 7f12b9a83640 1 --2- 172.21.15.67:0/3632667029 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f129c0b65a0 0x7f129c0b8ab0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f12b406c980 tx=0x7f12a4005f90 comp rx=0 tx=0).stop 2024-09-21T06:35:39.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.540+0000 7f12b9a83640 1 -- 172.21.15.67:0/3632667029 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12b4105330 msgr2=0x7f12b419a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:39.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.540+0000 7f12b9a83640 1 --2- 172.21.15.67:0/3632667029 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12b4105330 0x7f12b419a9a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f12a8005940 tx=0x7f12a8005e10 comp rx=0 tx=0).stop 2024-09-21T06:35:39.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.540+0000 7f12b3fff640 1 -- 172.21.15.67:0/3632667029 reap_dead start 2024-09-21T06:35:39.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.540+0000 7f12b9a83640 1 -- 172.21.15.67:0/3632667029 shutdown_connections 2024-09-21T06:35:39.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.540+0000 7f12b9a83640 1 -- 172.21.15.67:0/3632667029 >> 172.21.15.67:0/3632667029 conn(0x7f12b4079750 msgr2=0x7f12b40ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:39.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.541+0000 7f12b9a83640 1 -- 172.21.15.67:0/3632667029 shutdown_connections 2024-09-21T06:35:39.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:39.541+0000 7f12b9a83640 1 -- 172.21.15.67:0/3632667029 wait complete. 2024-09-21T06:35:39.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:39 smithi067 ceph-mon[31232]: pgmap v330: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:35:39.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:39 smithi100 ceph-mon[33401]: pgmap v330: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:35:39.875 INFO:teuthology.orchestra.run.smithi067.stdout:55834574925 2024-09-21T06:35:39.876 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.1 2024-09-21T06:35:39.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:39 smithi203 ceph-mon[33735]: pgmap v330: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:35:41.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.094+0000 7fba91857640 1 -- 172.21.15.67:0/3177138258 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba8c108d10 msgr2=0x7fba8c10f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.094+0000 7fba91857640 1 --2- 172.21.15.67:0/3177138258 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba8c108d10 0x7fba8c10f540 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fba84009ca0 tx=0x7fba8402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.094+0000 7fba91857640 1 -- 172.21.15.67:0/3177138258 shutdown_connections 2024-09-21T06:35:41.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.094+0000 7fba91857640 1 --2- 172.21.15.67:0/3177138258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba8c1083c0 0x7fba8c1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.094+0000 7fba91857640 1 --2- 172.21.15.67:0/3177138258 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba8c108d10 0x7fba8c10f540 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.094+0000 7fba91857640 1 --2- 172.21.15.67:0/3177138258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba8c102a80 0x7fba8c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.094+0000 7fba91857640 1 -- 172.21.15.67:0/3177138258 >> 172.21.15.67:0/3177138258 conn(0x7fba8c079740 msgr2=0x7fba8c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:41.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.094+0000 7fba91857640 1 -- 172.21.15.67:0/3177138258 shutdown_connections 2024-09-21T06:35:41.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.094+0000 7fba91857640 1 -- 172.21.15.67:0/3177138258 wait complete. 2024-09-21T06:35:41.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.095+0000 7fba91857640 1 Processor -- start 2024-09-21T06:35:41.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.095+0000 7fba91857640 1 -- start start 2024-09-21T06:35:41.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.095+0000 7fba91857640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba8c102a80 0x7fba8c19a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.095+0000 7fba91857640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba8c1083c0 0x7fba8c19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.095+0000 7fba91857640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba8c108d10 0x7fba8c19fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.095+0000 7fba91857640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c112880 con 0x7fba8c102a80 2024-09-21T06:35:41.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.095+0000 7fba91857640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c112700 con 0x7fba8c108d10 2024-09-21T06:35:41.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.095+0000 7fba91857640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c112a00 con 0x7fba8c1083c0 2024-09-21T06:35:41.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.095+0000 7fba8affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba8c102a80 0x7fba8c19a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.095+0000 7fba8affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba8c102a80 0x7fba8c19a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33208/0 (socket says 172.21.15.67:33208) 2024-09-21T06:35:41.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.095+0000 7fba8affd640 1 -- 172.21.15.67:0/2431875826 learned_addr learned my addr 172.21.15.67:0/2431875826 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:41.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.095+0000 7fba8a7fc640 1 --2- 172.21.15.67:0/2431875826 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba8c1083c0 0x7fba8c19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.095+0000 7fba8b7fe640 1 --2- 172.21.15.67:0/2431875826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba8c108d10 0x7fba8c19fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.096+0000 7fba8affd640 1 -- 172.21.15.67:0/2431875826 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba8c1083c0 msgr2=0x7fba8c19a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.096+0000 7fba8affd640 1 --2- 172.21.15.67:0/2431875826 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba8c1083c0 0x7fba8c19a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.096+0000 7fba8affd640 1 -- 172.21.15.67:0/2431875826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba8c108d10 msgr2=0x7fba8c19fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.096+0000 7fba8affd640 1 --2- 172.21.15.67:0/2431875826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba8c108d10 0x7fba8c19fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.096+0000 7fba8affd640 1 -- 172.21.15.67:0/2431875826 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba8c1a0230 con 0x7fba8c102a80 2024-09-21T06:35:41.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.096+0000 7fba8b7fe640 1 --2- 172.21.15.67:0/2431875826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba8c108d10 0x7fba8c19fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:41.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.096+0000 7fba8affd640 1 --2- 172.21.15.67:0/2431875826 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba8c102a80 0x7fba8c19a380 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00d950 tx=0x7fba7c00dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:41.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.096+0000 7fba90855640 1 -- 172.21.15.67:0/2431875826 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7c017070 con 0x7fba8c102a80 2024-09-21T06:35:41.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.096+0000 7fba91857640 1 -- 172.21.15.67:0/2431875826 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba8c1a0520 con 0x7fba8c102a80 2024-09-21T06:35:41.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.096+0000 7fba90855640 1 -- 172.21.15.67:0/2431875826 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba7c00de80 con 0x7fba8c102a80 2024-09-21T06:35:41.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.096+0000 7fba91857640 1 -- 172.21.15.67:0/2431875826 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba8c1a6d40 con 0x7fba8c102a80 2024-09-21T06:35:41.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.096+0000 7fba90855640 1 -- 172.21.15.67:0/2431875826 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7c00f620 con 0x7fba8c102a80 2024-09-21T06:35:41.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.097+0000 7fba91857640 1 -- 172.21.15.67:0/2431875826 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fba8c063b10 con 0x7fba8c102a80 2024-09-21T06:35:41.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.099+0000 7fba90855640 1 -- 172.21.15.67:0/2431875826 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fba7c009c30 con 0x7fba8c102a80 2024-09-21T06:35:41.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.100+0000 7fba90855640 1 --2- 172.21.15.67:0/2431875826 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fba740b6650 0x7fba740b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.100+0000 7fba8a7fc640 1 --2- 172.21.15.67:0/2431875826 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fba740b6650 0x7fba740b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.100+0000 7fba90855640 1 -- 172.21.15.67:0/2431875826 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7fba7c0e7930 con 0x7fba8c102a80 2024-09-21T06:35:41.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.100+0000 7fba90855640 1 --2- 172.21.15.67:0/2431875826 >> [v2:172.21.15.100:6824/2819057142,v1:172.21.15.100:6825/2819057142] conn(0x7fba740c1760 0x7fba740c3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.100+0000 7fba90855640 1 -- 172.21.15.67:0/2431875826 --> [v2:172.21.15.100:6824/2819057142,v1:172.21.15.100:6825/2819057142] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fba740c4320 con 0x7fba740c1760 2024-09-21T06:35:41.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.100+0000 7fba90855640 1 -- 172.21.15.67:0/2431875826 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fba7c0e7d80 con 0x7fba8c102a80 2024-09-21T06:35:41.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.100+0000 7fba8a7fc640 1 --2- 172.21.15.67:0/2431875826 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fba740b6650 0x7fba740b8b60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fba8c102640 tx=0x7fba8000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:41.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.101+0000 7fba8b7fe640 1 --2- 172.21.15.67:0/2431875826 >> [v2:172.21.15.100:6824/2819057142,v1:172.21.15.100:6825/2819057142] conn(0x7fba740c1760 0x7fba740c3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.101+0000 7fba8b7fe640 1 --2- 172.21.15.67:0/2431875826 >> [v2:172.21.15.100:6824/2819057142,v1:172.21.15.100:6825/2819057142] conn(0x7fba740c1760 0x7fba740c3c50 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:41.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.102+0000 7fba90855640 1 -- 172.21.15.67:0/2431875826 <== osd.10 v2:172.21.15.100:6824/2819057142 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fba740c4320 con 0x7fba740c1760 2024-09-21T06:35:41.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.116+0000 7fba91857640 1 -- 172.21.15.67:0/2431875826 --> [v2:172.21.15.100:6824/2819057142,v1:172.21.15.100:6825/2819057142] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fba8c063f40 con 0x7fba740c1760 2024-09-21T06:35:41.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.116+0000 7fba90855640 1 -- 172.21.15.67:0/2431875826 <== osd.10 v2:172.21.15.100:6824/2819057142 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fba8c063f40 con 0x7fba740c1760 2024-09-21T06:35:41.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.117+0000 7fba91857640 1 -- 172.21.15.67:0/2431875826 >> [v2:172.21.15.100:6824/2819057142,v1:172.21.15.100:6825/2819057142] conn(0x7fba740c1760 msgr2=0x7fba740c3c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.117+0000 7fba91857640 1 --2- 172.21.15.67:0/2431875826 >> [v2:172.21.15.100:6824/2819057142,v1:172.21.15.100:6825/2819057142] conn(0x7fba740c1760 0x7fba740c3c50 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.117+0000 7fba91857640 1 -- 172.21.15.67:0/2431875826 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fba740b6650 msgr2=0x7fba740b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.117+0000 7fba91857640 1 --2- 172.21.15.67:0/2431875826 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fba740b6650 0x7fba740b8b60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fba8c102640 tx=0x7fba8000a2d0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.117+0000 7fba91857640 1 -- 172.21.15.67:0/2431875826 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba8c102a80 msgr2=0x7fba8c19a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.117+0000 7fba91857640 1 --2- 172.21.15.67:0/2431875826 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba8c102a80 0x7fba8c19a380 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00d950 tx=0x7fba7c00dd50 comp rx=0 tx=0).stop 2024-09-21T06:35:41.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.117+0000 7fba8b7fe640 1 -- 172.21.15.67:0/2431875826 reap_dead start 2024-09-21T06:35:41.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.117+0000 7fba91857640 1 -- 172.21.15.67:0/2431875826 shutdown_connections 2024-09-21T06:35:41.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.117+0000 7fba91857640 1 -- 172.21.15.67:0/2431875826 >> 172.21.15.67:0/2431875826 conn(0x7fba8c079740 msgr2=0x7fba8c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:41.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.117+0000 7fba91857640 1 -- 172.21.15.67:0/2431875826 shutdown_connections 2024-09-21T06:35:41.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.117+0000 7fba91857640 1 -- 172.21.15.67:0/2431875826 wait complete. 2024-09-21T06:35:41.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.299+0000 7f1d36cb9640 1 -- 172.21.15.67:0/3725116025 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d30105270 msgr2=0x7f1d3010d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.299+0000 7f1d36cb9640 1 --2- 172.21.15.67:0/3725116025 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d30105270 0x7f1d3010d8b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1d20009ca0 tx=0x7f1d2002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.299+0000 7f1d36cb9640 1 -- 172.21.15.67:0/3725116025 shutdown_connections 2024-09-21T06:35:41.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.299+0000 7f1d36cb9640 1 --2- 172.21.15.67:0/3725116025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d3010ddf0 0x7f1d30110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.299+0000 7f1d36cb9640 1 --2- 172.21.15.67:0/3725116025 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d30105270 0x7f1d3010d8b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.299+0000 7f1d36cb9640 1 --2- 172.21.15.67:0/3725116025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d30104840 0x7f1d30104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.299+0000 7f1d36cb9640 1 -- 172.21.15.67:0/3725116025 >> 172.21.15.67:0/3725116025 conn(0x7f1d30079730 msgr2=0x7f1d30079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:41.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.299+0000 7f1d36cb9640 1 -- 172.21.15.67:0/3725116025 shutdown_connections 2024-09-21T06:35:41.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.299+0000 7f1d36cb9640 1 -- 172.21.15.67:0/3725116025 wait complete. 2024-09-21T06:35:41.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.300+0000 7f1d36cb9640 1 Processor -- start 2024-09-21T06:35:41.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.300+0000 7f1d36cb9640 1 -- start start 2024-09-21T06:35:41.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.301+0000 7f1d36cb9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d30104840 0x7f1d3019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.301+0000 7f1d36cb9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d30105270 0x7f1d3019a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.301+0000 7f1d36cb9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d3010ddf0 0x7f1d3019fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.301+0000 7f1d36cb9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1d30113970 con 0x7f1d3010ddf0 2024-09-21T06:35:41.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.301+0000 7f1d36cb9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1d301137f0 con 0x7f1d30105270 2024-09-21T06:35:41.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.301+0000 7f1d36cb9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1d30113af0 con 0x7f1d30104840 2024-09-21T06:35:41.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.301+0000 7f1d35256640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d3010ddf0 0x7f1d3019fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.301+0000 7f1d35256640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d3010ddf0 0x7f1d3019fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33232/0 (socket says 172.21.15.67:33232) 2024-09-21T06:35:41.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.301+0000 7f1d35256640 1 -- 172.21.15.67:0/2088101237 learned_addr learned my addr 172.21.15.67:0/2088101237 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:41.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.301+0000 7f1d34a55640 1 --2- 172.21.15.67:0/2088101237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d30104840 0x7f1d3019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.301+0000 7f1d2ffff640 1 --2- 172.21.15.67:0/2088101237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d30105270 0x7f1d3019a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.301+0000 7f1d35256640 1 -- 172.21.15.67:0/2088101237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d30104840 msgr2=0x7f1d3019a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.301+0000 7f1d35256640 1 --2- 172.21.15.67:0/2088101237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d30104840 0x7f1d3019a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.301+0000 7f1d35256640 1 -- 172.21.15.67:0/2088101237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d30105270 msgr2=0x7f1d3019a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.301+0000 7f1d35256640 1 --2- 172.21.15.67:0/2088101237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d30105270 0x7f1d3019a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.301+0000 7f1d35256640 1 -- 172.21.15.67:0/2088101237 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d301a02f0 con 0x7f1d3010ddf0 2024-09-21T06:35:41.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.301+0000 7f1d2ffff640 1 --2- 172.21.15.67:0/2088101237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d30105270 0x7f1d3019a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:41.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.301+0000 7f1d35256640 1 --2- 172.21.15.67:0/2088101237 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d3010ddf0 0x7f1d3019fbc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800d750 tx=0x7f1d1800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:41.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.301+0000 7f1d2dffb640 1 -- 172.21.15.67:0/2088101237 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d18017070 con 0x7f1d3010ddf0 2024-09-21T06:35:41.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.302+0000 7f1d36cb9640 1 -- 172.21.15.67:0/2088101237 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d301a05e0 con 0x7f1d3010ddf0 2024-09-21T06:35:41.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.302+0000 7f1d2dffb640 1 -- 172.21.15.67:0/2088101237 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d180054b0 con 0x7f1d3010ddf0 2024-09-21T06:35:41.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.302+0000 7f1d2dffb640 1 -- 172.21.15.67:0/2088101237 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d18004b90 con 0x7f1d3010ddf0 2024-09-21T06:35:41.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.302+0000 7f1d36cb9640 1 -- 172.21.15.67:0/2088101237 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d301a6d40 con 0x7f1d3010ddf0 2024-09-21T06:35:41.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.303+0000 7f1d36cb9640 1 -- 172.21.15.67:0/2088101237 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1d30063b50 con 0x7f1d3010ddf0 2024-09-21T06:35:41.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.304+0000 7f1d2dffb640 1 -- 172.21.15.67:0/2088101237 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f1d18004d30 con 0x7f1d3010ddf0 2024-09-21T06:35:41.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.305+0000 7f1d2dffb640 1 --2- 172.21.15.67:0/2088101237 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f1d1c0b65a0 0x7f1d1c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.305+0000 7f1d34a55640 1 --2- 172.21.15.67:0/2088101237 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f1d1c0b65a0 0x7f1d1c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.305+0000 7f1d2dffb640 1 -- 172.21.15.67:0/2088101237 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f1d180e7690 con 0x7f1d3010ddf0 2024-09-21T06:35:41.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.305+0000 7f1d2dffb640 1 --2- 172.21.15.67:0/2088101237 >> [v2:172.21.15.67:6818/3229936280,v1:172.21.15.67:6819/3229936280] conn(0x7f1d1c0c1630 0x7f1d1c0c3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.305+0000 7f1d2dffb640 1 -- 172.21.15.67:0/2088101237 --> [v2:172.21.15.67:6818/3229936280,v1:172.21.15.67:6819/3229936280] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1d18010e50 con 0x7f1d1c0c1630 2024-09-21T06:35:41.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.305+0000 7f1d2dffb640 1 -- 172.21.15.67:0/2088101237 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f1d180e7ae0 con 0x7f1d3010ddf0 2024-09-21T06:35:41.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.306+0000 7f1d2ffff640 1 --2- 172.21.15.67:0/2088101237 >> [v2:172.21.15.67:6818/3229936280,v1:172.21.15.67:6819/3229936280] conn(0x7f1d1c0c1630 0x7f1d1c0c3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.306+0000 7f1d34a55640 1 --2- 172.21.15.67:0/2088101237 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f1d1c0b65a0 0x7f1d1c0b8ab0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1d28005d00 tx=0x7f1d28005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:41.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.306+0000 7f1d2ffff640 1 --2- 172.21.15.67:0/2088101237 >> [v2:172.21.15.67:6818/3229936280,v1:172.21.15.67:6819/3229936280] conn(0x7f1d1c0c1630 0x7f1d1c0c3b20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:41.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.307+0000 7f1d2dffb640 1 -- 172.21.15.67:0/2088101237 <== osd.6 v2:172.21.15.67:6818/3229936280 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1d18010e50 con 0x7f1d1c0c1630 2024-09-21T06:35:41.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.321+0000 7f1d36cb9640 1 -- 172.21.15.67:0/2088101237 --> [v2:172.21.15.67:6818/3229936280,v1:172.21.15.67:6819/3229936280] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1d30063fd0 con 0x7f1d1c0c1630 2024-09-21T06:35:41.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.322+0000 7f1d2dffb640 1 -- 172.21.15.67:0/2088101237 <== osd.6 v2:172.21.15.67:6818/3229936280 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1d30063fd0 con 0x7f1d1c0c1630 2024-09-21T06:35:41.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.322+0000 7f1d36cb9640 1 -- 172.21.15.67:0/2088101237 >> [v2:172.21.15.67:6818/3229936280,v1:172.21.15.67:6819/3229936280] conn(0x7f1d1c0c1630 msgr2=0x7f1d1c0c3b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.322+0000 7f1d36cb9640 1 --2- 172.21.15.67:0/2088101237 >> [v2:172.21.15.67:6818/3229936280,v1:172.21.15.67:6819/3229936280] conn(0x7f1d1c0c1630 0x7f1d1c0c3b20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.322+0000 7f1d36cb9640 1 -- 172.21.15.67:0/2088101237 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f1d1c0b65a0 msgr2=0x7f1d1c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.322+0000 7f1d36cb9640 1 --2- 172.21.15.67:0/2088101237 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f1d1c0b65a0 0x7f1d1c0b8ab0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1d28005d00 tx=0x7f1d28005c50 comp rx=0 tx=0).stop 2024-09-21T06:35:41.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.322+0000 7f1d36cb9640 1 -- 172.21.15.67:0/2088101237 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d3010ddf0 msgr2=0x7f1d3019fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.322+0000 7f1d36cb9640 1 --2- 172.21.15.67:0/2088101237 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d3010ddf0 0x7f1d3019fbc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800d750 tx=0x7f1d1800dc10 comp rx=0 tx=0).stop 2024-09-21T06:35:41.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.322+0000 7f1d35256640 1 -- 172.21.15.67:0/2088101237 reap_dead start 2024-09-21T06:35:41.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.322+0000 7f1d36cb9640 1 -- 172.21.15.67:0/2088101237 shutdown_connections 2024-09-21T06:35:41.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.322+0000 7f1d36cb9640 1 -- 172.21.15.67:0/2088101237 >> 172.21.15.67:0/2088101237 conn(0x7f1d30079730 msgr2=0x7f1d30100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:41.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.322+0000 7f1d36cb9640 1 -- 172.21.15.67:0/2088101237 shutdown_connections 2024-09-21T06:35:41.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.323+0000 7f1d36cb9640 1 -- 172.21.15.67:0/2088101237 wait complete. 2024-09-21T06:35:41.678 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:41 smithi067 ceph-mon[31232]: pgmap v331: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 1 op/s 2024-09-21T06:35:41.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:41 smithi100 ceph-mon[33401]: pgmap v331: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 1 op/s 2024-09-21T06:35:41.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:41 smithi203 ceph-mon[33735]: pgmap v331: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 1 op/s 2024-09-21T06:35:41.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.981+0000 7fbebb896640 1 -- 172.21.15.67:0/1041854892 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbeb4103bc0 msgr2=0x7fbeb4103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.981+0000 7fbebb896640 1 --2- 172.21.15.67:0/1041854892 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbeb4103bc0 0x7fbeb4103fd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0009900 tx=0x7fbeb002a410 comp rx=0 tx=0).stop 2024-09-21T06:35:41.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.981+0000 7fbebb896640 1 -- 172.21.15.67:0/1041854892 shutdown_connections 2024-09-21T06:35:41.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.981+0000 7fbebb896640 1 --2- 172.21.15.67:0/1041854892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbeb4109e00 0x7fbeb4110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.981+0000 7fbebb896640 1 --2- 172.21.15.67:0/1041854892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbeb41094b0 0x7fbeb41098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.981+0000 7fbebb896640 1 --2- 172.21.15.67:0/1041854892 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbeb4103bc0 0x7fbeb4103fd0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.981+0000 7fbebb896640 1 -- 172.21.15.67:0/1041854892 >> 172.21.15.67:0/1041854892 conn(0x7fbeb407b910 msgr2=0x7fbeb407bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:41.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.981+0000 7fbebb896640 1 -- 172.21.15.67:0/1041854892 shutdown_connections 2024-09-21T06:35:41.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.981+0000 7fbebb896640 1 -- 172.21.15.67:0/1041854892 wait complete. 2024-09-21T06:35:41.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.982+0000 7fbebb896640 1 Processor -- start 2024-09-21T06:35:41.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.982+0000 7fbebb896640 1 -- start start 2024-09-21T06:35:41.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.982+0000 7fbebb896640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbeb4103bc0 0x7fbeb419c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.982+0000 7fbebb896640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbeb41094b0 0x7fbeb419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.982+0000 7fbebb896640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbeb4109e00 0x7fbeb41a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.982+0000 7fbebb896640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb41139b0 con 0x7fbeb4109e00 2024-09-21T06:35:41.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.982+0000 7fbebb896640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb4113830 con 0x7fbeb41094b0 2024-09-21T06:35:41.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.982+0000 7fbebb896640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb4113b30 con 0x7fbeb4103bc0 2024-09-21T06:35:41.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.983+0000 7fbeb9e33640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbeb4109e00 0x7fbeb41a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.983+0000 7fbeb9e33640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbeb4109e00 0x7fbeb41a1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33250/0 (socket says 172.21.15.67:33250) 2024-09-21T06:35:41.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.983+0000 7fbeb9e33640 1 -- 172.21.15.67:0/1296404453 learned_addr learned my addr 172.21.15.67:0/1296404453 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:41.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.983+0000 7fbeb9632640 1 --2- 172.21.15.67:0/1296404453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbeb4103bc0 0x7fbeb419c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.983+0000 7fbeb8e31640 1 --2- 172.21.15.67:0/1296404453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbeb41094b0 0x7fbeb419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.983+0000 7fbeb9e33640 1 -- 172.21.15.67:0/1296404453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbeb4103bc0 msgr2=0x7fbeb419c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.983+0000 7fbeb9e33640 1 --2- 172.21.15.67:0/1296404453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbeb4103bc0 0x7fbeb419c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.983+0000 7fbeb9e33640 1 -- 172.21.15.67:0/1296404453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbeb41094b0 msgr2=0x7fbeb419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:41.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.983+0000 7fbeb9e33640 1 --2- 172.21.15.67:0/1296404453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbeb41094b0 0x7fbeb419cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:41.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.983+0000 7fbeb9e33640 1 -- 172.21.15.67:0/1296404453 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb41a2510 con 0x7fbeb4109e00 2024-09-21T06:35:41.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.983+0000 7fbeb8e31640 1 --2- 172.21.15.67:0/1296404453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbeb41094b0 0x7fbeb419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:41.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.983+0000 7fbeb9e33640 1 --2- 172.21.15.67:0/1296404453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbeb4109e00 0x7fbeb41a1ea0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbeac00cce0 tx=0x7fbeac00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:41.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.983+0000 7fbeaa7fc640 1 -- 172.21.15.67:0/1296404453 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac00f450 con 0x7fbeb4109e00 2024-09-21T06:35:41.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.983+0000 7fbebb896640 1 -- 172.21.15.67:0/1296404453 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb41a2800 con 0x7fbeb4109e00 2024-09-21T06:35:41.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.983+0000 7fbeaa7fc640 1 -- 172.21.15.67:0/1296404453 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbeac00fa80 con 0x7fbeb4109e00 2024-09-21T06:35:41.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.983+0000 7fbeaa7fc640 1 -- 172.21.15.67:0/1296404453 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac018ad0 con 0x7fbeb4109e00 2024-09-21T06:35:41.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.983+0000 7fbebb896640 1 -- 172.21.15.67:0/1296404453 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb41a9020 con 0x7fbeb4109e00 2024-09-21T06:35:41.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.984+0000 7fbebb896640 1 -- 172.21.15.67:0/1296404453 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbeb4107e60 con 0x7fbeb4109e00 2024-09-21T06:35:41.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.986+0000 7fbeaa7fc640 1 -- 172.21.15.67:0/1296404453 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fbeac00f5f0 con 0x7fbeb4109e00 2024-09-21T06:35:41.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.987+0000 7fbeaa7fc640 1 --2- 172.21.15.67:0/1296404453 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fbea00b65d0 0x7fbea00b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.987+0000 7fbeb9632640 1 --2- 172.21.15.67:0/1296404453 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fbea00b65d0 0x7fbea00b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.987+0000 7fbeaa7fc640 1 -- 172.21.15.67:0/1296404453 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7fbeac0e8260 con 0x7fbeb4109e00 2024-09-21T06:35:41.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.987+0000 7fbeaa7fc640 1 --2- 172.21.15.67:0/1296404453 >> [v2:172.21.15.67:6810/4007517951,v1:172.21.15.67:6811/4007517951] conn(0x7fbea00c16e0 0x7fbea00c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:41.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.987+0000 7fbeaa7fc640 1 -- 172.21.15.67:0/1296404453 --> [v2:172.21.15.67:6810/4007517951,v1:172.21.15.67:6811/4007517951] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbeac012ec0 con 0x7fbea00c16e0 2024-09-21T06:35:41.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.987+0000 7fbeaa7fc640 1 -- 172.21.15.67:0/1296404453 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fbeac0e86b0 con 0x7fbeb4109e00 2024-09-21T06:35:41.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.988+0000 7fbeb8e31640 1 --2- 172.21.15.67:0/1296404453 >> [v2:172.21.15.67:6810/4007517951,v1:172.21.15.67:6811/4007517951] conn(0x7fbea00c16e0 0x7fbea00c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:41.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.988+0000 7fbeb9632640 1 --2- 172.21.15.67:0/1296404453 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fbea00b65d0 0x7fbea00b8ae0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbeb00057c0 tx=0x7fbeb00058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:41.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.988+0000 7fbeb8e31640 1 --2- 172.21.15.67:0/1296404453 >> [v2:172.21.15.67:6810/4007517951,v1:172.21.15.67:6811/4007517951] conn(0x7fbea00c16e0 0x7fbea00c3bd0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:41.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:41.989+0000 7fbeaa7fc640 1 -- 172.21.15.67:0/1296404453 <== osd.3 v2:172.21.15.67:6810/4007517951 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbeac012ec0 con 0x7fbea00c16e0 2024-09-21T06:35:42.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:42.003+0000 7fbebb896640 1 -- 172.21.15.67:0/1296404453 --> [v2:172.21.15.67:6810/4007517951,v1:172.21.15.67:6811/4007517951] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbeb4104df0 con 0x7fbea00c16e0 2024-09-21T06:35:42.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:42.003+0000 7fbeaa7fc640 1 -- 172.21.15.67:0/1296404453 <== osd.3 v2:172.21.15.67:6810/4007517951 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbeb4104df0 con 0x7fbea00c16e0 2024-09-21T06:35:42.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:42.003+0000 7fbebb896640 1 -- 172.21.15.67:0/1296404453 >> [v2:172.21.15.67:6810/4007517951,v1:172.21.15.67:6811/4007517951] conn(0x7fbea00c16e0 msgr2=0x7fbea00c3bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:42.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:42.003+0000 7fbebb896640 1 --2- 172.21.15.67:0/1296404453 >> [v2:172.21.15.67:6810/4007517951,v1:172.21.15.67:6811/4007517951] conn(0x7fbea00c16e0 0x7fbea00c3bd0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:42.003+0000 7fbebb896640 1 -- 172.21.15.67:0/1296404453 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fbea00b65d0 msgr2=0x7fbea00b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:42.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:42.003+0000 7fbebb896640 1 --2- 172.21.15.67:0/1296404453 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fbea00b65d0 0x7fbea00b8ae0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbeb00057c0 tx=0x7fbeb00058d0 comp rx=0 tx=0).stop 2024-09-21T06:35:42.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:42.003+0000 7fbebb896640 1 -- 172.21.15.67:0/1296404453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbeb4109e00 msgr2=0x7fbeb41a1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:42.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:42.003+0000 7fbebb896640 1 --2- 172.21.15.67:0/1296404453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbeb4109e00 0x7fbeb41a1ea0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbeac00cce0 tx=0x7fbeac00a620 comp rx=0 tx=0).stop 2024-09-21T06:35:42.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:42.003+0000 7fbeb9e33640 1 -- 172.21.15.67:0/1296404453 reap_dead start 2024-09-21T06:35:42.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:42.003+0000 7fbebb896640 1 -- 172.21.15.67:0/1296404453 shutdown_connections 2024-09-21T06:35:42.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:42.003+0000 7fbebb896640 1 -- 172.21.15.67:0/1296404453 >> 172.21.15.67:0/1296404453 conn(0x7fbeb407b910 msgr2=0x7fbeb4107400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:42.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:42.003+0000 7fbebb896640 1 -- 172.21.15.67:0/1296404453 shutdown_connections 2024-09-21T06:35:42.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:42.003+0000 7fbebb896640 1 -- 172.21.15.67:0/1296404453 wait complete. 2024-09-21T06:35:42.660 INFO:teuthology.orchestra.run.smithi067.stdout:150323855417 2024-09-21T06:35:42.660 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.5 2024-09-21T06:35:43.390 INFO:teuthology.orchestra.run.smithi067.stdout:197568495663 2024-09-21T06:35:43.390 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.7 2024-09-21T06:35:43.657 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:43 smithi067 ceph-mon[31232]: pgmap v332: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-21T06:35:43.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:43 smithi100 ceph-mon[33401]: pgmap v332: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-21T06:35:43.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:43 smithi203 ceph-mon[33735]: pgmap v332: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-21T06:35:44.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.225+0000 7fd308e53640 1 -- 172.21.15.67:0/4075839263 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd304105380 msgr2=0x7fd30410d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:44.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.225+0000 7fd308e53640 1 --2- 172.21.15.67:0/4075839263 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd304105380 0x7fd30410d8d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8009900 tx=0x7fd2f802a410 comp rx=0 tx=0).stop 2024-09-21T06:35:44.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.225+0000 7fd308e53640 1 -- 172.21.15.67:0/4075839263 shutdown_connections 2024-09-21T06:35:44.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.225+0000 7fd308e53640 1 --2- 172.21.15.67:0/4075839263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd30410de60 0x7fd304110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.225+0000 7fd308e53640 1 --2- 172.21.15.67:0/4075839263 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd304105380 0x7fd30410d8d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.225+0000 7fd308e53640 1 --2- 172.21.15.67:0/4075839263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd304104a30 0x7fd304104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.225+0000 7fd308e53640 1 -- 172.21.15.67:0/4075839263 >> 172.21.15.67:0/4075839263 conn(0x7fd3040797b0 msgr2=0x7fd304079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:44.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.225+0000 7fd308e53640 1 -- 172.21.15.67:0/4075839263 shutdown_connections 2024-09-21T06:35:44.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.225+0000 7fd308e53640 1 -- 172.21.15.67:0/4075839263 wait complete. 2024-09-21T06:35:44.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.226+0000 7fd308e53640 1 Processor -- start 2024-09-21T06:35:44.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.226+0000 7fd308e53640 1 -- start start 2024-09-21T06:35:44.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.226+0000 7fd308e53640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd304104a30 0x7fd30419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.226+0000 7fd308e53640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd304105380 0x7fd30419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.226+0000 7fd308e53640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd30410de60 0x7fd30419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.226+0000 7fd308e53640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd304113990 con 0x7fd304104a30 2024-09-21T06:35:44.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.226+0000 7fd308e53640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd304113810 con 0x7fd304105380 2024-09-21T06:35:44.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.226+0000 7fd308e53640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd304113b10 con 0x7fd30410de60 2024-09-21T06:35:44.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.226+0000 7fd30259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd304104a30 0x7fd30419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.226+0000 7fd30259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd304104a30 0x7fd30419a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33268/0 (socket says 172.21.15.67:33268) 2024-09-21T06:35:44.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.226+0000 7fd30259c640 1 -- 172.21.15.67:0/4061653889 learned_addr learned my addr 172.21.15.67:0/4061653889 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:44.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.226+0000 7fd301d9b640 1 --2- 172.21.15.67:0/4061653889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd304105380 0x7fd30419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.226+0000 7fd302d9d640 1 --2- 172.21.15.67:0/4061653889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd30410de60 0x7fd30419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.227+0000 7fd30259c640 1 -- 172.21.15.67:0/4061653889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd30410de60 msgr2=0x7fd30419fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:44.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.227+0000 7fd30259c640 1 --2- 172.21.15.67:0/4061653889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd30410de60 0x7fd30419fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.227+0000 7fd30259c640 1 -- 172.21.15.67:0/4061653889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd304105380 msgr2=0x7fd30419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:44.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.227+0000 7fd30259c640 1 --2- 172.21.15.67:0/4061653889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd304105380 0x7fd30419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.227+0000 7fd30259c640 1 -- 172.21.15.67:0/4061653889 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3041a0390 con 0x7fd304104a30 2024-09-21T06:35:44.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.227+0000 7fd301d9b640 1 --2- 172.21.15.67:0/4061653889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd304105380 0x7fd30419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:44.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.227+0000 7fd30259c640 1 --2- 172.21.15.67:0/4061653889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd304104a30 0x7fd30419a4e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd2f400b4a0 tx=0x7fd2f400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:44.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.227+0000 7fd2f37fe640 1 -- 172.21.15.67:0/4061653889 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f4014070 con 0x7fd304104a30 2024-09-21T06:35:44.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.227+0000 7fd2f37fe640 1 -- 172.21.15.67:0/4061653889 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd2f4010aa0 con 0x7fd304104a30 2024-09-21T06:35:44.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.227+0000 7fd308e53640 1 -- 172.21.15.67:0/4061653889 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3041a0680 con 0x7fd304104a30 2024-09-21T06:35:44.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.227+0000 7fd2f37fe640 1 -- 172.21.15.67:0/4061653889 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f4017b70 con 0x7fd304104a30 2024-09-21T06:35:44.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.227+0000 7fd308e53640 1 -- 172.21.15.67:0/4061653889 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3041a6ea0 con 0x7fd304104a30 2024-09-21T06:35:44.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.228+0000 7fd308e53640 1 -- 172.21.15.67:0/4061653889 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd304063b50 con 0x7fd304104a30 2024-09-21T06:35:44.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.230+0000 7fd2f37fe640 1 -- 172.21.15.67:0/4061653889 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fd2f4017070 con 0x7fd304104a30 2024-09-21T06:35:44.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.231+0000 7fd2f37fe640 1 --2- 172.21.15.67:0/4061653889 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd2ec0b6650 0x7fd2ec0b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.231+0000 7fd301d9b640 1 --2- 172.21.15.67:0/4061653889 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd2ec0b6650 0x7fd2ec0b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.231+0000 7fd2f37fe640 1 -- 172.21.15.67:0/4061653889 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7fd2f40e70a0 con 0x7fd304104a30 2024-09-21T06:35:44.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.232+0000 7fd2f37fe640 1 --2- 172.21.15.67:0/4061653889 >> [v2:172.21.15.203:6824/566502600,v1:172.21.15.203:6825/566502600] conn(0x7fd2ec0c1760 0x7fd2ec0c3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:44.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.232+0000 7fd2f37fe640 1 -- 172.21.15.67:0/4061653889 --> [v2:172.21.15.203:6824/566502600,v1:172.21.15.203:6825/566502600] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd2f400ee50 con 0x7fd2ec0c1760 2024-09-21T06:35:44.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.232+0000 7fd2f37fe640 1 -- 172.21.15.67:0/4061653889 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fd2f40e7490 con 0x7fd304104a30 2024-09-21T06:35:44.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.232+0000 7fd301d9b640 1 --2- 172.21.15.67:0/4061653889 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd2ec0b6650 0x7fd2ec0b8b60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd2f80057c0 tx=0x7fd2f8009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:44.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.232+0000 7fd302d9d640 1 --2- 172.21.15.67:0/4061653889 >> [v2:172.21.15.203:6824/566502600,v1:172.21.15.203:6825/566502600] conn(0x7fd2ec0c1760 0x7fd2ec0c3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:44.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.232+0000 7fd302d9d640 1 --2- 172.21.15.67:0/4061653889 >> [v2:172.21.15.203:6824/566502600,v1:172.21.15.203:6825/566502600] conn(0x7fd2ec0c1760 0x7fd2ec0c3c50 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:44.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.233+0000 7fd2f37fe640 1 -- 172.21.15.67:0/4061653889 <== osd.11 v2:172.21.15.203:6824/566502600 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd2f400ee50 con 0x7fd2ec0c1760 2024-09-21T06:35:44.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.247+0000 7fd308e53640 1 -- 172.21.15.67:0/4061653889 --> [v2:172.21.15.203:6824/566502600,v1:172.21.15.203:6825/566502600] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd304063f50 con 0x7fd2ec0c1760 2024-09-21T06:35:44.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.247+0000 7fd2f37fe640 1 -- 172.21.15.67:0/4061653889 <== osd.11 v2:172.21.15.203:6824/566502600 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd304063f50 con 0x7fd2ec0c1760 2024-09-21T06:35:44.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.248+0000 7fd308e53640 1 -- 172.21.15.67:0/4061653889 >> [v2:172.21.15.203:6824/566502600,v1:172.21.15.203:6825/566502600] conn(0x7fd2ec0c1760 msgr2=0x7fd2ec0c3c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:44.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.248+0000 7fd308e53640 1 --2- 172.21.15.67:0/4061653889 >> [v2:172.21.15.203:6824/566502600,v1:172.21.15.203:6825/566502600] conn(0x7fd2ec0c1760 0x7fd2ec0c3c50 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:44.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.248+0000 7fd308e53640 1 -- 172.21.15.67:0/4061653889 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd2ec0b6650 msgr2=0x7fd2ec0b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:44.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.248+0000 7fd308e53640 1 --2- 172.21.15.67:0/4061653889 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd2ec0b6650 0x7fd2ec0b8b60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd2f80057c0 tx=0x7fd2f8009680 comp rx=0 tx=0).stop 2024-09-21T06:35:44.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.248+0000 7fd308e53640 1 -- 172.21.15.67:0/4061653889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd304104a30 msgr2=0x7fd30419a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:44.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.248+0000 7fd308e53640 1 --2- 172.21.15.67:0/4061653889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd304104a30 0x7fd30419a4e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd2f400b4a0 tx=0x7fd2f400b960 comp rx=0 tx=0).stop 2024-09-21T06:35:44.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.248+0000 7fd302d9d640 1 -- 172.21.15.67:0/4061653889 reap_dead start 2024-09-21T06:35:44.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.248+0000 7fd308e53640 1 -- 172.21.15.67:0/4061653889 shutdown_connections 2024-09-21T06:35:44.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.248+0000 7fd308e53640 1 -- 172.21.15.67:0/4061653889 >> 172.21.15.67:0/4061653889 conn(0x7fd3040797b0 msgr2=0x7fd304100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:44.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.248+0000 7fd308e53640 1 -- 172.21.15.67:0/4061653889 shutdown_connections 2024-09-21T06:35:44.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:44.248+0000 7fd308e53640 1 -- 172.21.15.67:0/4061653889 wait complete. 2024-09-21T06:35:44.706 INFO:teuthology.orchestra.run.smithi067.stdout:270582939680 2024-09-21T06:35:44.706 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.10 2024-09-21T06:35:45.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.304+0000 7fcf9c876640 1 -- 172.21.15.67:0/3570672745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf98108d20 msgr2=0x7fcf9810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:45.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.304+0000 7fcf9c876640 1 --2- 172.21.15.67:0/3570672745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf98108d20 0x7fcf9810f550 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcf90009ca0 tx=0x7fcf9002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:35:45.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.304+0000 7fcf9c876640 1 -- 172.21.15.67:0/3570672745 shutdown_connections 2024-09-21T06:35:45.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.304+0000 7fcf9c876640 1 --2- 172.21.15.67:0/3570672745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf981083d0 0x7fcf981087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:45.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.304+0000 7fcf9c876640 1 --2- 172.21.15.67:0/3570672745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf98108d20 0x7fcf9810f550 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:45.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.304+0000 7fcf9c876640 1 --2- 172.21.15.67:0/3570672745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf98102ae0 0x7fcf98102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:45.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.304+0000 7fcf9c876640 1 -- 172.21.15.67:0/3570672745 >> 172.21.15.67:0/3570672745 conn(0x7fcf98079760 msgr2=0x7fcf98079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:45.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.304+0000 7fcf9c876640 1 -- 172.21.15.67:0/3570672745 shutdown_connections 2024-09-21T06:35:45.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.304+0000 7fcf9c876640 1 -- 172.21.15.67:0/3570672745 wait complete. 2024-09-21T06:35:45.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.305+0000 7fcf9c876640 1 Processor -- start 2024-09-21T06:35:45.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.305+0000 7fcf9c876640 1 -- start start 2024-09-21T06:35:45.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.306+0000 7fcf9c876640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf98102ae0 0x7fcf9819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:45.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.306+0000 7fcf9c876640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf981083d0 0x7fcf9819a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:45.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.306+0000 7fcf9c876640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf98108d20 0x7fcf9819fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:45.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.306+0000 7fcf9c876640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcf98113990 con 0x7fcf98108d20 2024-09-21T06:35:45.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.306+0000 7fcf9c876640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcf98113810 con 0x7fcf98102ae0 2024-09-21T06:35:45.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.306+0000 7fcf9c876640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcf98113b10 con 0x7fcf981083d0 2024-09-21T06:35:45.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.306+0000 7fcf96d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf98108d20 0x7fcf9819fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:45.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.306+0000 7fcf96d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf98108d20 0x7fcf9819fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33310/0 (socket says 172.21.15.67:33310) 2024-09-21T06:35:45.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.306+0000 7fcf96d9d640 1 -- 172.21.15.67:0/2850153852 learned_addr learned my addr 172.21.15.67:0/2850153852 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:45.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.306+0000 7fcf9659c640 1 --2- 172.21.15.67:0/2850153852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf98102ae0 0x7fcf9819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:45.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.306+0000 7fcf95d9b640 1 --2- 172.21.15.67:0/2850153852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf981083d0 0x7fcf9819a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:45.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.306+0000 7fcf96d9d640 1 -- 172.21.15.67:0/2850153852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf981083d0 msgr2=0x7fcf9819a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:45.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.306+0000 7fcf96d9d640 1 --2- 172.21.15.67:0/2850153852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf981083d0 0x7fcf9819a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:45.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.306+0000 7fcf96d9d640 1 -- 172.21.15.67:0/2850153852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf98102ae0 msgr2=0x7fcf9819a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:45.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.306+0000 7fcf96d9d640 1 --2- 172.21.15.67:0/2850153852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf98102ae0 0x7fcf9819a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:45.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.306+0000 7fcf96d9d640 1 -- 172.21.15.67:0/2850153852 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf981a02b0 con 0x7fcf98108d20 2024-09-21T06:35:45.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.306+0000 7fcf95d9b640 1 --2- 172.21.15.67:0/2850153852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf981083d0 0x7fcf9819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:45.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.307+0000 7fcf96d9d640 1 --2- 172.21.15.67:0/2850153852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf98108d20 0x7fcf9819fb80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcf90005e30 tx=0x7fcf900093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:45.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.307+0000 7fcf877fe640 1 -- 172.21.15.67:0/2850153852 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90032d10 con 0x7fcf98108d20 2024-09-21T06:35:45.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.307+0000 7fcf877fe640 1 -- 172.21.15.67:0/2850153852 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf90032eb0 con 0x7fcf98108d20 2024-09-21T06:35:45.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.307+0000 7fcf9c876640 1 -- 172.21.15.67:0/2850153852 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf981a0540 con 0x7fcf98108d20 2024-09-21T06:35:45.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.307+0000 7fcf877fe640 1 -- 172.21.15.67:0/2850153852 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90030b20 con 0x7fcf98108d20 2024-09-21T06:35:45.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.307+0000 7fcf9c876640 1 -- 172.21.15.67:0/2850153852 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf981a6d00 con 0x7fcf98108d20 2024-09-21T06:35:45.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.308+0000 7fcf9c876640 1 -- 172.21.15.67:0/2850153852 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcf98063b50 con 0x7fcf98108d20 2024-09-21T06:35:45.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.310+0000 7fcf877fe640 1 -- 172.21.15.67:0/2850153852 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fcf90030cc0 con 0x7fcf98108d20 2024-09-21T06:35:45.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.311+0000 7fcf877fe640 1 --2- 172.21.15.67:0/2850153852 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fcf800b65a0 0x7fcf800b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:45.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.311+0000 7fcf9659c640 1 --2- 172.21.15.67:0/2850153852 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fcf800b65a0 0x7fcf800b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:45.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.311+0000 7fcf877fe640 1 -- 172.21.15.67:0/2850153852 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7fcf90108c50 con 0x7fcf98108d20 2024-09-21T06:35:45.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.311+0000 7fcf877fe640 1 --2- 172.21.15.67:0/2850153852 >> [v2:172.21.15.100:6808/3773646501,v1:172.21.15.100:6809/3773646501] conn(0x7fcf800c16b0 0x7fcf800c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:45.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.311+0000 7fcf877fe640 1 -- 172.21.15.67:0/2850153852 --> [v2:172.21.15.100:6808/3773646501,v1:172.21.15.100:6809/3773646501] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcf800c4270 con 0x7fcf800c16b0 2024-09-21T06:35:45.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.311+0000 7fcf877fe640 1 -- 172.21.15.67:0/2850153852 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fcf901090a0 con 0x7fcf98108d20 2024-09-21T06:35:45.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.312+0000 7fcf9659c640 1 --2- 172.21.15.67:0/2850153852 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fcf800b65a0 0x7fcf800b8ab0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcf88005d00 tx=0x7fcf8800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:45.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.312+0000 7fcf95d9b640 1 --2- 172.21.15.67:0/2850153852 >> [v2:172.21.15.100:6808/3773646501,v1:172.21.15.100:6809/3773646501] conn(0x7fcf800c16b0 0x7fcf800c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:45.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.312+0000 7fcf95d9b640 1 --2- 172.21.15.67:0/2850153852 >> [v2:172.21.15.100:6808/3773646501,v1:172.21.15.100:6809/3773646501] conn(0x7fcf800c16b0 0x7fcf800c3ba0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:45.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.313+0000 7fcf877fe640 1 -- 172.21.15.67:0/2850153852 <== osd.4 v2:172.21.15.100:6808/3773646501 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcf800c4270 con 0x7fcf800c16b0 2024-09-21T06:35:45.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.327+0000 7fcf9c876640 1 -- 172.21.15.67:0/2850153852 --> [v2:172.21.15.100:6808/3773646501,v1:172.21.15.100:6809/3773646501] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcf98063fd0 con 0x7fcf800c16b0 2024-09-21T06:35:45.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.328+0000 7fcf877fe640 1 -- 172.21.15.67:0/2850153852 <== osd.4 v2:172.21.15.100:6808/3773646501 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcf98063fd0 con 0x7fcf800c16b0 2024-09-21T06:35:45.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.328+0000 7fcf9c876640 1 -- 172.21.15.67:0/2850153852 >> [v2:172.21.15.100:6808/3773646501,v1:172.21.15.100:6809/3773646501] conn(0x7fcf800c16b0 msgr2=0x7fcf800c3ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:45.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.328+0000 7fcf9c876640 1 --2- 172.21.15.67:0/2850153852 >> [v2:172.21.15.100:6808/3773646501,v1:172.21.15.100:6809/3773646501] conn(0x7fcf800c16b0 0x7fcf800c3ba0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:45.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.328+0000 7fcf9c876640 1 -- 172.21.15.67:0/2850153852 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fcf800b65a0 msgr2=0x7fcf800b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:45.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.328+0000 7fcf9c876640 1 --2- 172.21.15.67:0/2850153852 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fcf800b65a0 0x7fcf800b8ab0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcf88005d00 tx=0x7fcf8800b410 comp rx=0 tx=0).stop 2024-09-21T06:35:45.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.328+0000 7fcf9c876640 1 -- 172.21.15.67:0/2850153852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf98108d20 msgr2=0x7fcf9819fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:45.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.328+0000 7fcf9c876640 1 --2- 172.21.15.67:0/2850153852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf98108d20 0x7fcf9819fb80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcf90005e30 tx=0x7fcf900093d0 comp rx=0 tx=0).stop 2024-09-21T06:35:45.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.328+0000 7fcf96d9d640 1 -- 172.21.15.67:0/2850153852 reap_dead start 2024-09-21T06:35:45.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.328+0000 7fcf9c876640 1 -- 172.21.15.67:0/2850153852 shutdown_connections 2024-09-21T06:35:45.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.328+0000 7fcf9c876640 1 -- 172.21.15.67:0/2850153852 >> 172.21.15.67:0/2850153852 conn(0x7fcf98079760 msgr2=0x7fcf98104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:45.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.328+0000 7fcf9c876640 1 -- 172.21.15.67:0/2850153852 shutdown_connections 2024-09-21T06:35:45.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.329+0000 7fcf9c876640 1 -- 172.21.15.67:0/2850153852 wait complete. 2024-09-21T06:35:45.751 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:45 smithi067 ceph-mon[31232]: pgmap v333: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 937 B/s rd, 1 op/s 2024-09-21T06:35:45.778 INFO:teuthology.orchestra.run.smithi067.stdout:296352743452 2024-09-21T06:35:45.779 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.11 2024-09-21T06:35:45.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:45 smithi100 ceph-mon[33401]: pgmap v333: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 937 B/s rd, 1 op/s 2024-09-21T06:35:45.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:45 smithi203 ceph-mon[33735]: pgmap v333: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 937 B/s rd, 1 op/s 2024-09-21T06:35:46.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.954+0000 7f1378fee640 1 -- 172.21.15.67:0/1083095642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1374102a60 msgr2=0x7f1374102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:46.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.954+0000 7f1378fee640 1 --2- 172.21.15.67:0/1083095642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1374102a60 0x7f1374102e70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1364009900 tx=0x7f136402a410 comp rx=0 tx=0).stop 2024-09-21T06:35:46.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.954+0000 7f1378fee640 1 -- 172.21.15.67:0/1083095642 shutdown_connections 2024-09-21T06:35:46.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.954+0000 7f1378fee640 1 --2- 172.21.15.67:0/1083095642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1374108cf0 0x7f137410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:46.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.954+0000 7f1378fee640 1 --2- 172.21.15.67:0/1083095642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13741083a0 0x7f13741087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:46.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.954+0000 7f1378fee640 1 --2- 172.21.15.67:0/1083095642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1374102a60 0x7f1374102e70 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:46.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.954+0000 7f1378fee640 1 -- 172.21.15.67:0/1083095642 >> 172.21.15.67:0/1083095642 conn(0x7f1374079730 msgr2=0x7f1374079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:46.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.954+0000 7f1378fee640 1 -- 172.21.15.67:0/1083095642 shutdown_connections 2024-09-21T06:35:46.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.954+0000 7f1378fee640 1 -- 172.21.15.67:0/1083095642 wait complete. 2024-09-21T06:35:46.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.955+0000 7f1378fee640 1 Processor -- start 2024-09-21T06:35:46.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.955+0000 7f1378fee640 1 -- start start 2024-09-21T06:35:46.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.955+0000 7f1378fee640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1374102a60 0x7f137419a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:46.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.955+0000 7f1378fee640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13741083a0 0x7f137419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:46.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.955+0000 7f1378fee640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1374108cf0 0x7f137419fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:46.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.955+0000 7f1378fee640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1374112810 con 0x7f1374108cf0 2024-09-21T06:35:46.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.955+0000 7f1378fee640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1374112690 con 0x7f13741083a0 2024-09-21T06:35:46.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.955+0000 7f1378fee640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1374112990 con 0x7f1374102a60 2024-09-21T06:35:46.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.956+0000 7f1372d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1374108cf0 0x7f137419fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:46.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.956+0000 7f1372d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1374108cf0 0x7f137419fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33330/0 (socket says 172.21.15.67:33330) 2024-09-21T06:35:46.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.956+0000 7f1372d9d640 1 -- 172.21.15.67:0/3690616385 learned_addr learned my addr 172.21.15.67:0/3690616385 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:46.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.956+0000 7f1371d9b640 1 --2- 172.21.15.67:0/3690616385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13741083a0 0x7f137419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:46.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.956+0000 7f137259c640 1 --2- 172.21.15.67:0/3690616385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1374102a60 0x7f137419a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:46.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.956+0000 7f1372d9d640 1 -- 172.21.15.67:0/3690616385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1374102a60 msgr2=0x7f137419a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:46.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.956+0000 7f1372d9d640 1 --2- 172.21.15.67:0/3690616385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1374102a60 0x7f137419a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:46.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.956+0000 7f1372d9d640 1 -- 172.21.15.67:0/3690616385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13741083a0 msgr2=0x7f137419a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:46.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.956+0000 7f1372d9d640 1 --2- 172.21.15.67:0/3690616385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13741083a0 0x7f137419a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:46.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.956+0000 7f1372d9d640 1 -- 172.21.15.67:0/3690616385 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13741a0340 con 0x7f1374108cf0 2024-09-21T06:35:46.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.956+0000 7f1372d9d640 1 --2- 172.21.15.67:0/3690616385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1374108cf0 0x7f137419fc10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f136c00eaa0 tx=0x7f136c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:46.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.956+0000 7f13637fe640 1 -- 172.21.15.67:0/3690616385 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c00cdb0 con 0x7f1374108cf0 2024-09-21T06:35:46.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.956+0000 7f1378fee640 1 -- 172.21.15.67:0/3690616385 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13741a0630 con 0x7f1374108cf0 2024-09-21T06:35:46.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.956+0000 7f13637fe640 1 -- 172.21.15.67:0/3690616385 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f136c005020 con 0x7f1374108cf0 2024-09-21T06:35:46.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.956+0000 7f13637fe640 1 -- 172.21.15.67:0/3690616385 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c0116d0 con 0x7f1374108cf0 2024-09-21T06:35:46.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.957+0000 7f1378fee640 1 -- 172.21.15.67:0/3690616385 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13741a6d90 con 0x7f1374108cf0 2024-09-21T06:35:46.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.958+0000 7f1378fee640 1 -- 172.21.15.67:0/3690616385 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f13741014d0 con 0x7f1374108cf0 2024-09-21T06:35:46.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.959+0000 7f13637fe640 1 -- 172.21.15.67:0/3690616385 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f136c0051d0 con 0x7f1374108cf0 2024-09-21T06:35:46.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.960+0000 7f13637fe640 1 --2- 172.21.15.67:0/3690616385 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f135c0b65a0 0x7f135c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:46.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.960+0000 7f137259c640 1 --2- 172.21.15.67:0/3690616385 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f135c0b65a0 0x7f135c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:46.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.960+0000 7f13637fe640 1 -- 172.21.15.67:0/3690616385 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f136c0e7eb0 con 0x7f1374108cf0 2024-09-21T06:35:46.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.960+0000 7f13637fe640 1 --2- 172.21.15.67:0/3690616385 >> [v2:172.21.15.67:6826/2998307533,v1:172.21.15.67:6827/2998307533] conn(0x7f135c0c16b0 0x7f135c0c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:46.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.960+0000 7f13637fe640 1 -- 172.21.15.67:0/3690616385 --> [v2:172.21.15.67:6826/2998307533,v1:172.21.15.67:6827/2998307533] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f136c014ec0 con 0x7f135c0c16b0 2024-09-21T06:35:46.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.960+0000 7f13637fe640 1 -- 172.21.15.67:0/3690616385 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f136c0e8300 con 0x7f1374108cf0 2024-09-21T06:35:46.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.961+0000 7f1371d9b640 1 --2- 172.21.15.67:0/3690616385 >> [v2:172.21.15.67:6826/2998307533,v1:172.21.15.67:6827/2998307533] conn(0x7f135c0c16b0 0x7f135c0c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:46.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.961+0000 7f137259c640 1 --2- 172.21.15.67:0/3690616385 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f135c0b65a0 0x7f135c0b8ab0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1364000c00 tx=0x7f1364009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:46.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.961+0000 7f1371d9b640 1 --2- 172.21.15.67:0/3690616385 >> [v2:172.21.15.67:6826/2998307533,v1:172.21.15.67:6827/2998307533] conn(0x7f135c0c16b0 0x7f135c0c3ba0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:46.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.962+0000 7f13637fe640 1 -- 172.21.15.67:0/3690616385 <== osd.9 v2:172.21.15.67:6826/2998307533 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f136c014ec0 con 0x7f135c0c16b0 2024-09-21T06:35:46.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.976+0000 7f1378fee640 1 -- 172.21.15.67:0/3690616385 --> [v2:172.21.15.67:6826/2998307533,v1:172.21.15.67:6827/2998307533] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1374103ce0 con 0x7f135c0c16b0 2024-09-21T06:35:46.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.976+0000 7f13637fe640 1 -- 172.21.15.67:0/3690616385 <== osd.9 v2:172.21.15.67:6826/2998307533 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1374103ce0 con 0x7f135c0c16b0 2024-09-21T06:35:46.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.977+0000 7f1378fee640 1 -- 172.21.15.67:0/3690616385 >> [v2:172.21.15.67:6826/2998307533,v1:172.21.15.67:6827/2998307533] conn(0x7f135c0c16b0 msgr2=0x7f135c0c3ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:46.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.977+0000 7f1378fee640 1 --2- 172.21.15.67:0/3690616385 >> [v2:172.21.15.67:6826/2998307533,v1:172.21.15.67:6827/2998307533] conn(0x7f135c0c16b0 0x7f135c0c3ba0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:46.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.977+0000 7f1378fee640 1 -- 172.21.15.67:0/3690616385 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f135c0b65a0 msgr2=0x7f135c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:46.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.977+0000 7f1378fee640 1 --2- 172.21.15.67:0/3690616385 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f135c0b65a0 0x7f135c0b8ab0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1364000c00 tx=0x7f1364009680 comp rx=0 tx=0).stop 2024-09-21T06:35:46.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.977+0000 7f1378fee640 1 -- 172.21.15.67:0/3690616385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1374108cf0 msgr2=0x7f137419fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:46.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.977+0000 7f1378fee640 1 --2- 172.21.15.67:0/3690616385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1374108cf0 0x7f137419fc10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f136c00eaa0 tx=0x7f136c00ef60 comp rx=0 tx=0).stop 2024-09-21T06:35:46.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.977+0000 7f1372d9d640 1 -- 172.21.15.67:0/3690616385 reap_dead start 2024-09-21T06:35:46.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.977+0000 7f1378fee640 1 -- 172.21.15.67:0/3690616385 shutdown_connections 2024-09-21T06:35:46.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.977+0000 7f1378fee640 1 -- 172.21.15.67:0/3690616385 >> 172.21.15.67:0/3690616385 conn(0x7f1374079730 msgr2=0x7f13741067f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:46.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.977+0000 7f1378fee640 1 -- 172.21.15.67:0/3690616385 shutdown_connections 2024-09-21T06:35:46.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.977+0000 7f1378fee640 1 -- 172.21.15.67:0/3690616385 wait complete. 2024-09-21T06:35:46.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.095+0000 7fde7135c640 1 -- 172.21.15.67:0/2324873601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde6c10de60 msgr2=0x7fde6c110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:46.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.095+0000 7fde7135c640 1 --2- 172.21.15.67:0/2324873601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde6c10de60 0x7fde6c110330 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fde64009ca0 tx=0x7fde6402a8e0 comp rx=0 tx=0).stop 2024-09-21T06:35:46.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.096+0000 7fde7135c640 1 -- 172.21.15.67:0/2324873601 shutdown_connections 2024-09-21T06:35:46.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.096+0000 7fde7135c640 1 --2- 172.21.15.67:0/2324873601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde6c10de60 0x7fde6c110330 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:46.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.096+0000 7fde7135c640 1 --2- 172.21.15.67:0/2324873601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde6c105380 0x7fde6c10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:46.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.096+0000 7fde7135c640 1 --2- 172.21.15.67:0/2324873601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde6c104a30 0x7fde6c104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:46.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.096+0000 7fde7135c640 1 -- 172.21.15.67:0/2324873601 >> 172.21.15.67:0/2324873601 conn(0x7fde6c079760 msgr2=0x7fde6c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:46.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.096+0000 7fde7135c640 1 -- 172.21.15.67:0/2324873601 shutdown_connections 2024-09-21T06:35:46.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.096+0000 7fde7135c640 1 -- 172.21.15.67:0/2324873601 wait complete. 2024-09-21T06:35:46.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.096+0000 7fde7135c640 1 Processor -- start 2024-09-21T06:35:46.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.096+0000 7fde7135c640 1 -- start start 2024-09-21T06:35:46.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.097+0000 7fde7135c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde6c104a30 0x7fde6c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:46.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.097+0000 7fde7135c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde6c105380 0x7fde6c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:46.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.097+0000 7fde7135c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde6c10de60 0x7fde6c19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:46.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.097+0000 7fde7135c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fde6c1139b0 con 0x7fde6c105380 2024-09-21T06:35:46.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.097+0000 7fde7135c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fde6c113830 con 0x7fde6c10de60 2024-09-21T06:35:46.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.097+0000 7fde7135c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fde6c113b30 con 0x7fde6c104a30 2024-09-21T06:35:46.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.097+0000 7fde6a7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde6c105380 0x7fde6c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:46.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.097+0000 7fde6a7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde6c105380 0x7fde6c19aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33282/0 (socket says 172.21.15.67:33282) 2024-09-21T06:35:46.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.097+0000 7fde6a7fc640 1 -- 172.21.15.67:0/551145173 learned_addr learned my addr 172.21.15.67:0/551145173 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:46.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.097+0000 7fde6affd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde6c104a30 0x7fde6c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:46.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.097+0000 7fde6b7fe640 1 --2- 172.21.15.67:0/551145173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde6c10de60 0x7fde6c19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:46.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.097+0000 7fde6a7fc640 1 -- 172.21.15.67:0/551145173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde6c104a30 msgr2=0x7fde6c19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:46.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.097+0000 7fde6a7fc640 1 --2- 172.21.15.67:0/551145173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde6c104a30 0x7fde6c19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:46.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.097+0000 7fde6a7fc640 1 -- 172.21.15.67:0/551145173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde6c10de60 msgr2=0x7fde6c19fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:46.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.097+0000 7fde6a7fc640 1 --2- 172.21.15.67:0/551145173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde6c10de60 0x7fde6c19fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:46.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.097+0000 7fde6a7fc640 1 -- 172.21.15.67:0/551145173 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde6c1a03d0 con 0x7fde6c105380 2024-09-21T06:35:46.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.098+0000 7fde6a7fc640 1 --2- 172.21.15.67:0/551145173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde6c105380 0x7fde6c19aa00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fde54006fd0 tx=0x7fde5400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:46.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.098+0000 7fde63fff640 1 -- 172.21.15.67:0/551145173 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde54011830 con 0x7fde6c105380 2024-09-21T06:35:46.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.098+0000 7fde63fff640 1 -- 172.21.15.67:0/551145173 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde54016070 con 0x7fde6c105380 2024-09-21T06:35:46.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.098+0000 7fde7135c640 1 -- 172.21.15.67:0/551145173 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde6c1a06c0 con 0x7fde6c105380 2024-09-21T06:35:46.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.098+0000 7fde63fff640 1 -- 172.21.15.67:0/551145173 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde54010d30 con 0x7fde6c105380 2024-09-21T06:35:46.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.098+0000 7fde7135c640 1 -- 172.21.15.67:0/551145173 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde6c1a6e20 con 0x7fde6c105380 2024-09-21T06:35:46.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.099+0000 7fde7135c640 1 -- 172.21.15.67:0/551145173 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fde6c063b10 con 0x7fde6c105380 2024-09-21T06:35:46.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.101+0000 7fde63fff640 1 -- 172.21.15.67:0/551145173 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fde540119d0 con 0x7fde6c105380 2024-09-21T06:35:46.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.101+0000 7fde63fff640 1 --2- 172.21.15.67:0/551145173 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fde580b65a0 0x7fde580b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:46.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.102+0000 7fde6affd640 1 --2- 172.21.15.67:0/551145173 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fde580b65a0 0x7fde580b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:46.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.102+0000 7fde63fff640 1 -- 172.21.15.67:0/551145173 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7fde540e7770 con 0x7fde6c105380 2024-09-21T06:35:46.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.102+0000 7fde63fff640 1 --2- 172.21.15.67:0/551145173 >> [v2:172.21.15.203:6816/3452445943,v1:172.21.15.203:6817/3452445943] conn(0x7fde580c16b0 0x7fde580c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:46.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.102+0000 7fde63fff640 1 -- 172.21.15.67:0/551145173 --> [v2:172.21.15.203:6816/3452445943,v1:172.21.15.203:6817/3452445943] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fde54022e50 con 0x7fde580c16b0 2024-09-21T06:35:46.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.102+0000 7fde63fff640 1 -- 172.21.15.67:0/551145173 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fde540e7bc0 con 0x7fde6c105380 2024-09-21T06:35:46.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.102+0000 7fde6affd640 1 --2- 172.21.15.67:0/551145173 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fde580b65a0 0x7fde580b8ab0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fde5c005fd0 tx=0x7fde5c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:46.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.102+0000 7fde6b7fe640 1 --2- 172.21.15.67:0/551145173 >> [v2:172.21.15.203:6816/3452445943,v1:172.21.15.203:6817/3452445943] conn(0x7fde580c16b0 0x7fde580c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:46.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.103+0000 7fde6b7fe640 1 --2- 172.21.15.67:0/551145173 >> [v2:172.21.15.203:6816/3452445943,v1:172.21.15.203:6817/3452445943] conn(0x7fde580c16b0 0x7fde580c3ba0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:46.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.104+0000 7fde63fff640 1 -- 172.21.15.67:0/551145173 <== osd.8 v2:172.21.15.203:6816/3452445943 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fde54022e50 con 0x7fde580c16b0 2024-09-21T06:35:46.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.118+0000 7fde7135c640 1 -- 172.21.15.67:0/551145173 --> [v2:172.21.15.203:6816/3452445943,v1:172.21.15.203:6817/3452445943] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fde6c063f90 con 0x7fde580c16b0 2024-09-21T06:35:46.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.118+0000 7fde63fff640 1 -- 172.21.15.67:0/551145173 <== osd.8 v2:172.21.15.203:6816/3452445943 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fde6c063f90 con 0x7fde580c16b0 2024-09-21T06:35:46.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.118+0000 7fde7135c640 1 -- 172.21.15.67:0/551145173 >> [v2:172.21.15.203:6816/3452445943,v1:172.21.15.203:6817/3452445943] conn(0x7fde580c16b0 msgr2=0x7fde580c3ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:46.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.118+0000 7fde7135c640 1 --2- 172.21.15.67:0/551145173 >> [v2:172.21.15.203:6816/3452445943,v1:172.21.15.203:6817/3452445943] conn(0x7fde580c16b0 0x7fde580c3ba0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:46.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.118+0000 7fde7135c640 1 -- 172.21.15.67:0/551145173 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fde580b65a0 msgr2=0x7fde580b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:46.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.118+0000 7fde7135c640 1 --2- 172.21.15.67:0/551145173 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fde580b65a0 0x7fde580b8ab0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fde5c005fd0 tx=0x7fde5c00b410 comp rx=0 tx=0).stop 2024-09-21T06:35:46.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.119+0000 7fde7135c640 1 -- 172.21.15.67:0/551145173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde6c105380 msgr2=0x7fde6c19aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:46.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.119+0000 7fde7135c640 1 --2- 172.21.15.67:0/551145173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde6c105380 0x7fde6c19aa00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fde54006fd0 tx=0x7fde5400b8e0 comp rx=0 tx=0).stop 2024-09-21T06:35:46.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.119+0000 7fde6b7fe640 1 -- 172.21.15.67:0/551145173 reap_dead start 2024-09-21T06:35:46.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.119+0000 7fde7135c640 1 -- 172.21.15.67:0/551145173 shutdown_connections 2024-09-21T06:35:46.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.119+0000 7fde7135c640 1 -- 172.21.15.67:0/551145173 >> 172.21.15.67:0/551145173 conn(0x7fde6c079760 msgr2=0x7fde6c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:46.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.119+0000 7fde7135c640 1 -- 172.21.15.67:0/551145173 shutdown_connections 2024-09-21T06:35:46.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:45.119+0000 7fde7135c640 1 -- 172.21.15.67:0/551145173 wait complete. 2024-09-21T06:35:46.992 INFO:teuthology.orchestra.run.smithi067.stdout:249108103207 2024-09-21T06:35:46.992 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.9 2024-09-21T06:35:47.647 INFO:teuthology.orchestra.run.smithi067.stdout:77309411401 2024-09-21T06:35:47.647 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.2 2024-09-21T06:35:47.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:47 smithi203 ceph-mon[33735]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:35:47.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:47 smithi067 ceph-mon[31232]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:35:47.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:47 smithi100 ceph-mon[33401]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:35:48.081 INFO:teuthology.orchestra.run.smithi067.stdout:34359738451 2024-09-21T06:35:48.081 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.0 2024-09-21T06:35:48.558 INFO:teuthology.orchestra.run.smithi067.stdout:103079215172 2024-09-21T06:35:48.558 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.3 2024-09-21T06:35:48.968 INFO:teuthology.orchestra.run.smithi067.stdout:176093659189 2024-09-21T06:35:48.968 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.6 2024-09-21T06:35:49.416 INFO:teuthology.orchestra.run.smithi067.stdout:223338299436 2024-09-21T06:35:49.416 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.8 2024-09-21T06:35:49.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:49 smithi067 ceph-mon[31232]: pgmap v335: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-09-21T06:35:49.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:49 smithi100 ceph-mon[33401]: pgmap v335: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-09-21T06:35:49.846 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:49.848 INFO:teuthology.orchestra.run.smithi067.stdout:128849018944 2024-09-21T06:35:49.848 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.4 2024-09-21T06:35:49.873 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:49.923 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:49.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:49 smithi203 ceph-mon[33735]: pgmap v335: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-09-21T06:35:50.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.707+0000 7f732f829640 1 -- 172.21.15.67:0/2465278689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7328108cf0 msgr2=0x7f732810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:50.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.707+0000 7f732f829640 1 --2- 172.21.15.67:0/2465278689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7328108cf0 0x7f732810f520 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7320009ca0 tx=0x7f732002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:35:50.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.707+0000 7f732f829640 1 -- 172.21.15.67:0/2465278689 shutdown_connections 2024-09-21T06:35:50.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.707+0000 7f732f829640 1 --2- 172.21.15.67:0/2465278689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73281083a0 0x7f73281087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:50.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.707+0000 7f732f829640 1 --2- 172.21.15.67:0/2465278689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7328108cf0 0x7f732810f520 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:50.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.707+0000 7f732f829640 1 --2- 172.21.15.67:0/2465278689 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7328102a60 0x7f7328102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:50.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.707+0000 7f732f829640 1 -- 172.21.15.67:0/2465278689 >> 172.21.15.67:0/2465278689 conn(0x7f7328079730 msgr2=0x7f7328079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:50.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.707+0000 7f732f829640 1 -- 172.21.15.67:0/2465278689 shutdown_connections 2024-09-21T06:35:50.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.707+0000 7f732f829640 1 -- 172.21.15.67:0/2465278689 wait complete. 2024-09-21T06:35:50.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.708+0000 7f732f829640 1 Processor -- start 2024-09-21T06:35:50.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.708+0000 7f732f829640 1 -- start start 2024-09-21T06:35:50.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.708+0000 7f732f829640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7328102a60 0x7f732819b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:50.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.708+0000 7f732f829640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73281083a0 0x7f732819b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:50.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.708+0000 7f732f829640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7328108cf0 0x7f732819bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:50.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.708+0000 7f732f829640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7328112860 con 0x7f7328108cf0 2024-09-21T06:35:50.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.708+0000 7f732f829640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f73281126e0 con 0x7f73281083a0 2024-09-21T06:35:50.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.708+0000 7f732f829640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f73281129e0 con 0x7f7328102a60 2024-09-21T06:35:50.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.709+0000 7f732ddc6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7328108cf0 0x7f732819bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:50.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.709+0000 7f732ddc6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7328108cf0 0x7f732819bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44874/0 (socket says 172.21.15.67:44874) 2024-09-21T06:35:50.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.709+0000 7f732ddc6640 1 -- 172.21.15.67:0/1746241063 learned_addr learned my addr 172.21.15.67:0/1746241063 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:50.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.709+0000 7f732cdc4640 1 --2- 172.21.15.67:0/1746241063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73281083a0 0x7f732819b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:50.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.709+0000 7f732d5c5640 1 --2- 172.21.15.67:0/1746241063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7328102a60 0x7f732819b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:50.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.709+0000 7f732ddc6640 1 -- 172.21.15.67:0/1746241063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7328102a60 msgr2=0x7f732819b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:50.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.709+0000 7f732ddc6640 1 --2- 172.21.15.67:0/1746241063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7328102a60 0x7f732819b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:50.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.709+0000 7f732ddc6640 1 -- 172.21.15.67:0/1746241063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73281083a0 msgr2=0x7f732819b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:50.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.709+0000 7f732ddc6640 1 --2- 172.21.15.67:0/1746241063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73281083a0 0x7f732819b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:50.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.709+0000 7f732ddc6640 1 -- 172.21.15.67:0/1746241063 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73281a6440 con 0x7f7328108cf0 2024-09-21T06:35:50.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.709+0000 7f732d5c5640 1 --2- 172.21.15.67:0/1746241063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7328102a60 0x7f732819b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:50.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.709+0000 7f732ddc6640 1 --2- 172.21.15.67:0/1746241063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7328108cf0 0x7f732819bc30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7320031040 tx=0x7f7320032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:50.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.709+0000 7f731e7fc640 1 -- 172.21.15.67:0/1746241063 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7320004040 con 0x7f7328108cf0 2024-09-21T06:35:50.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.709+0000 7f731e7fc640 1 -- 172.21.15.67:0/1746241063 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7320004670 con 0x7f7328108cf0 2024-09-21T06:35:50.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.710+0000 7f731e7fc640 1 -- 172.21.15.67:0/1746241063 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7320030bb0 con 0x7f7328108cf0 2024-09-21T06:35:50.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.710+0000 7f732f829640 1 -- 172.21.15.67:0/1746241063 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73281a66d0 con 0x7f7328108cf0 2024-09-21T06:35:50.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.710+0000 7f732f829640 1 -- 172.21.15.67:0/1746241063 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73281a6bc0 con 0x7f7328108cf0 2024-09-21T06:35:50.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.711+0000 7f732f829640 1 -- 172.21.15.67:0/1746241063 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7328102e70 con 0x7f7328108cf0 2024-09-21T06:35:50.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.712+0000 7f731e7fc640 1 -- 172.21.15.67:0/1746241063 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f73200041e0 con 0x7f7328108cf0 2024-09-21T06:35:50.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.713+0000 7f731e7fc640 1 --2- 172.21.15.67:0/1746241063 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f73140b65d0 0x7f73140b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:50.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.713+0000 7f732d5c5640 1 --2- 172.21.15.67:0/1746241063 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f73140b65d0 0x7f73140b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:50.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.713+0000 7f731e7fc640 1 -- 172.21.15.67:0/1746241063 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f7320108a00 con 0x7f7328108cf0 2024-09-21T06:35:50.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.713+0000 7f732d5c5640 1 --2- 172.21.15.67:0/1746241063 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f73140b65d0 0x7f73140b8ae0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7324005fd0 tx=0x7f7324005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:50.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.718+0000 7f731e7fc640 1 -- 172.21.15.67:0/1746241063 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f732010ae20 con 0x7f7328108cf0 2024-09-21T06:35:50.752 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:50.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.912+0000 7f732f829640 1 -- 172.21.15.67:0/1746241063 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f732810ab20 con 0x7f7328108cf0 2024-09-21T06:35:50.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.912+0000 7f731e7fc640 1 -- 172.21.15.67:0/1746241063 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f73200cedd0 con 0x7f7328108cf0 2024-09-21T06:35:50.914 INFO:teuthology.orchestra.run.smithi067.stdout:296352743453 2024-09-21T06:35:50.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.915+0000 7f732f829640 1 -- 172.21.15.67:0/1746241063 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f73140b65d0 msgr2=0x7f73140b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:50.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.915+0000 7f732f829640 1 --2- 172.21.15.67:0/1746241063 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f73140b65d0 0x7f73140b8ae0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7324005fd0 tx=0x7f7324005850 comp rx=0 tx=0).stop 2024-09-21T06:35:50.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.915+0000 7f732f829640 1 -- 172.21.15.67:0/1746241063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7328108cf0 msgr2=0x7f732819bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:50.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.915+0000 7f732f829640 1 --2- 172.21.15.67:0/1746241063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7328108cf0 0x7f732819bc30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7320031040 tx=0x7f7320032bc0 comp rx=0 tx=0).stop 2024-09-21T06:35:50.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.915+0000 7f732f829640 1 -- 172.21.15.67:0/1746241063 shutdown_connections 2024-09-21T06:35:50.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.915+0000 7f732f829640 1 --2- 172.21.15.67:0/1746241063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73281083a0 0x7f732819b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:50.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.915+0000 7f732f829640 1 --2- 172.21.15.67:0/1746241063 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f73140b65d0 0x7f73140b8ae0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:50.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.915+0000 7f732f829640 1 --2- 172.21.15.67:0/1746241063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7328108cf0 0x7f732819bc30 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:50.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.915+0000 7f732f829640 1 --2- 172.21.15.67:0/1746241063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7328102a60 0x7f732819b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:50.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.915+0000 7f732f829640 1 -- 172.21.15.67:0/1746241063 >> 172.21.15.67:0/1746241063 conn(0x7f7328079730 msgr2=0x7f7328104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:50.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.915+0000 7f732f829640 1 -- 172.21.15.67:0/1746241063 shutdown_connections 2024-09-21T06:35:50.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:50.915+0000 7f732f829640 1 -- 172.21.15.67:0/1746241063 wait complete. 2024-09-21T06:35:51.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.445+0000 7f76f02f9640 1 -- 172.21.15.67:0/3443885604 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76e8109df0 msgr2=0x7f76e8110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:51.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.445+0000 7f76f02f9640 1 --2- 172.21.15.67:0/3443885604 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76e8109df0 0x7f76e8110620 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009ca0 tx=0x7f76e002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:35:51.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.445+0000 7f76f02f9640 1 -- 172.21.15.67:0/3443885604 shutdown_connections 2024-09-21T06:35:51.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.445+0000 7f76f02f9640 1 --2- 172.21.15.67:0/3443885604 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76e8109df0 0x7f76e8110620 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:51.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.445+0000 7f76f02f9640 1 --2- 172.21.15.67:0/3443885604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76e81094a0 0x7f76e81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:51.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.445+0000 7f76f02f9640 1 --2- 172.21.15.67:0/3443885604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76e8103b60 0x7f76e8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:51.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.445+0000 7f76f02f9640 1 -- 172.21.15.67:0/3443885604 >> 172.21.15.67:0/3443885604 conn(0x7f76e807b850 msgr2=0x7f76e807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:51.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.446+0000 7f76f02f9640 1 -- 172.21.15.67:0/3443885604 shutdown_connections 2024-09-21T06:35:51.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.446+0000 7f76f02f9640 1 -- 172.21.15.67:0/3443885604 wait complete. 2024-09-21T06:35:51.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.447+0000 7f76f02f9640 1 Processor -- start 2024-09-21T06:35:51.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.447+0000 7f76f02f9640 1 -- start start 2024-09-21T06:35:51.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.447+0000 7f76f02f9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76e8103b60 0x7f76e819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:51.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.447+0000 7f76f02f9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76e81094a0 0x7f76e819a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:51.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.447+0000 7f76f02f9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76e8109df0 0x7f76e819fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:51.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.447+0000 7f76f02f9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f76e81139b0 con 0x7f76e8109df0 2024-09-21T06:35:51.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.447+0000 7f76f02f9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f76e8113830 con 0x7f76e8103b60 2024-09-21T06:35:51.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.447+0000 7f76f02f9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f76e8113b30 con 0x7f76e81094a0 2024-09-21T06:35:51.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.447+0000 7f76ee896640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76e8109df0 0x7f76e819fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:51.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.447+0000 7f76ee896640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76e8109df0 0x7f76e819fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44884/0 (socket says 172.21.15.67:44884) 2024-09-21T06:35:51.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.447+0000 7f76ee896640 1 -- 172.21.15.67:0/1792680917 learned_addr learned my addr 172.21.15.67:0/1792680917 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:51.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.447+0000 7f76ee095640 1 --2- 172.21.15.67:0/1792680917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76e8103b60 0x7f76e819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:51.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.447+0000 7f76ed894640 1 --2- 172.21.15.67:0/1792680917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76e81094a0 0x7f76e819a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:51.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.448+0000 7f76ee896640 1 -- 172.21.15.67:0/1792680917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76e81094a0 msgr2=0x7f76e819a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:51.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.448+0000 7f76ee896640 1 --2- 172.21.15.67:0/1792680917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76e81094a0 0x7f76e819a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:51.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.448+0000 7f76ee896640 1 -- 172.21.15.67:0/1792680917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76e8103b60 msgr2=0x7f76e819a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:51.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.448+0000 7f76ee896640 1 --2- 172.21.15.67:0/1792680917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76e8103b60 0x7f76e819a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:51.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.448+0000 7f76ee896640 1 -- 172.21.15.67:0/1792680917 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e81a02b0 con 0x7f76e8109df0 2024-09-21T06:35:51.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.448+0000 7f76ee095640 1 --2- 172.21.15.67:0/1792680917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76e8103b60 0x7f76e819a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:35:51.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.448+0000 7f76ee896640 1 --2- 172.21.15.67:0/1792680917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76e8109df0 0x7f76e819fb80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f76e0005e30 tx=0x7f76e00093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:51.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.448+0000 7f76df7fe640 1 -- 172.21.15.67:0/1792680917 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0032d10 con 0x7f76e8109df0 2024-09-21T06:35:51.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.448+0000 7f76df7fe640 1 -- 172.21.15.67:0/1792680917 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f76e0032eb0 con 0x7f76e8109df0 2024-09-21T06:35:51.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.448+0000 7f76f02f9640 1 -- 172.21.15.67:0/1792680917 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e81a0540 con 0x7f76e8109df0 2024-09-21T06:35:51.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.448+0000 7f76df7fe640 1 -- 172.21.15.67:0/1792680917 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0030b20 con 0x7f76e8109df0 2024-09-21T06:35:51.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.448+0000 7f76f02f9640 1 -- 172.21.15.67:0/1792680917 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e81a6d00 con 0x7f76e8109df0 2024-09-21T06:35:51.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.449+0000 7f76f02f9640 1 -- 172.21.15.67:0/1792680917 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76e8103f70 con 0x7f76e8109df0 2024-09-21T06:35:51.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.450+0000 7f76df7fe640 1 -- 172.21.15.67:0/1792680917 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f76e0030cc0 con 0x7f76e8109df0 2024-09-21T06:35:51.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.451+0000 7f76df7fe640 1 --2- 172.21.15.67:0/1792680917 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f76d40b6620 0x7f76d40b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:51.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.451+0000 7f76ee095640 1 --2- 172.21.15.67:0/1792680917 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f76d40b6620 0x7f76d40b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:51.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.451+0000 7f76df7fe640 1 -- 172.21.15.67:0/1792680917 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f76e0108b00 con 0x7f76e8109df0 2024-09-21T06:35:51.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.452+0000 7f76ee095640 1 --2- 172.21.15.67:0/1792680917 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f76d40b6620 0x7f76d40b8b30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009fc0 tx=0x7f76e4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:51.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.455+0000 7f76df7fe640 1 -- 172.21.15.67:0/1792680917 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f76e00ceed0 con 0x7f76e8109df0 2024-09-21T06:35:51.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.652+0000 7f76f02f9640 1 -- 172.21.15.67:0/1792680917 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f76e819b470 con 0x7f76e8109df0 2024-09-21T06:35:51.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.653+0000 7f76df7fe640 1 -- 172.21.15.67:0/1792680917 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f76e819b470 con 0x7f76e8109df0 2024-09-21T06:35:51.654 INFO:teuthology.orchestra.run.smithi067.stdout:270582939681 2024-09-21T06:35:51.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.655+0000 7f76f02f9640 1 -- 172.21.15.67:0/1792680917 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f76d40b6620 msgr2=0x7f76d40b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:51.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.655+0000 7f76f02f9640 1 --2- 172.21.15.67:0/1792680917 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f76d40b6620 0x7f76d40b8b30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009fc0 tx=0x7f76e4005f90 comp rx=0 tx=0).stop 2024-09-21T06:35:51.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.655+0000 7f76f02f9640 1 -- 172.21.15.67:0/1792680917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76e8109df0 msgr2=0x7f76e819fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:51.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.655+0000 7f76f02f9640 1 --2- 172.21.15.67:0/1792680917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76e8109df0 0x7f76e819fb80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f76e0005e30 tx=0x7f76e00093d0 comp rx=0 tx=0).stop 2024-09-21T06:35:51.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.655+0000 7f76f02f9640 1 -- 172.21.15.67:0/1792680917 shutdown_connections 2024-09-21T06:35:51.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.655+0000 7f76f02f9640 1 --2- 172.21.15.67:0/1792680917 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f76d40b6620 0x7f76d40b8b30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:51.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.655+0000 7f76f02f9640 1 --2- 172.21.15.67:0/1792680917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76e8109df0 0x7f76e819fb80 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:51.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.655+0000 7f76f02f9640 1 --2- 172.21.15.67:0/1792680917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76e81094a0 0x7f76e819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:51.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.655+0000 7f76f02f9640 1 --2- 172.21.15.67:0/1792680917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76e8103b60 0x7f76e819a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:51.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.655+0000 7f76f02f9640 1 -- 172.21.15.67:0/1792680917 >> 172.21.15.67:0/1792680917 conn(0x7f76e807b850 msgr2=0x7f76e81074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:51.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.655+0000 7f76f02f9640 1 -- 172.21.15.67:0/1792680917 shutdown_connections 2024-09-21T06:35:51.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:51.655+0000 7f76f02f9640 1 -- 172.21.15.67:0/1792680917 wait complete. 2024-09-21T06:35:51.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:51 smithi067 ceph-mon[31232]: pgmap v336: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-09-21T06:35:51.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:51 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1746241063' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T06:35:51.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:51 smithi100 ceph-mon[33401]: pgmap v336: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-09-21T06:35:51.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:51 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1746241063' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T06:35:51.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:51 smithi203 ceph-mon[33735]: pgmap v336: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-09-21T06:35:51.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:51 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1746241063' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T06:35:52.030 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743452 got 296352743453 for osd.11 2024-09-21T06:35:52.030 DEBUG:teuthology.parallel:result is None 2024-09-21T06:35:52.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.208+0000 7f881c1e1640 1 -- 172.21.15.67:0/1166561590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8814104a30 msgr2=0x7f8814104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:52.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.208+0000 7f881c1e1640 1 --2- 172.21.15.67:0/1166561590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8814104a30 0x7f8814104e40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8804009ca0 tx=0x7f880402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:35:52.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.208+0000 7f881c1e1640 1 -- 172.21.15.67:0/1166561590 shutdown_connections 2024-09-21T06:35:52.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.208+0000 7f881c1e1640 1 --2- 172.21.15.67:0/1166561590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8814105380 0x7f881410d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:52.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.208+0000 7f881c1e1640 1 --2- 172.21.15.67:0/1166561590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f881410de60 0x7f8814110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:52.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.208+0000 7f881c1e1640 1 --2- 172.21.15.67:0/1166561590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8814104a30 0x7f8814104e40 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:52.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.208+0000 7f881c1e1640 1 -- 172.21.15.67:0/1166561590 >> 172.21.15.67:0/1166561590 conn(0x7f8814079760 msgr2=0x7f8814079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:52.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.208+0000 7f881c1e1640 1 -- 172.21.15.67:0/1166561590 shutdown_connections 2024-09-21T06:35:52.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.209+0000 7f881c1e1640 1 -- 172.21.15.67:0/1166561590 wait complete. 2024-09-21T06:35:52.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.209+0000 7f881c1e1640 1 Processor -- start 2024-09-21T06:35:52.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.209+0000 7f881c1e1640 1 -- start start 2024-09-21T06:35:52.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.209+0000 7f881c1e1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8814104a30 0x7f881419a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:52.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.209+0000 7f881c1e1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8814105380 0x7f881419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:52.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.209+0000 7f881c1e1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f881410de60 0x7f881419fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:52.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.210+0000 7f881c1e1640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f88141139a0 con 0x7f8814105380 2024-09-21T06:35:52.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.210+0000 7f881c1e1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8814113820 con 0x7f881410de60 2024-09-21T06:35:52.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.210+0000 7f881c1e1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8814113b20 con 0x7f8814104a30 2024-09-21T06:35:52.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.210+0000 7f881977c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8814105380 0x7f881419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:52.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.210+0000 7f881977c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8814105380 0x7f881419a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44906/0 (socket says 172.21.15.67:44906) 2024-09-21T06:35:52.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.210+0000 7f881977c640 1 -- 172.21.15.67:0/3406857123 learned_addr learned my addr 172.21.15.67:0/3406857123 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:52.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.210+0000 7f8819f7d640 1 --2- 172.21.15.67:0/3406857123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8814104a30 0x7f881419a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:52.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.210+0000 7f881a77e640 1 --2- 172.21.15.67:0/3406857123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f881410de60 0x7f881419fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:52.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.210+0000 7f881977c640 1 -- 172.21.15.67:0/3406857123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8814104a30 msgr2=0x7f881419a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:52.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.210+0000 7f881977c640 1 --2- 172.21.15.67:0/3406857123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8814104a30 0x7f881419a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:52.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.210+0000 7f881977c640 1 -- 172.21.15.67:0/3406857123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f881410de60 msgr2=0x7f881419fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:52.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.210+0000 7f881977c640 1 --2- 172.21.15.67:0/3406857123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f881410de60 0x7f881419fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:52.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.210+0000 7f881977c640 1 -- 172.21.15.67:0/3406857123 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88141a0340 con 0x7f8814105380 2024-09-21T06:35:52.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.210+0000 7f881a77e640 1 --2- 172.21.15.67:0/3406857123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f881410de60 0x7f881419fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:52.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.210+0000 7f8819f7d640 1 --2- 172.21.15.67:0/3406857123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8814104a30 0x7f881419a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:52.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.210+0000 7f881977c640 1 --2- 172.21.15.67:0/3406857123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8814105380 0x7f881419a970 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f880800d740 tx=0x7f880800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:52.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.210+0000 7f8812ffd640 1 -- 172.21.15.67:0/3406857123 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8808017070 con 0x7f8814105380 2024-09-21T06:35:52.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.210+0000 7f8812ffd640 1 -- 172.21.15.67:0/3406857123 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f880800de50 con 0x7f8814105380 2024-09-21T06:35:52.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.210+0000 7f881c1e1640 1 -- 172.21.15.67:0/3406857123 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88141a0630 con 0x7f8814105380 2024-09-21T06:35:52.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.210+0000 7f8812ffd640 1 -- 172.21.15.67:0/3406857123 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880800f620 con 0x7f8814105380 2024-09-21T06:35:52.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.211+0000 7f881c1e1640 1 -- 172.21.15.67:0/3406857123 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88141a6d90 con 0x7f8814105380 2024-09-21T06:35:52.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.212+0000 7f881c1e1640 1 -- 172.21.15.67:0/3406857123 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f881406c1f0 con 0x7f8814105380 2024-09-21T06:35:52.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.213+0000 7f8812ffd640 1 -- 172.21.15.67:0/3406857123 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f880801a070 con 0x7f8814105380 2024-09-21T06:35:52.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.214+0000 7f8812ffd640 1 --2- 172.21.15.67:0/3406857123 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f87fc0b6620 0x7f87fc0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:52.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.214+0000 7f8819f7d640 1 --2- 172.21.15.67:0/3406857123 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f87fc0b6620 0x7f87fc0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:52.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.214+0000 7f8812ffd640 1 -- 172.21.15.67:0/3406857123 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f88080e7220 con 0x7f8814105380 2024-09-21T06:35:52.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.215+0000 7f8819f7d640 1 --2- 172.21.15.67:0/3406857123 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f87fc0b6620 0x7f87fc0b8b30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8804006010 tx=0x7f8804000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:52.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.217+0000 7f8812ffd640 1 -- 172.21.15.67:0/3406857123 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f88080ad670 con 0x7f8814105380 2024-09-21T06:35:52.229 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:52.229 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:52.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.410+0000 7f881c1e1640 1 -- 172.21.15.67:0/3406857123 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f8814104e40 con 0x7f8814105380 2024-09-21T06:35:52.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.411+0000 7f8812ffd640 1 -- 172.21.15.67:0/3406857123 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f88080b1a50 con 0x7f8814105380 2024-09-21T06:35:52.412 INFO:teuthology.orchestra.run.smithi067.stdout:103079215173 2024-09-21T06:35:52.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.413+0000 7f881c1e1640 1 -- 172.21.15.67:0/3406857123 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f87fc0b6620 msgr2=0x7f87fc0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:52.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.413+0000 7f881c1e1640 1 --2- 172.21.15.67:0/3406857123 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f87fc0b6620 0x7f87fc0b8b30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8804006010 tx=0x7f8804000f40 comp rx=0 tx=0).stop 2024-09-21T06:35:52.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.413+0000 7f881c1e1640 1 -- 172.21.15.67:0/3406857123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8814105380 msgr2=0x7f881419a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:52.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.413+0000 7f881c1e1640 1 --2- 172.21.15.67:0/3406857123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8814105380 0x7f881419a970 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f880800d740 tx=0x7f880800dc00 comp rx=0 tx=0).stop 2024-09-21T06:35:52.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.413+0000 7f881c1e1640 1 -- 172.21.15.67:0/3406857123 shutdown_connections 2024-09-21T06:35:52.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.413+0000 7f881c1e1640 1 --2- 172.21.15.67:0/3406857123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f881410de60 0x7f881419fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:52.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.413+0000 7f881c1e1640 1 --2- 172.21.15.67:0/3406857123 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f87fc0b6620 0x7f87fc0b8b30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:52.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.413+0000 7f881c1e1640 1 --2- 172.21.15.67:0/3406857123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8814105380 0x7f881419a970 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:52.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.413+0000 7f881c1e1640 1 --2- 172.21.15.67:0/3406857123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8814104a30 0x7f881419a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:52.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.413+0000 7f881c1e1640 1 -- 172.21.15.67:0/3406857123 >> 172.21.15.67:0/3406857123 conn(0x7f8814079760 msgr2=0x7f8814100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:52.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.413+0000 7f881c1e1640 1 -- 172.21.15.67:0/3406857123 shutdown_connections 2024-09-21T06:35:52.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:52.413+0000 7f881c1e1640 1 -- 172.21.15.67:0/3406857123 wait complete. 2024-09-21T06:35:52.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:52 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1792680917' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T06:35:52.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:52 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3406857123' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T06:35:52.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:52 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1792680917' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T06:35:52.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:52 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3406857123' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T06:35:53.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:52 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1792680917' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T06:35:53.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:52 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3406857123' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T06:35:53.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.377+0000 7f6c64791640 1 -- 172.21.15.67:0/3314511405 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c5c108f60 msgr2=0x7f6c5c10f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:53.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.377+0000 7f6c64791640 1 --2- 172.21.15.67:0/3314511405 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c5c108f60 0x7f6c5c10f790 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6c54009ca0 tx=0x7f6c5402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.377+0000 7f6c64791640 1 -- 172.21.15.67:0/3314511405 shutdown_connections 2024-09-21T06:35:53.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.377+0000 7f6c64791640 1 --2- 172.21.15.67:0/3314511405 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c5c108f60 0x7f6c5c10f790 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.377+0000 7f6c64791640 1 --2- 172.21.15.67:0/3314511405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c5c108610 0x7f6c5c108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.377+0000 7f6c64791640 1 --2- 172.21.15.67:0/3314511405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c5c1031d0 0x7f6c5c1035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.377+0000 7f6c64791640 1 -- 172.21.15.67:0/3314511405 >> 172.21.15.67:0/3314511405 conn(0x7f6c5c079e40 msgr2=0x7f6c5c07a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:53.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.377+0000 7f6c64791640 1 -- 172.21.15.67:0/3314511405 shutdown_connections 2024-09-21T06:35:53.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.377+0000 7f6c64791640 1 -- 172.21.15.67:0/3314511405 wait complete. 2024-09-21T06:35:53.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.378+0000 7f6c64791640 1 Processor -- start 2024-09-21T06:35:53.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.378+0000 7f6c64791640 1 -- start start 2024-09-21T06:35:53.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.378+0000 7f6c64791640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c5c1031d0 0x7f6c5c19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:53.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.378+0000 7f6c64791640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c5c108610 0x7f6c5c19b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:53.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.378+0000 7f6c64791640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c5c108f60 0x7f6c5c19be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:53.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.378+0000 7f6c64791640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5c112b20 con 0x7f6c5c108610 2024-09-21T06:35:53.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.378+0000 7f6c64791640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5c1129a0 con 0x7f6c5c108f60 2024-09-21T06:35:53.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.378+0000 7f6c64791640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5c112ca0 con 0x7f6c5c1031d0 2024-09-21T06:35:53.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.378+0000 7f6c61d2c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c5c108610 0x7f6c5c19b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:53.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.378+0000 7f6c6252d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c5c1031d0 0x7f6c5c19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:53.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.378+0000 7f6c62d2e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c5c108f60 0x7f6c5c19be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:53.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.378+0000 7f6c61d2c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c5c108610 0x7f6c5c19b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44916/0 (socket says 172.21.15.67:44916) 2024-09-21T06:35:53.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.378+0000 7f6c61d2c640 1 -- 172.21.15.67:0/1680360265 learned_addr learned my addr 172.21.15.67:0/1680360265 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:53.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.379+0000 7f6c61d2c640 1 -- 172.21.15.67:0/1680360265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c5c1031d0 msgr2=0x7f6c5c19b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:53.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.379+0000 7f6c61d2c640 1 --2- 172.21.15.67:0/1680360265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c5c1031d0 0x7f6c5c19b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.379+0000 7f6c61d2c640 1 -- 172.21.15.67:0/1680360265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c5c108f60 msgr2=0x7f6c5c19be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:53.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.379+0000 7f6c61d2c640 1 --2- 172.21.15.67:0/1680360265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c5c108f60 0x7f6c5c19be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.379+0000 7f6c61d2c640 1 -- 172.21.15.67:0/1680360265 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c5c1a66c0 con 0x7f6c5c108610 2024-09-21T06:35:53.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.379+0000 7f6c61d2c640 1 --2- 172.21.15.67:0/1680360265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c5c108610 0x7f6c5c19b8f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00c970 tx=0x7f6c4c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:53.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.379+0000 7f6c537fe640 1 -- 172.21.15.67:0/1680360265 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c004220 con 0x7f6c5c108610 2024-09-21T06:35:53.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.379+0000 7f6c537fe640 1 -- 172.21.15.67:0/1680360265 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c4c007e40 con 0x7f6c5c108610 2024-09-21T06:35:53.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.379+0000 7f6c537fe640 1 -- 172.21.15.67:0/1680360265 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c0047a0 con 0x7f6c5c108610 2024-09-21T06:35:53.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.379+0000 7f6c64791640 1 -- 172.21.15.67:0/1680360265 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c5c199200 con 0x7f6c5c108610 2024-09-21T06:35:53.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.379+0000 7f6c64791640 1 -- 172.21.15.67:0/1680360265 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c5c199740 con 0x7f6c5c108610 2024-09-21T06:35:53.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.381+0000 7f6c64791640 1 -- 172.21.15.67:0/1680360265 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c5c1035e0 con 0x7f6c5c108610 2024-09-21T06:35:53.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.382+0000 7f6c537fe640 1 -- 172.21.15.67:0/1680360265 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f6c4c0079b0 con 0x7f6c5c108610 2024-09-21T06:35:53.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.383+0000 7f6c537fe640 1 --2- 172.21.15.67:0/1680360265 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f6c480b6620 0x7f6c480b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:53.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.383+0000 7f6c6252d640 1 --2- 172.21.15.67:0/1680360265 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f6c480b6620 0x7f6c480b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:53.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.383+0000 7f6c537fe640 1 -- 172.21.15.67:0/1680360265 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f6c4c0e7770 con 0x7f6c5c108610 2024-09-21T06:35:53.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.384+0000 7f6c6252d640 1 --2- 172.21.15.67:0/1680360265 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f6c480b6620 0x7f6c480b8b30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c1029f0 tx=0x7f6c5800f820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:53.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.386+0000 7f6c537fe640 1 -- 172.21.15.67:0/1680360265 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6c4c014030 con 0x7f6c5c108610 2024-09-21T06:35:53.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.567+0000 7f7898072640 1 -- 172.21.15.67:0/855093569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7890109e00 msgr2=0x7f7890110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:53.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.567+0000 7f7898072640 1 --2- 172.21.15.67:0/855093569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7890109e00 0x7f7890110630 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7888009ca0 tx=0x7f788802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.567+0000 7f7898072640 1 -- 172.21.15.67:0/855093569 shutdown_connections 2024-09-21T06:35:53.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.567+0000 7f7898072640 1 --2- 172.21.15.67:0/855093569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7890109e00 0x7f7890110630 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.567+0000 7f7898072640 1 --2- 172.21.15.67:0/855093569 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78901094b0 0x7f78901098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.567+0000 7f7898072640 1 --2- 172.21.15.67:0/855093569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7890103bc0 0x7f7890103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.567+0000 7f7898072640 1 -- 172.21.15.67:0/855093569 >> 172.21.15.67:0/855093569 conn(0x7f789007b910 msgr2=0x7f789007bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:53.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.567+0000 7f7898072640 1 -- 172.21.15.67:0/855093569 shutdown_connections 2024-09-21T06:35:53.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.567+0000 7f7898072640 1 -- 172.21.15.67:0/855093569 wait complete. 2024-09-21T06:35:53.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.568+0000 7f7898072640 1 Processor -- start 2024-09-21T06:35:53.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.568+0000 7f7898072640 1 -- start start 2024-09-21T06:35:53.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.568+0000 7f7898072640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7890103bc0 0x7f789019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:53.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.568+0000 7f7898072640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78901094b0 0x7f789019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:53.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.568+0000 7f7898072640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7890109e00 0x7f789019fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:53.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.568+0000 7f7898072640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f78901139c0 con 0x7f78901094b0 2024-09-21T06:35:53.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.568+0000 7f7898072640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7890113840 con 0x7f7890109e00 2024-09-21T06:35:53.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.568+0000 7f7898072640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7890113b40 con 0x7f7890103bc0 2024-09-21T06:35:53.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.569+0000 7f789560d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78901094b0 0x7f789019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:53.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.569+0000 7f789560d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78901094b0 0x7f789019a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44934/0 (socket says 172.21.15.67:44934) 2024-09-21T06:35:53.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.569+0000 7f789560d640 1 -- 172.21.15.67:0/4199491365 learned_addr learned my addr 172.21.15.67:0/4199491365 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:53.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.569+0000 7f789660f640 1 --2- 172.21.15.67:0/4199491365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7890109e00 0x7f789019fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:53.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.569+0000 7f789560d640 1 -- 172.21.15.67:0/4199491365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7890103bc0 msgr2=0x7f789019a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:53.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.569+0000 7f789560d640 1 --2- 172.21.15.67:0/4199491365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7890103bc0 0x7f789019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.569+0000 7f789560d640 1 -- 172.21.15.67:0/4199491365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7890109e00 msgr2=0x7f789019fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:53.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.569+0000 7f789560d640 1 --2- 172.21.15.67:0/4199491365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7890109e00 0x7f789019fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.569+0000 7f789560d640 1 -- 172.21.15.67:0/4199491365 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78901a02c0 con 0x7f78901094b0 2024-09-21T06:35:53.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.569+0000 7f789560d640 1 --2- 172.21.15.67:0/4199491365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78901094b0 0x7f789019a8f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f788c00e9c0 tx=0x7f788c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:53.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.569+0000 7f7886ffd640 1 -- 172.21.15.67:0/4199491365 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c00ccc0 con 0x7f78901094b0 2024-09-21T06:35:53.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.569+0000 7f7898072640 1 -- 172.21.15.67:0/4199491365 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78901a05b0 con 0x7f78901094b0 2024-09-21T06:35:53.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.569+0000 7f7886ffd640 1 -- 172.21.15.67:0/4199491365 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f788c0054b0 con 0x7f78901094b0 2024-09-21T06:35:53.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.569+0000 7f7886ffd640 1 -- 172.21.15.67:0/4199491365 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c010430 con 0x7f78901094b0 2024-09-21T06:35:53.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.569+0000 7f7898072640 1 -- 172.21.15.67:0/4199491365 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78901a6d10 con 0x7f78901094b0 2024-09-21T06:35:53.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.570+0000 7f7898072640 1 -- 172.21.15.67:0/4199491365 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7890103fd0 con 0x7f78901094b0 2024-09-21T06:35:53.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.574+0000 7f7886ffd640 1 -- 172.21.15.67:0/4199491365 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f788c004f20 con 0x7f78901094b0 2024-09-21T06:35:53.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.574+0000 7f7886ffd640 1 --2- 172.21.15.67:0/4199491365 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f787c0b65a0 0x7f787c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:53.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.574+0000 7f7895e0e640 1 --2- 172.21.15.67:0/4199491365 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f787c0b65a0 0x7f787c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:53.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.574+0000 7f7886ffd640 1 -- 172.21.15.67:0/4199491365 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f788c0e7f50 con 0x7f78901094b0 2024-09-21T06:35:53.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.575+0000 7f7895e0e640 1 --2- 172.21.15.67:0/4199491365 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f787c0b65a0 0x7f787c0b8ab0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7880005fd0 tx=0x7f7880005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:53.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.578+0000 7f7886ffd640 1 -- 172.21.15.67:0/4199491365 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f788c0ae3b0 con 0x7f78901094b0 2024-09-21T06:35:53.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.580+0000 7f6c64791640 1 -- 172.21.15.67:0/1680360265 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f6c5c10ad50 con 0x7f6c5c108610 2024-09-21T06:35:53.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.582+0000 7f6c537fe640 1 -- 172.21.15.67:0/1680360265 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6c4c0b1fa0 con 0x7f6c5c108610 2024-09-21T06:35:53.583 INFO:teuthology.orchestra.run.smithi067.stdout:150323855420 2024-09-21T06:35:53.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.585+0000 7f6c64791640 1 -- 172.21.15.67:0/1680360265 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f6c480b6620 msgr2=0x7f6c480b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:53.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.585+0000 7f6c64791640 1 --2- 172.21.15.67:0/1680360265 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f6c480b6620 0x7f6c480b8b30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c1029f0 tx=0x7f6c5800f820 comp rx=0 tx=0).stop 2024-09-21T06:35:53.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.585+0000 7f6c64791640 1 -- 172.21.15.67:0/1680360265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c5c108610 msgr2=0x7f6c5c19b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:53.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.585+0000 7f6c64791640 1 --2- 172.21.15.67:0/1680360265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c5c108610 0x7f6c5c19b8f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00c970 tx=0x7f6c4c00ce30 comp rx=0 tx=0).stop 2024-09-21T06:35:53.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.585+0000 7f6c64791640 1 -- 172.21.15.67:0/1680360265 shutdown_connections 2024-09-21T06:35:53.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.585+0000 7f6c64791640 1 --2- 172.21.15.67:0/1680360265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c5c108f60 0x7f6c5c19be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.585+0000 7f6c64791640 1 --2- 172.21.15.67:0/1680360265 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f6c480b6620 0x7f6c480b8b30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.585+0000 7f6c64791640 1 --2- 172.21.15.67:0/1680360265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c5c108610 0x7f6c5c19b8f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.585+0000 7f6c64791640 1 --2- 172.21.15.67:0/1680360265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c5c1031d0 0x7f6c5c19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.585+0000 7f6c64791640 1 -- 172.21.15.67:0/1680360265 >> 172.21.15.67:0/1680360265 conn(0x7f6c5c079e40 msgr2=0x7f6c5c106660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:53.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.586+0000 7f6c64791640 1 -- 172.21.15.67:0/1680360265 shutdown_connections 2024-09-21T06:35:53.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.586+0000 7f6c64791640 1 -- 172.21.15.67:0/1680360265 wait complete. 2024-09-21T06:35:53.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.772+0000 7f7898072640 1 -- 172.21.15.67:0/4199491365 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f789019b480 con 0x7f78901094b0 2024-09-21T06:35:53.774 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:53 smithi067 ceph-mon[31232]: pgmap v337: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:35:53.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.773+0000 7f7886ffd640 1 -- 172.21.15.67:0/4199491365 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f789019b480 con 0x7f78901094b0 2024-09-21T06:35:53.775 INFO:teuthology.orchestra.run.smithi067.stdout:55834574928 2024-09-21T06:35:53.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.775+0000 7f7898072640 1 -- 172.21.15.67:0/4199491365 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f787c0b65a0 msgr2=0x7f787c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:53.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.775+0000 7f7898072640 1 --2- 172.21.15.67:0/4199491365 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f787c0b65a0 0x7f787c0b8ab0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7880005fd0 tx=0x7f7880005850 comp rx=0 tx=0).stop 2024-09-21T06:35:53.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.775+0000 7f7898072640 1 -- 172.21.15.67:0/4199491365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78901094b0 msgr2=0x7f789019a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:53.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.775+0000 7f7898072640 1 --2- 172.21.15.67:0/4199491365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78901094b0 0x7f789019a8f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f788c00e9c0 tx=0x7f788c00ee80 comp rx=0 tx=0).stop 2024-09-21T06:35:53.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.775+0000 7f7898072640 1 -- 172.21.15.67:0/4199491365 shutdown_connections 2024-09-21T06:35:53.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.775+0000 7f7898072640 1 --2- 172.21.15.67:0/4199491365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7890109e00 0x7f789019fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.775+0000 7f7898072640 1 --2- 172.21.15.67:0/4199491365 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f787c0b65a0 0x7f787c0b8ab0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.775+0000 7f7898072640 1 --2- 172.21.15.67:0/4199491365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78901094b0 0x7f789019a8f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.775+0000 7f7898072640 1 --2- 172.21.15.67:0/4199491365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7890103bc0 0x7f789019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:53.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.775+0000 7f7898072640 1 -- 172.21.15.67:0/4199491365 >> 172.21.15.67:0/4199491365 conn(0x7f789007b910 msgr2=0x7f7890107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:53.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.775+0000 7f7898072640 1 -- 172.21.15.67:0/4199491365 shutdown_connections 2024-09-21T06:35:53.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:53.775+0000 7f7898072640 1 -- 172.21.15.67:0/4199491365 wait complete. 2024-09-21T06:35:53.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:53 smithi203 ceph-mon[33735]: pgmap v337: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:35:54.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:53 smithi100 ceph-mon[33401]: pgmap v337: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:35:54.060 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215172 got 103079215173 for osd.3 2024-09-21T06:35:54.061 DEBUG:teuthology.parallel:result is None 2024-09-21T06:35:54.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.259+0000 7f66effce640 1 -- 172.21.15.67:0/3998811617 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66e81083a0 msgr2=0x7f66e81087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:54.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.259+0000 7f66effce640 1 --2- 172.21.15.67:0/3998811617 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66e81083a0 0x7f66e81087b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009900 tx=0x7f66d802a410 comp rx=0 tx=0).stop 2024-09-21T06:35:54.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.259+0000 7f66effce640 1 -- 172.21.15.67:0/3998811617 shutdown_connections 2024-09-21T06:35:54.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.259+0000 7f66effce640 1 --2- 172.21.15.67:0/3998811617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66e8108cf0 0x7f66e810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:54.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.259+0000 7f66effce640 1 --2- 172.21.15.67:0/3998811617 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66e81083a0 0x7f66e81087b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:54.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.259+0000 7f66effce640 1 --2- 172.21.15.67:0/3998811617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66e8102a60 0x7f66e8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:54.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.259+0000 7f66effce640 1 -- 172.21.15.67:0/3998811617 >> 172.21.15.67:0/3998811617 conn(0x7f66e8079730 msgr2=0x7f66e8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:54.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.259+0000 7f66effce640 1 -- 172.21.15.67:0/3998811617 shutdown_connections 2024-09-21T06:35:54.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.259+0000 7f66effce640 1 -- 172.21.15.67:0/3998811617 wait complete. 2024-09-21T06:35:54.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.260+0000 7f66effce640 1 Processor -- start 2024-09-21T06:35:54.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.260+0000 7f66effce640 1 -- start start 2024-09-21T06:35:54.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.260+0000 7f66effce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66e8102a60 0x7f66e81992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:54.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.260+0000 7f66effce640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66e81083a0 0x7f66e8199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:54.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.260+0000 7f66effce640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66e8108cf0 0x7f66e819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:54.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.260+0000 7f66effce640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f66e8112810 con 0x7f66e8108cf0 2024-09-21T06:35:54.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.260+0000 7f66effce640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f66e8112690 con 0x7f66e8102a60 2024-09-21T06:35:54.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.260+0000 7f66effce640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f66e8112990 con 0x7f66e81083a0 2024-09-21T06:35:54.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.260+0000 7f66ee56b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66e8108cf0 0x7f66e819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:54.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.260+0000 7f66ee56b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66e8108cf0 0x7f66e819eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44964/0 (socket says 172.21.15.67:44964) 2024-09-21T06:35:54.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.260+0000 7f66ee56b640 1 -- 172.21.15.67:0/1609144211 learned_addr learned my addr 172.21.15.67:0/1609144211 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:54.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.260+0000 7f66edd6a640 1 --2- 172.21.15.67:0/1609144211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66e8102a60 0x7f66e81992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:54.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.260+0000 7f66ed569640 1 --2- 172.21.15.67:0/1609144211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66e81083a0 0x7f66e8199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:54.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.261+0000 7f66ee56b640 1 -- 172.21.15.67:0/1609144211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66e81083a0 msgr2=0x7f66e8199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:54.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.261+0000 7f66ee56b640 1 --2- 172.21.15.67:0/1609144211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66e81083a0 0x7f66e8199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:54.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.261+0000 7f66ee56b640 1 -- 172.21.15.67:0/1609144211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66e8102a60 msgr2=0x7f66e81992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:54.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.261+0000 7f66ee56b640 1 --2- 172.21.15.67:0/1609144211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66e8102a60 0x7f66e81992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:54.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.261+0000 7f66ee56b640 1 -- 172.21.15.67:0/1609144211 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e819f250 con 0x7f66e8108cf0 2024-09-21T06:35:54.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.261+0000 7f66edd6a640 1 --2- 172.21.15.67:0/1609144211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66e8102a60 0x7f66e81992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:54.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.261+0000 7f66ee56b640 1 --2- 172.21.15.67:0/1609144211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66e8108cf0 0x7f66e819eb20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f66e000eaa0 tx=0x7f66e000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:54.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.261+0000 7f66deffd640 1 -- 172.21.15.67:0/1609144211 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e000cdb0 con 0x7f66e8108cf0 2024-09-21T06:35:54.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.261+0000 7f66deffd640 1 -- 172.21.15.67:0/1609144211 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f66e0005020 con 0x7f66e8108cf0 2024-09-21T06:35:54.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.261+0000 7f66effce640 1 -- 172.21.15.67:0/1609144211 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e819f540 con 0x7f66e8108cf0 2024-09-21T06:35:54.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.261+0000 7f66deffd640 1 -- 172.21.15.67:0/1609144211 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e00116d0 con 0x7f66e8108cf0 2024-09-21T06:35:54.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.261+0000 7f66effce640 1 -- 172.21.15.67:0/1609144211 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e81a5ca0 con 0x7f66e8108cf0 2024-09-21T06:35:54.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.262+0000 7f66effce640 1 -- 172.21.15.67:0/1609144211 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66e8102e70 con 0x7f66e8108cf0 2024-09-21T06:35:54.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.264+0000 7f66deffd640 1 -- 172.21.15.67:0/1609144211 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f66e00051d0 con 0x7f66e8108cf0 2024-09-21T06:35:54.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.264+0000 7f66deffd640 1 --2- 172.21.15.67:0/1609144211 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f66d40b65a0 0x7f66d40b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:54.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.265+0000 7f66edd6a640 1 --2- 172.21.15.67:0/1609144211 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f66d40b65a0 0x7f66d40b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:54.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.265+0000 7f66deffd640 1 -- 172.21.15.67:0/1609144211 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f66e00e7e00 con 0x7f66e8108cf0 2024-09-21T06:35:54.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.265+0000 7f66edd6a640 1 --2- 172.21.15.67:0/1609144211 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f66d40b65a0 0x7f66d40b8ab0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f66e40058c0 tx=0x7f66e4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:54.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.268+0000 7f66deffd640 1 -- 172.21.15.67:0/1609144211 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f66e0016030 con 0x7f66e8108cf0 2024-09-21T06:35:54.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.457+0000 7f66effce640 1 -- 172.21.15.67:0/1609144211 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f66e819a410 con 0x7f66e8108cf0 2024-09-21T06:35:54.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.458+0000 7f66deffd640 1 -- 172.21.15.67:0/1609144211 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f66e819a410 con 0x7f66e8108cf0 2024-09-21T06:35:54.459 INFO:teuthology.orchestra.run.smithi067.stdout:197568495665 2024-09-21T06:35:54.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.460+0000 7f66effce640 1 -- 172.21.15.67:0/1609144211 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f66d40b65a0 msgr2=0x7f66d40b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:54.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.460+0000 7f66effce640 1 --2- 172.21.15.67:0/1609144211 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f66d40b65a0 0x7f66d40b8ab0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f66e40058c0 tx=0x7f66e4005850 comp rx=0 tx=0).stop 2024-09-21T06:35:54.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.460+0000 7f66effce640 1 -- 172.21.15.67:0/1609144211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66e8108cf0 msgr2=0x7f66e819eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:54.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.460+0000 7f66effce640 1 --2- 172.21.15.67:0/1609144211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66e8108cf0 0x7f66e819eb20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f66e000eaa0 tx=0x7f66e000ef60 comp rx=0 tx=0).stop 2024-09-21T06:35:54.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.460+0000 7f66effce640 1 -- 172.21.15.67:0/1609144211 shutdown_connections 2024-09-21T06:35:54.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.460+0000 7f66effce640 1 --2- 172.21.15.67:0/1609144211 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f66d40b65a0 0x7f66d40b8ab0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:54.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.460+0000 7f66effce640 1 --2- 172.21.15.67:0/1609144211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66e8108cf0 0x7f66e819eb20 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:54.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.460+0000 7f66effce640 1 --2- 172.21.15.67:0/1609144211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66e81083a0 0x7f66e8199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:54.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.460+0000 7f66effce640 1 --2- 172.21.15.67:0/1609144211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66e8102a60 0x7f66e81992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:54.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.460+0000 7f66effce640 1 -- 172.21.15.67:0/1609144211 >> 172.21.15.67:0/1609144211 conn(0x7f66e8079730 msgr2=0x7f66e81044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:54.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.460+0000 7f66effce640 1 -- 172.21.15.67:0/1609144211 shutdown_connections 2024-09-21T06:35:54.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:54.460+0000 7f66effce640 1 -- 172.21.15.67:0/1609144211 wait complete. 2024-09-21T06:35:54.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:54 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1680360265' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T06:35:54.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:54 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/4199491365' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T06:35:54.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:54 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1609144211' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T06:35:55.024 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:54 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1680360265' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T06:35:55.024 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:54 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/4199491365' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T06:35:55.024 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:54 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1609144211' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T06:35:55.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:54 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1680360265' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T06:35:55.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:54 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/4199491365' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T06:35:55.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:54 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1609144211' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T06:35:55.360 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574925 got 55834574928 for osd.1 2024-09-21T06:35:55.360 DEBUG:teuthology.parallel:result is None 2024-09-21T06:35:55.543 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:55.546 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:55.947 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:55 smithi067 ceph-mon[31232]: pgmap v338: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-09-21T06:35:55.981 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939680 got 270582939681 for osd.10 2024-09-21T06:35:55.981 DEBUG:teuthology.parallel:result is None 2024-09-21T06:35:55.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:55 smithi203 ceph-mon[33735]: pgmap v338: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-09-21T06:35:56.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:55 smithi100 ceph-mon[33401]: pgmap v338: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-09-21T06:35:56.399 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:56.426 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:56.428 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855417 got 150323855420 for osd.5 2024-09-21T06:35:56.428 DEBUG:teuthology.parallel:result is None 2024-09-21T06:35:57.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.334+0000 7efeb9a51640 1 -- 172.21.15.67:0/3750604630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeb41083b0 msgr2=0x7efeb41087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:57.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.334+0000 7efeb9a51640 1 --2- 172.21.15.67:0/3750604630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeb41083b0 0x7efeb41087c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efe9c009900 tx=0x7efe9c02a3f0 comp rx=0 tx=0).stop 2024-09-21T06:35:57.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.334+0000 7efeb9a51640 1 -- 172.21.15.67:0/3750604630 shutdown_connections 2024-09-21T06:35:57.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.334+0000 7efeb9a51640 1 --2- 172.21.15.67:0/3750604630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeb4108d00 0x7efeb410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:57.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.334+0000 7efeb9a51640 1 --2- 172.21.15.67:0/3750604630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeb41083b0 0x7efeb41087c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:57.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.334+0000 7efeb9a51640 1 --2- 172.21.15.67:0/3750604630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efeb4102a70 0x7efeb4102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:57.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.334+0000 7efeb9a51640 1 -- 172.21.15.67:0/3750604630 >> 172.21.15.67:0/3750604630 conn(0x7efeb4079730 msgr2=0x7efeb4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:57.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.334+0000 7efeb9a51640 1 -- 172.21.15.67:0/3750604630 shutdown_connections 2024-09-21T06:35:57.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.334+0000 7efeb9a51640 1 -- 172.21.15.67:0/3750604630 wait complete. 2024-09-21T06:35:57.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.334+0000 7efeb9a51640 1 Processor -- start 2024-09-21T06:35:57.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.335+0000 7efeb9a51640 1 -- start start 2024-09-21T06:35:57.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.335+0000 7efeb9a51640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeb4102a70 0x7efeb419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:57.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.335+0000 7efeb9a51640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeb41083b0 0x7efeb419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:57.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.335+0000 7efeb9a51640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efeb4108d00 0x7efeb419fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:57.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.335+0000 7efeb9a51640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efeb4112820 con 0x7efeb41083b0 2024-09-21T06:35:57.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.335+0000 7efeb9a51640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efeb41126a0 con 0x7efeb4102a70 2024-09-21T06:35:57.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.335+0000 7efeb9a51640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7efeb41129a0 con 0x7efeb4108d00 2024-09-21T06:35:57.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.335+0000 7efeaa7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeb41083b0 0x7efeb419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:57.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.335+0000 7efeaa7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeb41083b0 0x7efeb419a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44994/0 (socket says 172.21.15.67:44994) 2024-09-21T06:35:57.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.335+0000 7efeaa7fc640 1 -- 172.21.15.67:0/3458559821 learned_addr learned my addr 172.21.15.67:0/3458559821 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:57.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.335+0000 7efeb2ffd640 1 --2- 172.21.15.67:0/3458559821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeb4102a70 0x7efeb419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:57.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.335+0000 7efeb37fe640 1 --2- 172.21.15.67:0/3458559821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efeb4108d00 0x7efeb419fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:57.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.335+0000 7efeaa7fc640 1 -- 172.21.15.67:0/3458559821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efeb4108d00 msgr2=0x7efeb419fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:57.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.335+0000 7efeaa7fc640 1 --2- 172.21.15.67:0/3458559821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efeb4108d00 0x7efeb419fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:57.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.336+0000 7efeaa7fc640 1 -- 172.21.15.67:0/3458559821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeb4102a70 msgr2=0x7efeb419a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:57.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.336+0000 7efeaa7fc640 1 --2- 172.21.15.67:0/3458559821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeb4102a70 0x7efeb419a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:57.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.336+0000 7efeaa7fc640 1 -- 172.21.15.67:0/3458559821 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb41a02b0 con 0x7efeb41083b0 2024-09-21T06:35:57.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.336+0000 7efeaa7fc640 1 --2- 172.21.15.67:0/3458559821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeb41083b0 0x7efeb419a9a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7efe9c005940 tx=0x7efe9c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:57.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.336+0000 7efeb8a4f640 1 -- 172.21.15.67:0/3458559821 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c03f030 con 0x7efeb41083b0 2024-09-21T06:35:57.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.336+0000 7efeb8a4f640 1 -- 172.21.15.67:0/3458559821 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe9c003c90 con 0x7efeb41083b0 2024-09-21T06:35:57.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.336+0000 7efeb9a51640 1 -- 172.21.15.67:0/3458559821 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb41a0540 con 0x7efeb41083b0 2024-09-21T06:35:57.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.336+0000 7efeb8a4f640 1 -- 172.21.15.67:0/3458559821 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c035070 con 0x7efeb41083b0 2024-09-21T06:35:57.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.336+0000 7efeb9a51640 1 -- 172.21.15.67:0/3458559821 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb41a0a20 con 0x7efeb41083b0 2024-09-21T06:35:57.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.337+0000 7efeb9a51640 1 -- 172.21.15.67:0/3458559821 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeb4103cf0 con 0x7efeb41083b0 2024-09-21T06:35:57.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.339+0000 7efeb8a4f640 1 -- 172.21.15.67:0/3458559821 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7efe9c042030 con 0x7efeb41083b0 2024-09-21T06:35:57.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.339+0000 7efeb8a4f640 1 --2- 172.21.15.67:0/3458559821 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7efea00b65d0 0x7efea00b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:57.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.340+0000 7efeb2ffd640 1 --2- 172.21.15.67:0/3458559821 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7efea00b65d0 0x7efea00b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:57.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.340+0000 7efeb8a4f640 1 -- 172.21.15.67:0/3458559821 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7efe9c1080e0 con 0x7efeb41083b0 2024-09-21T06:35:57.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.340+0000 7efeb2ffd640 1 --2- 172.21.15.67:0/3458559821 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7efea00b65d0 0x7efea00b8ae0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7efea400abe0 tx=0x7efea400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:57.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.343+0000 7efeb8a4f640 1 -- 172.21.15.67:0/3458559821 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efe9c0ce4b0 con 0x7efeb41083b0 2024-09-21T06:35:57.369 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:35:57.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.546+0000 7efeb9a51640 1 -- 172.21.15.67:0/3458559821 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7efeb410bb30 con 0x7efeb41083b0 2024-09-21T06:35:57.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.546+0000 7efeb8a4f640 1 -- 172.21.15.67:0/3458559821 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7efe9c0d2890 con 0x7efeb41083b0 2024-09-21T06:35:57.549 INFO:teuthology.orchestra.run.smithi067.stdout:249108103208 2024-09-21T06:35:57.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.549+0000 7efeb9a51640 1 -- 172.21.15.67:0/3458559821 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7efea00b65d0 msgr2=0x7efea00b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:57.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.549+0000 7efeb9a51640 1 --2- 172.21.15.67:0/3458559821 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7efea00b65d0 0x7efea00b8ae0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7efea400abe0 tx=0x7efea400a2d0 comp rx=0 tx=0).stop 2024-09-21T06:35:57.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.549+0000 7efeb9a51640 1 -- 172.21.15.67:0/3458559821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeb41083b0 msgr2=0x7efeb419a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:57.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.549+0000 7efeb9a51640 1 --2- 172.21.15.67:0/3458559821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeb41083b0 0x7efeb419a9a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7efe9c005940 tx=0x7efe9c0042f0 comp rx=0 tx=0).stop 2024-09-21T06:35:57.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.549+0000 7efeb9a51640 1 -- 172.21.15.67:0/3458559821 shutdown_connections 2024-09-21T06:35:57.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.549+0000 7efeb9a51640 1 --2- 172.21.15.67:0/3458559821 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7efea00b65d0 0x7efea00b8ae0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:57.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.549+0000 7efeb9a51640 1 --2- 172.21.15.67:0/3458559821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efeb4108d00 0x7efeb419fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:57.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.549+0000 7efeb9a51640 1 --2- 172.21.15.67:0/3458559821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeb41083b0 0x7efeb419a9a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:57.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.549+0000 7efeb9a51640 1 --2- 172.21.15.67:0/3458559821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeb4102a70 0x7efeb419a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:57.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.549+0000 7efeb9a51640 1 -- 172.21.15.67:0/3458559821 >> 172.21.15.67:0/3458559821 conn(0x7efeb4079730 msgr2=0x7efeb4101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:57.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.549+0000 7efeb9a51640 1 -- 172.21.15.67:0/3458559821 shutdown_connections 2024-09-21T06:35:57.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:57.549+0000 7efeb9a51640 1 -- 172.21.15.67:0/3458559821 wait complete. 2024-09-21T06:35:57.864 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:57 smithi067 ceph-mon[31232]: pgmap v339: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:35:57.864 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:57 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3458559821' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T06:35:57.891 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495663 got 197568495665 for osd.7 2024-09-21T06:35:57.892 DEBUG:teuthology.parallel:result is None 2024-09-21T06:35:57.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:57 smithi203 ceph-mon[33735]: pgmap v339: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:35:57.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:57 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3458559821' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T06:35:58.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:57 smithi100 ceph-mon[33401]: pgmap v339: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:35:58.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:57 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3458559821' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T06:35:58.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.055+0000 7f7250f53640 1 -- 172.21.15.67:0/2884133358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f724c109df0 msgr2=0x7f724c110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:58.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.055+0000 7f7250f53640 1 --2- 172.21.15.67:0/2884133358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f724c109df0 0x7f724c110620 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7244009ca0 tx=0x7f724402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.055+0000 7f7250f53640 1 -- 172.21.15.67:0/2884133358 shutdown_connections 2024-09-21T06:35:58.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.055+0000 7f7250f53640 1 --2- 172.21.15.67:0/2884133358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f724c1094a0 0x7f724c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.055+0000 7f7250f53640 1 --2- 172.21.15.67:0/2884133358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f724c109df0 0x7f724c110620 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.055+0000 7f7250f53640 1 --2- 172.21.15.67:0/2884133358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f724c103b60 0x7f724c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.055+0000 7f7250f53640 1 -- 172.21.15.67:0/2884133358 >> 172.21.15.67:0/2884133358 conn(0x7f724c07b850 msgr2=0x7f724c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:58.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.055+0000 7f7250f53640 1 -- 172.21.15.67:0/2884133358 shutdown_connections 2024-09-21T06:35:58.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.055+0000 7f7250f53640 1 -- 172.21.15.67:0/2884133358 wait complete. 2024-09-21T06:35:58.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.056+0000 7f7250f53640 1 Processor -- start 2024-09-21T06:35:58.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.056+0000 7f7250f53640 1 -- start start 2024-09-21T06:35:58.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.057+0000 7f7250f53640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f724c103b60 0x7f724c19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:58.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.057+0000 7f7250f53640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f724c1094a0 0x7f724c19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:58.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.057+0000 7f7250f53640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f724c109df0 0x7f724c19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:58.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.057+0000 7f7250f53640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f724c1139b0 con 0x7f724c109df0 2024-09-21T06:35:58.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.057+0000 7f7250f53640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f724c113830 con 0x7f724c1094a0 2024-09-21T06:35:58.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.057+0000 7f7250f53640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f724c113b30 con 0x7f724c103b60 2024-09-21T06:35:58.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.057+0000 7f724ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f724c109df0 0x7f724c19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:58.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.057+0000 7f724ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f724c109df0 0x7f724c19fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44518/0 (socket says 172.21.15.67:44518) 2024-09-21T06:35:58.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.057+0000 7f724ad9d640 1 -- 172.21.15.67:0/586991936 learned_addr learned my addr 172.21.15.67:0/586991936 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:58.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.057+0000 7f7249d9b640 1 --2- 172.21.15.67:0/586991936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f724c1094a0 0x7f724c19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:58.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.057+0000 7f724a59c640 1 --2- 172.21.15.67:0/586991936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f724c103b60 0x7f724c19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:58.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.057+0000 7f724ad9d640 1 -- 172.21.15.67:0/586991936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f724c103b60 msgr2=0x7f724c19a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:58.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.057+0000 7f724ad9d640 1 --2- 172.21.15.67:0/586991936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f724c103b60 0x7f724c19a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.057+0000 7f724ad9d640 1 -- 172.21.15.67:0/586991936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f724c1094a0 msgr2=0x7f724c19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:58.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.057+0000 7f724ad9d640 1 --2- 172.21.15.67:0/586991936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f724c1094a0 0x7f724c19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.057+0000 7f724ad9d640 1 -- 172.21.15.67:0/586991936 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f724c1a0370 con 0x7f724c109df0 2024-09-21T06:35:58.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.057+0000 7f7249d9b640 1 --2- 172.21.15.67:0/586991936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f724c1094a0 0x7f724c19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:58.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.057+0000 7f724ad9d640 1 --2- 172.21.15.67:0/586991936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f724c109df0 0x7f724c19fc40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7244005e30 tx=0x7f72440093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:58.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.058+0000 7f72437fe640 1 -- 172.21.15.67:0/586991936 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7244032d10 con 0x7f724c109df0 2024-09-21T06:35:58.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.058+0000 7f72437fe640 1 -- 172.21.15.67:0/586991936 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7244032eb0 con 0x7f724c109df0 2024-09-21T06:35:58.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.058+0000 7f7250f53640 1 -- 172.21.15.67:0/586991936 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f724c1a0600 con 0x7f724c109df0 2024-09-21T06:35:58.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.058+0000 7f72437fe640 1 -- 172.21.15.67:0/586991936 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7244030b20 con 0x7f724c109df0 2024-09-21T06:35:58.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.058+0000 7f7250f53640 1 -- 172.21.15.67:0/586991936 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f724c1a6dc0 con 0x7f724c109df0 2024-09-21T06:35:58.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.059+0000 7f7250f53640 1 -- 172.21.15.67:0/586991936 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f724c103f70 con 0x7f724c109df0 2024-09-21T06:35:58.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.061+0000 7f72437fe640 1 -- 172.21.15.67:0/586991936 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f7244030cc0 con 0x7f724c109df0 2024-09-21T06:35:58.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.062+0000 7f72437fe640 1 --2- 172.21.15.67:0/586991936 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f72380b6620 0x7f72380b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:58.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.062+0000 7f724a59c640 1 --2- 172.21.15.67:0/586991936 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f72380b6620 0x7f72380b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:58.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.062+0000 7f72437fe640 1 -- 172.21.15.67:0/586991936 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f7244108b70 con 0x7f724c109df0 2024-09-21T06:35:58.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.062+0000 7f724a59c640 1 --2- 172.21.15.67:0/586991936 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f72380b6620 0x7f72380b8b30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7234005fd0 tx=0x7f723400b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:58.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.065+0000 7f72437fe640 1 -- 172.21.15.67:0/586991936 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f72440cef40 con 0x7f724c109df0 2024-09-21T06:35:58.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.256+0000 7f7250f53640 1 -- 172.21.15.67:0/586991936 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f724c19b530 con 0x7f724c109df0 2024-09-21T06:35:58.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.256+0000 7f72437fe640 1 -- 172.21.15.67:0/586991936 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f724c19b530 con 0x7f724c109df0 2024-09-21T06:35:58.259 INFO:teuthology.orchestra.run.smithi067.stdout:77309411405 2024-09-21T06:35:58.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.258+0000 7f7250f53640 1 -- 172.21.15.67:0/586991936 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f72380b6620 msgr2=0x7f72380b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:58.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.258+0000 7f7250f53640 1 --2- 172.21.15.67:0/586991936 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f72380b6620 0x7f72380b8b30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7234005fd0 tx=0x7f723400b590 comp rx=0 tx=0).stop 2024-09-21T06:35:58.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.258+0000 7f7250f53640 1 -- 172.21.15.67:0/586991936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f724c109df0 msgr2=0x7f724c19fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:58.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.258+0000 7f7250f53640 1 --2- 172.21.15.67:0/586991936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f724c109df0 0x7f724c19fc40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7244005e30 tx=0x7f72440093d0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.258+0000 7f7250f53640 1 -- 172.21.15.67:0/586991936 shutdown_connections 2024-09-21T06:35:58.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.259+0000 7f7250f53640 1 --2- 172.21.15.67:0/586991936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f724c1094a0 0x7f724c19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.259+0000 7f7250f53640 1 --2- 172.21.15.67:0/586991936 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f72380b6620 0x7f72380b8b30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.259+0000 7f7250f53640 1 --2- 172.21.15.67:0/586991936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f724c109df0 0x7f724c19fc40 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.259+0000 7f7250f53640 1 --2- 172.21.15.67:0/586991936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f724c103b60 0x7f724c19a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.259+0000 7f7250f53640 1 -- 172.21.15.67:0/586991936 >> 172.21.15.67:0/586991936 conn(0x7f724c07b850 msgr2=0x7f724c107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:58.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.259+0000 7f7250f53640 1 -- 172.21.15.67:0/586991936 shutdown_connections 2024-09-21T06:35:58.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.259+0000 7f7250f53640 1 -- 172.21.15.67:0/586991936 wait complete. 2024-09-21T06:35:58.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.285+0000 7f91d5b0b640 1 -- 172.21.15.67:0/120278151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91d0102a60 msgr2=0x7f91d0102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:58.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.285+0000 7f91d5b0b640 1 --2- 172.21.15.67:0/120278151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91d0102a60 0x7f91d0102e70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009900 tx=0x7f91c002a410 comp rx=0 tx=0).stop 2024-09-21T06:35:58.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.285+0000 7f91d5b0b640 1 -- 172.21.15.67:0/120278151 shutdown_connections 2024-09-21T06:35:58.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.285+0000 7f91d5b0b640 1 --2- 172.21.15.67:0/120278151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91d0108cf0 0x7f91d010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.285+0000 7f91d5b0b640 1 --2- 172.21.15.67:0/120278151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91d01083a0 0x7f91d01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.285+0000 7f91d5b0b640 1 --2- 172.21.15.67:0/120278151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91d0102a60 0x7f91d0102e70 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.285+0000 7f91d5b0b640 1 -- 172.21.15.67:0/120278151 >> 172.21.15.67:0/120278151 conn(0x7f91d0079730 msgr2=0x7f91d0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:58.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.285+0000 7f91d5b0b640 1 -- 172.21.15.67:0/120278151 shutdown_connections 2024-09-21T06:35:58.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.285+0000 7f91d5b0b640 1 -- 172.21.15.67:0/120278151 wait complete. 2024-09-21T06:35:58.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.286+0000 7f91d5b0b640 1 Processor -- start 2024-09-21T06:35:58.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.286+0000 7f91d5b0b640 1 -- start start 2024-09-21T06:35:58.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.286+0000 7f91d5b0b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91d0102a60 0x7f91d010dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:58.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.286+0000 7f91d5b0b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91d01083a0 0x7f91d010e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:58.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.286+0000 7f91d5b0b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91d0108cf0 0x7f91d01ac770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:58.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.286+0000 7f91d5b0b640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f91d0112810 con 0x7f91d01083a0 2024-09-21T06:35:58.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.286+0000 7f91d5b0b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f91d0112690 con 0x7f91d0108cf0 2024-09-21T06:35:58.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.286+0000 7f91d5b0b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f91d0112990 con 0x7f91d0102a60 2024-09-21T06:35:58.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.286+0000 7f91ceffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91d01083a0 0x7f91d010e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:58.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.287+0000 7f91cffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91d0108cf0 0x7f91d01ac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:58.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.287+0000 7f91cf7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91d0102a60 0x7f91d010dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:58.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.287+0000 7f91ceffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91d01083a0 0x7f91d010e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44536/0 (socket says 172.21.15.67:44536) 2024-09-21T06:35:58.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.287+0000 7f91cffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91d0108cf0 0x7f91d01ac770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.67:57766/0 (socket says 172.21.15.67:57766) 2024-09-21T06:35:58.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.287+0000 7f91ceffd640 1 -- 172.21.15.67:0/464948836 learned_addr learned my addr 172.21.15.67:0/464948836 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:58.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.287+0000 7f91cffff640 1 -- 172.21.15.67:0/464948836 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91d0102a60 msgr2=0x7f91d010dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:58.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.287+0000 7f91cffff640 1 --2- 172.21.15.67:0/464948836 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91d0102a60 0x7f91d010dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.287+0000 7f91cffff640 1 -- 172.21.15.67:0/464948836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91d01083a0 msgr2=0x7f91d010e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:58.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.287+0000 7f91cffff640 1 --2- 172.21.15.67:0/464948836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91d01083a0 0x7f91d010e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.287+0000 7f91cffff640 1 -- 172.21.15.67:0/464948836 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d01accb0 con 0x7f91d0108cf0 2024-09-21T06:35:58.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.287+0000 7f91ceffd640 1 --2- 172.21.15.67:0/464948836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91d01083a0 0x7f91d010e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:35:58.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.287+0000 7f91cffff640 1 --2- 172.21.15.67:0/464948836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91d0108cf0 0x7f91d01ac770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f91c800e940 tx=0x7f91c800ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:58.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.287+0000 7f91ccff9640 1 -- 172.21.15.67:0/464948836 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c8004d20 con 0x7f91d0108cf0 2024-09-21T06:35:58.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.287+0000 7f91ccff9640 1 -- 172.21.15.67:0/464948836 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91c8005020 con 0x7f91d0108cf0 2024-09-21T06:35:58.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.287+0000 7f91d5b0b640 1 -- 172.21.15.67:0/464948836 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91d01ace80 con 0x7f91d0108cf0 2024-09-21T06:35:58.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.288+0000 7f91ccff9640 1 -- 172.21.15.67:0/464948836 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c80116d0 con 0x7f91d0108cf0 2024-09-21T06:35:58.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.288+0000 7f91d5b0b640 1 -- 172.21.15.67:0/464948836 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91d01ad2e0 con 0x7f91d0108cf0 2024-09-21T06:35:58.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.289+0000 7f91d5b0b640 1 -- 172.21.15.67:0/464948836 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91d0102e70 con 0x7f91d0108cf0 2024-09-21T06:35:58.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.290+0000 7f91ccff9640 1 -- 172.21.15.67:0/464948836 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f91c800cb20 con 0x7f91d0108cf0 2024-09-21T06:35:58.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.291+0000 7f91ccff9640 1 --2- 172.21.15.67:0/464948836 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f91b80b6620 0x7f91b80b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:58.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.291+0000 7f91cf7fe640 1 --2- 172.21.15.67:0/464948836 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f91b80b6620 0x7f91b80b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:58.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.292+0000 7f91ccff9640 1 -- 172.21.15.67:0/464948836 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f91c80e7a90 con 0x7f91d0108cf0 2024-09-21T06:35:58.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.292+0000 7f91cf7fe640 1 --2- 172.21.15.67:0/464948836 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f91b80b6620 0x7f91b80b8b30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f91d010f660 tx=0x7f91c00058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:58.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.296+0000 7f91ccff9640 1 -- 172.21.15.67:0/464948836 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f91c8016030 con 0x7f91d0108cf0 2024-09-21T06:35:58.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.489+0000 7f91d5b0b640 1 -- 172.21.15.67:0/464948836 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f91d01ad630 con 0x7f91d0108cf0 2024-09-21T06:35:58.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.490+0000 7f91ccff9640 1 -- 172.21.15.67:0/464948836 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f91c800ce10 con 0x7f91d0108cf0 2024-09-21T06:35:58.492 INFO:teuthology.orchestra.run.smithi067.stdout:223338299438 2024-09-21T06:35:58.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.492+0000 7f91d5b0b640 1 -- 172.21.15.67:0/464948836 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f91b80b6620 msgr2=0x7f91b80b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:58.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.492+0000 7f91d5b0b640 1 --2- 172.21.15.67:0/464948836 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f91b80b6620 0x7f91b80b8b30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f91d010f660 tx=0x7f91c00058d0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.492+0000 7f91d5b0b640 1 -- 172.21.15.67:0/464948836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91d0108cf0 msgr2=0x7f91d01ac770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:58.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.492+0000 7f91d5b0b640 1 --2- 172.21.15.67:0/464948836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91d0108cf0 0x7f91d01ac770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f91c800e940 tx=0x7f91c800ee00 comp rx=0 tx=0).stop 2024-09-21T06:35:58.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.493+0000 7f91d5b0b640 1 -- 172.21.15.67:0/464948836 shutdown_connections 2024-09-21T06:35:58.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.493+0000 7f91d5b0b640 1 --2- 172.21.15.67:0/464948836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91d0108cf0 0x7f91d01ac770 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.493+0000 7f91d5b0b640 1 --2- 172.21.15.67:0/464948836 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f91b80b6620 0x7f91b80b8b30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.493+0000 7f91d5b0b640 1 --2- 172.21.15.67:0/464948836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91d01083a0 0x7f91d010e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.493+0000 7f91d5b0b640 1 --2- 172.21.15.67:0/464948836 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91d0102a60 0x7f91d010dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:58.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.493+0000 7f91d5b0b640 1 -- 172.21.15.67:0/464948836 >> 172.21.15.67:0/464948836 conn(0x7f91d0079730 msgr2=0x7f91d0100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:58.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.493+0000 7f91d5b0b640 1 -- 172.21.15.67:0/464948836 shutdown_connections 2024-09-21T06:35:58.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:58.493+0000 7f91d5b0b640 1 -- 172.21.15.67:0/464948836 wait complete. 2024-09-21T06:35:58.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:58 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/586991936' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T06:35:58.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:58 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/464948836' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T06:35:58.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:58 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/586991936' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T06:35:58.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:58 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/464948836' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T06:35:59.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:58 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/586991936' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T06:35:59.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:58 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/464948836' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T06:35:59.307 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299436 got 223338299438 for osd.8 2024-09-21T06:35:59.307 DEBUG:teuthology.parallel:result is None 2024-09-21T06:35:59.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.703+0000 7f3dd6ee0640 1 -- 172.21.15.67:0/846590964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dd0108d20 msgr2=0x7f3dd010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:59.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.703+0000 7f3dd6ee0640 1 --2- 172.21.15.67:0/846590964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dd0108d20 0x7f3dd010f550 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8009ca0 tx=0x7f3dc802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.703+0000 7f3dd6ee0640 1 -- 172.21.15.67:0/846590964 shutdown_connections 2024-09-21T06:35:59.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.703+0000 7f3dd6ee0640 1 --2- 172.21.15.67:0/846590964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dd01083d0 0x7f3dd01087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.703+0000 7f3dd6ee0640 1 --2- 172.21.15.67:0/846590964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dd0108d20 0x7f3dd010f550 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.703+0000 7f3dd6ee0640 1 --2- 172.21.15.67:0/846590964 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dd0102ae0 0x7f3dd0102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.703+0000 7f3dd6ee0640 1 -- 172.21.15.67:0/846590964 >> 172.21.15.67:0/846590964 conn(0x7f3dd00797b0 msgr2=0x7f3dd0079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:59.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.704+0000 7f3dd6ee0640 1 -- 172.21.15.67:0/846590964 shutdown_connections 2024-09-21T06:35:59.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.704+0000 7f3dd6ee0640 1 -- 172.21.15.67:0/846590964 wait complete. 2024-09-21T06:35:59.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.704+0000 7f3dd6ee0640 1 Processor -- start 2024-09-21T06:35:59.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.704+0000 7f3dd6ee0640 1 -- start start 2024-09-21T06:35:59.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.705+0000 7f3dd6ee0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dd0102ae0 0x7f3dd019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:59.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.705+0000 7f3dd6ee0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dd01083d0 0x7f3dd019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:59.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.705+0000 7f3dd6ee0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dd0108d20 0x7f3dd019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:59.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.705+0000 7f3dd6ee0640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd01139e0 con 0x7f3dd0108d20 2024-09-21T06:35:59.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.705+0000 7f3dd6ee0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd0113860 con 0x7f3dd01083d0 2024-09-21T06:35:59.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.705+0000 7f3dd6ee0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd0113b60 con 0x7f3dd0102ae0 2024-09-21T06:35:59.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.705+0000 7f3dd547d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dd0108d20 0x7f3dd019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:59.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.705+0000 7f3dd547d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dd0108d20 0x7f3dd019fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44550/0 (socket says 172.21.15.67:44550) 2024-09-21T06:35:59.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.705+0000 7f3dd547d640 1 -- 172.21.15.67:0/639423116 learned_addr learned my addr 172.21.15.67:0/639423116 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:59.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.705+0000 7f3dcffff640 1 --2- 172.21.15.67:0/639423116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dd01083d0 0x7f3dd019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:59.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.705+0000 7f3dd4c7c640 1 --2- 172.21.15.67:0/639423116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dd0102ae0 0x7f3dd019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:59.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.705+0000 7f3dd547d640 1 -- 172.21.15.67:0/639423116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dd0102ae0 msgr2=0x7f3dd019a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:59.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.705+0000 7f3dd547d640 1 --2- 172.21.15.67:0/639423116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dd0102ae0 0x7f3dd019a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.705+0000 7f3dd547d640 1 -- 172.21.15.67:0/639423116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dd01083d0 msgr2=0x7f3dd019a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:59.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.705+0000 7f3dd547d640 1 --2- 172.21.15.67:0/639423116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dd01083d0 0x7f3dd019a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.705+0000 7f3dd547d640 1 -- 172.21.15.67:0/639423116 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd01a02a0 con 0x7f3dd0108d20 2024-09-21T06:35:59.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.705+0000 7f3dd4c7c640 1 --2- 172.21.15.67:0/639423116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dd0102ae0 0x7f3dd019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:35:59.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.705+0000 7f3dd547d640 1 --2- 172.21.15.67:0/639423116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dd0108d20 0x7f3dd019fc30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8005e30 tx=0x7f3dc80093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:59.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.705+0000 7f3dcdffb640 1 -- 172.21.15.67:0/639423116 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8032d10 con 0x7f3dd0108d20 2024-09-21T06:35:59.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.705+0000 7f3dcdffb640 1 -- 172.21.15.67:0/639423116 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3dc8032eb0 con 0x7f3dd0108d20 2024-09-21T06:35:59.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.705+0000 7f3dcdffb640 1 -- 172.21.15.67:0/639423116 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8030b20 con 0x7f3dd0108d20 2024-09-21T06:35:59.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.706+0000 7f3dd6ee0640 1 -- 172.21.15.67:0/639423116 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dd01a0530 con 0x7f3dd0108d20 2024-09-21T06:35:59.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.706+0000 7f3dd6ee0640 1 -- 172.21.15.67:0/639423116 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dd01a0a70 con 0x7f3dd0108d20 2024-09-21T06:35:59.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.707+0000 7f3dd6ee0640 1 -- 172.21.15.67:0/639423116 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dd0102ef0 con 0x7f3dd0108d20 2024-09-21T06:35:59.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.709+0000 7f3dcdffb640 1 -- 172.21.15.67:0/639423116 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f3dc8030cc0 con 0x7f3dd0108d20 2024-09-21T06:35:59.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.710+0000 7f3dcdffb640 1 --2- 172.21.15.67:0/639423116 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f3dc00b65d0 0x7f3dc00b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:59.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.710+0000 7f3dd4c7c640 1 --2- 172.21.15.67:0/639423116 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f3dc00b65d0 0x7f3dc00b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:59.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.710+0000 7f3dcdffb640 1 -- 172.21.15.67:0/639423116 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f3dc8108c50 con 0x7f3dd0108d20 2024-09-21T06:35:59.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.711+0000 7f3dd4c7c640 1 --2- 172.21.15.67:0/639423116 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f3dc00b65d0 0x7f3dc00b8ae0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400baa0 tx=0x7f3dc4005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:59.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.714+0000 7f3dcdffb640 1 -- 172.21.15.67:0/639423116 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3dc80cefc0 con 0x7f3dd0108d20 2024-09-21T06:35:59.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:35:59 smithi067 ceph-mon[31232]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:35:59.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.907+0000 7f3dd6ee0640 1 -- 172.21.15.67:0/639423116 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f3dd019b520 con 0x7f3dd0108d20 2024-09-21T06:35:59.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.907+0000 7f3dcdffb640 1 -- 172.21.15.67:0/639423116 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3dd019b520 con 0x7f3dd0108d20 2024-09-21T06:35:59.910 INFO:teuthology.orchestra.run.smithi067.stdout:176093659192 2024-09-21T06:35:59.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.910+0000 7f3dd6ee0640 1 -- 172.21.15.67:0/639423116 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f3dc00b65d0 msgr2=0x7f3dc00b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:59.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.910+0000 7f3dd6ee0640 1 --2- 172.21.15.67:0/639423116 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f3dc00b65d0 0x7f3dc00b8ae0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400baa0 tx=0x7f3dc4005d90 comp rx=0 tx=0).stop 2024-09-21T06:35:59.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.910+0000 7f3dd6ee0640 1 -- 172.21.15.67:0/639423116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dd0108d20 msgr2=0x7f3dd019fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:59.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.910+0000 7f3dd6ee0640 1 --2- 172.21.15.67:0/639423116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dd0108d20 0x7f3dd019fc30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8005e30 tx=0x7f3dc80093d0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.910+0000 7f3dd6ee0640 1 -- 172.21.15.67:0/639423116 shutdown_connections 2024-09-21T06:35:59.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.910+0000 7f3dd6ee0640 1 --2- 172.21.15.67:0/639423116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dd01083d0 0x7f3dd019a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.910+0000 7f3dd6ee0640 1 --2- 172.21.15.67:0/639423116 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f3dc00b65d0 0x7f3dc00b8ae0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.910+0000 7f3dd6ee0640 1 --2- 172.21.15.67:0/639423116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dd0108d20 0x7f3dd019fc30 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.910+0000 7f3dd6ee0640 1 --2- 172.21.15.67:0/639423116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dd0102ae0 0x7f3dd019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.910+0000 7f3dd6ee0640 1 -- 172.21.15.67:0/639423116 >> 172.21.15.67:0/639423116 conn(0x7f3dd00797b0 msgr2=0x7f3dd0106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:59.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.910+0000 7f3dd6ee0640 1 -- 172.21.15.67:0/639423116 shutdown_connections 2024-09-21T06:35:59.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.910+0000 7f3dd6ee0640 1 -- 172.21.15.67:0/639423116 wait complete. 2024-09-21T06:35:59.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.943+0000 7f30e7891640 1 -- 172.21.15.67:0/4058326567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30e0101f70 msgr2=0x7f30e0102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:59.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.943+0000 7f30e7891640 1 --2- 172.21.15.67:0/4058326567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30e0101f70 0x7f30e0102380 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f30dc009900 tx=0x7f30dc02a410 comp rx=0 tx=0).stop 2024-09-21T06:35:59.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.943+0000 7f30e7891640 1 -- 172.21.15.67:0/4058326567 shutdown_connections 2024-09-21T06:35:59.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.943+0000 7f30e7891640 1 --2- 172.21.15.67:0/4058326567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30e01094b0 0x7f30e01098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.943+0000 7f30e7891640 1 --2- 172.21.15.67:0/4058326567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30e0109e00 0x7f30e0110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.943+0000 7f30e7891640 1 --2- 172.21.15.67:0/4058326567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30e0101f70 0x7f30e0102380 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.943+0000 7f30e7891640 1 -- 172.21.15.67:0/4058326567 >> 172.21.15.67:0/4058326567 conn(0x7f30e0077c60 msgr2=0x7f30e00780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:35:59.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.944+0000 7f30e7891640 1 -- 172.21.15.67:0/4058326567 shutdown_connections 2024-09-21T06:35:59.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.944+0000 7f30e7891640 1 -- 172.21.15.67:0/4058326567 wait complete. 2024-09-21T06:35:59.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.944+0000 7f30e7891640 1 Processor -- start 2024-09-21T06:35:59.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.944+0000 7f30e7891640 1 -- start start 2024-09-21T06:35:59.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.945+0000 7f30e7891640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30e0101f70 0x7f30e019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:59.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.945+0000 7f30e7891640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30e01094b0 0x7f30e019a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:59.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.945+0000 7f30e7891640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30e0109e00 0x7f30e019fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:59.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.945+0000 7f30e7891640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f30e0113970 con 0x7f30e0109e00 2024-09-21T06:35:59.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.945+0000 7f30e7891640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f30e01137f0 con 0x7f30e01094b0 2024-09-21T06:35:59.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.945+0000 7f30e7891640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f30e0113af0 con 0x7f30e0101f70 2024-09-21T06:35:59.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.945+0000 7f30e5e2e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30e0109e00 0x7f30e019fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:59.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.945+0000 7f30e5e2e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30e0109e00 0x7f30e019fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44578/0 (socket says 172.21.15.67:44578) 2024-09-21T06:35:59.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.945+0000 7f30e5e2e640 1 -- 172.21.15.67:0/1925473071 learned_addr learned my addr 172.21.15.67:0/1925473071 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:35:59.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.945+0000 7f30e4e2c640 1 --2- 172.21.15.67:0/1925473071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30e01094b0 0x7f30e019a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:59.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.945+0000 7f30e562d640 1 --2- 172.21.15.67:0/1925473071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30e0101f70 0x7f30e019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:59.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.945+0000 7f30e5e2e640 1 -- 172.21.15.67:0/1925473071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30e0101f70 msgr2=0x7f30e019a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:59.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.945+0000 7f30e5e2e640 1 --2- 172.21.15.67:0/1925473071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30e0101f70 0x7f30e019a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.945+0000 7f30e5e2e640 1 -- 172.21.15.67:0/1925473071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30e01094b0 msgr2=0x7f30e019a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:35:59.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.945+0000 7f30e5e2e640 1 --2- 172.21.15.67:0/1925473071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30e01094b0 0x7f30e019a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:35:59.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.945+0000 7f30e5e2e640 1 -- 172.21.15.67:0/1925473071 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30e01a0350 con 0x7f30e0109e00 2024-09-21T06:35:59.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.945+0000 7f30e4e2c640 1 --2- 172.21.15.67:0/1925473071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30e01094b0 0x7f30e019a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:35:59.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.945+0000 7f30e5e2e640 1 --2- 172.21.15.67:0/1925473071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30e0109e00 0x7f30e019fc20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f30d800eaa0 tx=0x7f30d800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:59.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.945+0000 7f30d67fc640 1 -- 172.21.15.67:0/1925473071 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d800cdb0 con 0x7f30e0109e00 2024-09-21T06:35:59.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.945+0000 7f30e7891640 1 -- 172.21.15.67:0/1925473071 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30e01a0640 con 0x7f30e0109e00 2024-09-21T06:35:59.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.945+0000 7f30d67fc640 1 -- 172.21.15.67:0/1925473071 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f30d8005020 con 0x7f30e0109e00 2024-09-21T06:35:59.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.945+0000 7f30d67fc640 1 -- 172.21.15.67:0/1925473071 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d80116d0 con 0x7f30e0109e00 2024-09-21T06:35:59.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.946+0000 7f30e7891640 1 -- 172.21.15.67:0/1925473071 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30e01a6da0 con 0x7f30e0109e00 2024-09-21T06:35:59.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.947+0000 7f30e7891640 1 -- 172.21.15.67:0/1925473071 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30e0102380 con 0x7f30e0109e00 2024-09-21T06:35:59.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.948+0000 7f30d67fc640 1 -- 172.21.15.67:0/1925473071 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f30d8004a40 con 0x7f30e0109e00 2024-09-21T06:35:59.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.949+0000 7f30d67fc640 1 --2- 172.21.15.67:0/1925473071 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f30cc0b6620 0x7f30cc0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:35:59.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.949+0000 7f30e562d640 1 --2- 172.21.15.67:0/1925473071 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f30cc0b6620 0x7f30cc0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:35:59.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.949+0000 7f30d67fc640 1 -- 172.21.15.67:0/1925473071 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f30d80e7a70 con 0x7f30e0109e00 2024-09-21T06:35:59.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.949+0000 7f30e562d640 1 --2- 172.21.15.67:0/1925473071 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f30cc0b6620 0x7f30cc0b8b30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f30dc000c00 tx=0x7f30dc009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:35:59.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:35:59.953+0000 7f30d67fc640 1 -- 172.21.15.67:0/1925473071 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f30d8016030 con 0x7f30e0109e00 2024-09-21T06:35:59.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:35:59 smithi203 ceph-mon[33735]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:36:00.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:35:59 smithi100 ceph-mon[33401]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:36:00.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:00.144+0000 7f30e7891640 1 -- 172.21.15.67:0/1925473071 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f30e010b640 con 0x7f30e0109e00 2024-09-21T06:36:00.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:00.145+0000 7f30d67fc640 1 -- 172.21.15.67:0/1925473071 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f30d80adde0 con 0x7f30e0109e00 2024-09-21T06:36:00.147 INFO:teuthology.orchestra.run.smithi067.stdout:128849018947 2024-09-21T06:36:00.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:00.147+0000 7f30e7891640 1 -- 172.21.15.67:0/1925473071 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f30cc0b6620 msgr2=0x7f30cc0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:00.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:00.147+0000 7f30e7891640 1 --2- 172.21.15.67:0/1925473071 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f30cc0b6620 0x7f30cc0b8b30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f30dc000c00 tx=0x7f30dc009680 comp rx=0 tx=0).stop 2024-09-21T06:36:00.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:00.147+0000 7f30e7891640 1 -- 172.21.15.67:0/1925473071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30e0109e00 msgr2=0x7f30e019fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:00.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:00.147+0000 7f30e7891640 1 --2- 172.21.15.67:0/1925473071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30e0109e00 0x7f30e019fc20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f30d800eaa0 tx=0x7f30d800ef60 comp rx=0 tx=0).stop 2024-09-21T06:36:00.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:00.147+0000 7f30e7891640 1 -- 172.21.15.67:0/1925473071 shutdown_connections 2024-09-21T06:36:00.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:00.147+0000 7f30e7891640 1 --2- 172.21.15.67:0/1925473071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30e01094b0 0x7f30e019a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:00.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:00.147+0000 7f30e7891640 1 --2- 172.21.15.67:0/1925473071 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f30cc0b6620 0x7f30cc0b8b30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:00.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:00.147+0000 7f30e7891640 1 --2- 172.21.15.67:0/1925473071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30e0109e00 0x7f30e019fc20 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:00.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:00.147+0000 7f30e7891640 1 --2- 172.21.15.67:0/1925473071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30e0101f70 0x7f30e019a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:00.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:00.147+0000 7f30e7891640 1 -- 172.21.15.67:0/1925473071 >> 172.21.15.67:0/1925473071 conn(0x7f30e0077c60 msgr2=0x7f30e0107600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:00.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:00.147+0000 7f30e7891640 1 -- 172.21.15.67:0/1925473071 shutdown_connections 2024-09-21T06:36:00.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:00.147+0000 7f30e7891640 1 -- 172.21.15.67:0/1925473071 wait complete. 2024-09-21T06:36:00.649 INFO:tasks.cephadm.ceph_manager.ceph:need seq 176093659189 got 176093659192 for osd.6 2024-09-21T06:36:00.650 DEBUG:teuthology.parallel:result is None 2024-09-21T06:36:00.911 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:00 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/639423116' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T06:36:00.911 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:00 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1925473071' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T06:36:00.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:00 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/639423116' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T06:36:00.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:00 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1925473071' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T06:36:01.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:00 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/639423116' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T06:36:01.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:00 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1925473071' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T06:36:01.563 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103207 got 249108103208 for osd.9 2024-09-21T06:36:01.563 DEBUG:teuthology.parallel:result is None 2024-09-21T06:36:01.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:01 smithi067 ceph-mon[31232]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:36:01.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:01 smithi203 ceph-mon[33735]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:36:01.995 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018944 got 128849018947 for osd.4 2024-09-21T06:36:01.995 DEBUG:teuthology.parallel:result is None 2024-09-21T06:36:02.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:01 smithi100 ceph-mon[33401]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:36:02.754 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:36:02.762 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411401 got 77309411405 for osd.2 2024-09-21T06:36:02.762 DEBUG:teuthology.parallel:result is None 2024-09-21T06:36:03.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.246+0000 7f9c6de29640 1 -- 172.21.15.67:0/1449172068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c68108cf0 msgr2=0x7f9c6810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:03.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.246+0000 7f9c6de29640 1 --2- 172.21.15.67:0/1449172068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c68108cf0 0x7f9c6810f520 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9c60009ca0 tx=0x7f9c6002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:36:03.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.246+0000 7f9c6de29640 1 -- 172.21.15.67:0/1449172068 shutdown_connections 2024-09-21T06:36:03.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.246+0000 7f9c6de29640 1 --2- 172.21.15.67:0/1449172068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c68108cf0 0x7f9c6810f520 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:03.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.246+0000 7f9c6de29640 1 --2- 172.21.15.67:0/1449172068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c681083a0 0x7f9c681087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:03.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.246+0000 7f9c6de29640 1 --2- 172.21.15.67:0/1449172068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c68102a60 0x7f9c68102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:03.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.246+0000 7f9c6de29640 1 -- 172.21.15.67:0/1449172068 >> 172.21.15.67:0/1449172068 conn(0x7f9c68079730 msgr2=0x7f9c68079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:03.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.247+0000 7f9c6de29640 1 -- 172.21.15.67:0/1449172068 shutdown_connections 2024-09-21T06:36:03.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.247+0000 7f9c6de29640 1 -- 172.21.15.67:0/1449172068 wait complete. 2024-09-21T06:36:03.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.247+0000 7f9c6de29640 1 Processor -- start 2024-09-21T06:36:03.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.248+0000 7f9c6de29640 1 -- start start 2024-09-21T06:36:03.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.248+0000 7f9c6de29640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c68102a60 0x7f9c6819a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:03.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.248+0000 7f9c6de29640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c681083a0 0x7f9c6819a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:03.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.248+0000 7f9c6de29640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c68108cf0 0x7f9c6819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:03.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.248+0000 7f9c6de29640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9c68112860 con 0x7f9c68108cf0 2024-09-21T06:36:03.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.248+0000 7f9c6de29640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9c681126e0 con 0x7f9c68102a60 2024-09-21T06:36:03.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.248+0000 7f9c6de29640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9c681129e0 con 0x7f9c681083a0 2024-09-21T06:36:03.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.248+0000 7f9c67fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c68108cf0 0x7f9c6819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:03.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.248+0000 7f9c67fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c68108cf0 0x7f9c6819fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44588/0 (socket says 172.21.15.67:44588) 2024-09-21T06:36:03.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.248+0000 7f9c67fff640 1 -- 172.21.15.67:0/1672262905 learned_addr learned my addr 172.21.15.67:0/1672262905 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:03.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.248+0000 7f9c677fe640 1 --2- 172.21.15.67:0/1672262905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c68102a60 0x7f9c6819a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:03.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.248+0000 7f9c66ffd640 1 --2- 172.21.15.67:0/1672262905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c681083a0 0x7f9c6819a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:03.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.248+0000 7f9c67fff640 1 -- 172.21.15.67:0/1672262905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c681083a0 msgr2=0x7f9c6819a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:03.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.248+0000 7f9c67fff640 1 --2- 172.21.15.67:0/1672262905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c681083a0 0x7f9c6819a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:03.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.248+0000 7f9c67fff640 1 -- 172.21.15.67:0/1672262905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c68102a60 msgr2=0x7f9c6819a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:03.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.248+0000 7f9c67fff640 1 --2- 172.21.15.67:0/1672262905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c68102a60 0x7f9c6819a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:03.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.248+0000 7f9c67fff640 1 -- 172.21.15.67:0/1672262905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c681a0270 con 0x7f9c68108cf0 2024-09-21T06:36:03.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.249+0000 7f9c67fff640 1 --2- 172.21.15.67:0/1672262905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c68108cf0 0x7f9c6819fc00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9c60005cc0 tx=0x7f9c60032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:03.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.249+0000 7f9c64ff9640 1 -- 172.21.15.67:0/1672262905 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60032e00 con 0x7f9c68108cf0 2024-09-21T06:36:03.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.249+0000 7f9c64ff9640 1 -- 172.21.15.67:0/1672262905 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c600044d0 con 0x7f9c68108cf0 2024-09-21T06:36:03.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.249+0000 7f9c6de29640 1 -- 172.21.15.67:0/1672262905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c681a0500 con 0x7f9c68108cf0 2024-09-21T06:36:03.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.249+0000 7f9c64ff9640 1 -- 172.21.15.67:0/1672262905 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60030b70 con 0x7f9c68108cf0 2024-09-21T06:36:03.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.249+0000 7f9c6de29640 1 -- 172.21.15.67:0/1672262905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c681a0a40 con 0x7f9c68108cf0 2024-09-21T06:36:03.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.250+0000 7f9c6de29640 1 -- 172.21.15.67:0/1672262905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c68102e70 con 0x7f9c68108cf0 2024-09-21T06:36:03.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.252+0000 7f9c64ff9640 1 -- 172.21.15.67:0/1672262905 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f9c60030d10 con 0x7f9c68108cf0 2024-09-21T06:36:03.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.252+0000 7f9c64ff9640 1 --2- 172.21.15.67:0/1672262905 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9c500b65d0 0x7f9c500b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:03.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.253+0000 7f9c677fe640 1 --2- 172.21.15.67:0/1672262905 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9c500b65d0 0x7f9c500b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:03.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.253+0000 7f9c64ff9640 1 -- 172.21.15.67:0/1672262905 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f9c60108b50 con 0x7f9c68108cf0 2024-09-21T06:36:03.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.253+0000 7f9c677fe640 1 --2- 172.21.15.67:0/1672262905 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9c500b65d0 0x7f9c500b8ae0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9c5800bbe0 tx=0x7f9c5800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:03.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.256+0000 7f9c64ff9640 1 -- 172.21.15.67:0/1672262905 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9c600cef20 con 0x7f9c68108cf0 2024-09-21T06:36:03.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.445+0000 7f9c6de29640 1 -- 172.21.15.67:0/1672262905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f9c6819b4f0 con 0x7f9c68108cf0 2024-09-21T06:36:03.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.445+0000 7f9c64ff9640 1 -- 172.21.15.67:0/1672262905 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9c6819b4f0 con 0x7f9c68108cf0 2024-09-21T06:36:03.448 INFO:teuthology.orchestra.run.smithi067.stdout:34359738456 2024-09-21T06:36:03.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.447+0000 7f9c6de29640 1 -- 172.21.15.67:0/1672262905 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9c500b65d0 msgr2=0x7f9c500b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:03.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.447+0000 7f9c6de29640 1 --2- 172.21.15.67:0/1672262905 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9c500b65d0 0x7f9c500b8ae0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9c5800bbe0 tx=0x7f9c5800b390 comp rx=0 tx=0).stop 2024-09-21T06:36:03.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.447+0000 7f9c6de29640 1 -- 172.21.15.67:0/1672262905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c68108cf0 msgr2=0x7f9c6819fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:03.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.447+0000 7f9c6de29640 1 --2- 172.21.15.67:0/1672262905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c68108cf0 0x7f9c6819fc00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9c60005cc0 tx=0x7f9c60032a10 comp rx=0 tx=0).stop 2024-09-21T06:36:03.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.447+0000 7f9c6de29640 1 -- 172.21.15.67:0/1672262905 shutdown_connections 2024-09-21T06:36:03.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.447+0000 7f9c6de29640 1 --2- 172.21.15.67:0/1672262905 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9c500b65d0 0x7f9c500b8ae0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:03.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.447+0000 7f9c6de29640 1 --2- 172.21.15.67:0/1672262905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c68108cf0 0x7f9c6819fc00 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:03.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.447+0000 7f9c6de29640 1 --2- 172.21.15.67:0/1672262905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c681083a0 0x7f9c6819a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:03.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.447+0000 7f9c6de29640 1 --2- 172.21.15.67:0/1672262905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c68102a60 0x7f9c6819a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:03.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.447+0000 7f9c6de29640 1 -- 172.21.15.67:0/1672262905 >> 172.21.15.67:0/1672262905 conn(0x7f9c68079730 msgr2=0x7f9c6806b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:03.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.447+0000 7f9c6de29640 1 -- 172.21.15.67:0/1672262905 shutdown_connections 2024-09-21T06:36:03.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:03.447+0000 7f9c6de29640 1 -- 172.21.15.67:0/1672262905 wait complete. 2024-09-21T06:36:03.764 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:03 smithi067 ceph-mon[31232]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:36:03.764 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:03 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1672262905' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T06:36:03.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:03 smithi203 ceph-mon[33735]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:36:03.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:03 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1672262905' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T06:36:04.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:03 smithi100 ceph-mon[33401]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:36:04.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:03 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1672262905' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T06:36:04.112 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738451 got 34359738456 for osd.0 2024-09-21T06:36:04.112 DEBUG:teuthology.parallel:result is None 2024-09-21T06:36:04.112 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T06:36:04.112 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T06:36:04.375 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:36:04.778 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:04 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]} : dispatch 2024-09-21T06:36:04.778 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:04 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]} : dispatch 2024-09-21T06:36:04.778 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:04 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]} : dispatch 2024-09-21T06:36:04.778 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:04 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]} : dispatch 2024-09-21T06:36:04.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.869+0000 7fcd21ccf640 1 -- 172.21.15.67:0/3176549727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd1c102a60 msgr2=0x7fcd1c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:04.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.869+0000 7fcd21ccf640 1 --2- 172.21.15.67:0/3176549727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd1c102a60 0x7fcd1c102e70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009900 tx=0x7fcd0c02a410 comp rx=0 tx=0).stop 2024-09-21T06:36:04.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.869+0000 7fcd21ccf640 1 -- 172.21.15.67:0/3176549727 shutdown_connections 2024-09-21T06:36:04.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.869+0000 7fcd21ccf640 1 --2- 172.21.15.67:0/3176549727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd1c108cf0 0x7fcd1c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:04.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.869+0000 7fcd21ccf640 1 --2- 172.21.15.67:0/3176549727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd1c1083a0 0x7fcd1c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:04.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.869+0000 7fcd21ccf640 1 --2- 172.21.15.67:0/3176549727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd1c102a60 0x7fcd1c102e70 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:04.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.869+0000 7fcd21ccf640 1 -- 172.21.15.67:0/3176549727 >> 172.21.15.67:0/3176549727 conn(0x7fcd1c079730 msgr2=0x7fcd1c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:04.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.869+0000 7fcd21ccf640 1 -- 172.21.15.67:0/3176549727 shutdown_connections 2024-09-21T06:36:04.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.869+0000 7fcd21ccf640 1 -- 172.21.15.67:0/3176549727 wait complete. 2024-09-21T06:36:04.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.870+0000 7fcd21ccf640 1 Processor -- start 2024-09-21T06:36:04.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.870+0000 7fcd21ccf640 1 -- start start 2024-09-21T06:36:04.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.870+0000 7fcd21ccf640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd1c102a60 0x7fcd1c19c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:04.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.870+0000 7fcd21ccf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd1c1083a0 0x7fcd1c19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:04.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.870+0000 7fcd21ccf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd1c108cf0 0x7fcd1c1a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:04.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.870+0000 7fcd21ccf640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c112810 con 0x7fcd1c102a60 2024-09-21T06:36:04.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.870+0000 7fcd21ccf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c112690 con 0x7fcd1c1083a0 2024-09-21T06:36:04.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.870+0000 7fcd21ccf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c112990 con 0x7fcd1c108cf0 2024-09-21T06:36:04.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.870+0000 7fcd1b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd1c102a60 0x7fcd1c19c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:04.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.870+0000 7fcd1b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd1c102a60 0x7fcd1c19c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44600/0 (socket says 172.21.15.67:44600) 2024-09-21T06:36:04.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.870+0000 7fcd1b7fe640 1 -- 172.21.15.67:0/631429837 learned_addr learned my addr 172.21.15.67:0/631429837 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:04.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.871+0000 7fcd1affd640 1 --2- 172.21.15.67:0/631429837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd1c1083a0 0x7fcd1c19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:04.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.871+0000 7fcd1bfff640 1 --2- 172.21.15.67:0/631429837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd1c108cf0 0x7fcd1c1a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:04.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.871+0000 7fcd1b7fe640 1 -- 172.21.15.67:0/631429837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd1c108cf0 msgr2=0x7fcd1c1a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:04.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.871+0000 7fcd1b7fe640 1 --2- 172.21.15.67:0/631429837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd1c108cf0 0x7fcd1c1a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:04.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.871+0000 7fcd1b7fe640 1 -- 172.21.15.67:0/631429837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd1c1083a0 msgr2=0x7fcd1c19cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:04.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.871+0000 7fcd1b7fe640 1 --2- 172.21.15.67:0/631429837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd1c1083a0 0x7fcd1c19cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:04.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.871+0000 7fcd1b7fe640 1 -- 172.21.15.67:0/631429837 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd1c1a2570 con 0x7fcd1c102a60 2024-09-21T06:36:04.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.871+0000 7fcd1affd640 1 --2- 172.21.15.67:0/631429837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd1c1083a0 0x7fcd1c19cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:36:04.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.871+0000 7fcd1bfff640 1 --2- 172.21.15.67:0/631429837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd1c108cf0 0x7fcd1c1a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:36:04.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.871+0000 7fcd1b7fe640 1 --2- 172.21.15.67:0/631429837 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd1c102a60 0x7fcd1c19c600 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009230 tx=0x7fcd0c0319a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:04.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.871+0000 7fcd18ff9640 1 -- 172.21.15.67:0/631429837 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c03f030 con 0x7fcd1c102a60 2024-09-21T06:36:04.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.871+0000 7fcd21ccf640 1 -- 172.21.15.67:0/631429837 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd1c1a2800 con 0x7fcd1c102a60 2024-09-21T06:36:04.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.871+0000 7fcd18ff9640 1 -- 172.21.15.67:0/631429837 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd0c0043d0 con 0x7fcd1c102a60 2024-09-21T06:36:04.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.871+0000 7fcd18ff9640 1 -- 172.21.15.67:0/631429837 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c035070 con 0x7fcd1c102a60 2024-09-21T06:36:04.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.871+0000 7fcd21ccf640 1 -- 172.21.15.67:0/631429837 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd1c1a8fc0 con 0x7fcd1c102a60 2024-09-21T06:36:04.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.872+0000 7fcd21ccf640 1 -- 172.21.15.67:0/631429837 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd1c103ce0 con 0x7fcd1c102a60 2024-09-21T06:36:04.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.875+0000 7fcd18ff9640 1 -- 172.21.15.67:0/631429837 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fcd0c042030 con 0x7fcd1c102a60 2024-09-21T06:36:04.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.876+0000 7fcd18ff9640 1 --2- 172.21.15.67:0/631429837 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fcd040b65a0 0x7fcd040b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:04.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.876+0000 7fcd1affd640 1 --2- 172.21.15.67:0/631429837 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fcd040b65a0 0x7fcd040b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:04.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.876+0000 7fcd18ff9640 1 -- 172.21.15.67:0/631429837 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9208+0+0 (secure 0 0 0) 0x7fcd0c108130 con 0x7fcd1c102a60 2024-09-21T06:36:04.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.876+0000 7fcd1affd640 1 --2- 172.21.15.67:0/631429837 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fcd040b65a0 0x7fcd040b8ab0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c100ee0 tx=0x7fcd10005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:04.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:04.879+0000 7fcd18ff9640 1 -- 172.21.15.67:0/631429837 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcd0c0ce490 con 0x7fcd1c102a60 2024-09-21T06:36:04.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:04 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]} : dispatch 2024-09-21T06:36:04.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:04 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]} : dispatch 2024-09-21T06:36:04.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:04 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]} : dispatch 2024-09-21T06:36:04.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:04 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]} : dispatch 2024-09-21T06:36:05.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:04 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]} : dispatch 2024-09-21T06:36:05.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:04 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]} : dispatch 2024-09-21T06:36:05.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:04 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]} : dispatch 2024-09-21T06:36:05.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:04 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]} : dispatch 2024-09-21T06:36:05.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:05.063+0000 7fcd21ccf640 1 -- 172.21.15.67:0/631429837 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcd1c06b640 con 0x7fcd040b65a0 2024-09-21T06:36:05.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:05.069+0000 7fcd18ff9640 1 -- 172.21.15.67:0/631429837 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340527 (secure 0 0 0) 0x7fcd1c06b640 con 0x7fcd040b65a0 2024-09-21T06:36:05.075 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:36:05.076 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-09-21T06:36:05.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:05.072+0000 7fcd21ccf640 1 -- 172.21.15.67:0/631429837 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fcd040b65a0 msgr2=0x7fcd040b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:05.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:05.072+0000 7fcd21ccf640 1 --2- 172.21.15.67:0/631429837 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fcd040b65a0 0x7fcd040b8ab0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c100ee0 tx=0x7fcd10005f90 comp rx=0 tx=0).stop 2024-09-21T06:36:05.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:05.072+0000 7fcd21ccf640 1 -- 172.21.15.67:0/631429837 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd1c102a60 msgr2=0x7fcd1c19c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:05.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:05.072+0000 7fcd21ccf640 1 --2- 172.21.15.67:0/631429837 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd1c102a60 0x7fcd1c19c600 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009230 tx=0x7fcd0c0319a0 comp rx=0 tx=0).stop 2024-09-21T06:36:05.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:05.072+0000 7fcd21ccf640 1 -- 172.21.15.67:0/631429837 shutdown_connections 2024-09-21T06:36:05.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:05.072+0000 7fcd21ccf640 1 --2- 172.21.15.67:0/631429837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd1c1083a0 0x7fcd1c19cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:05.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:05.072+0000 7fcd21ccf640 1 --2- 172.21.15.67:0/631429837 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fcd040b65a0 0x7fcd040b8ab0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:05.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:05.072+0000 7fcd21ccf640 1 --2- 172.21.15.67:0/631429837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd1c108cf0 0x7fcd1c1a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:05.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:05.072+0000 7fcd21ccf640 1 --2- 172.21.15.67:0/631429837 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd1c102a60 0x7fcd1c19c600 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:05.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:05.072+0000 7fcd21ccf640 1 -- 172.21.15.67:0/631429837 >> 172.21.15.67:0/631429837 conn(0x7fcd1c079730 msgr2=0x7fcd1c06af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:05.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:05.072+0000 7fcd21ccf640 1 -- 172.21.15.67:0/631429837 shutdown_connections 2024-09-21T06:36:05.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:05.072+0000 7fcd21ccf640 1 -- 172.21.15.67:0/631429837 wait complete. 2024-09-21T06:36:05.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:05 smithi067 ceph-mon[31232]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:36:05.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:05 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-21T06:36:05.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:05 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-09-21T06:36:05.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:05 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-21T06:36:05.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:05 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-21T06:36:05.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:05 smithi067 ceph-mon[31232]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T06:36:05.948 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-09-21T06:36:04.619927+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660185,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":195,"num_read_kb":194,"num_write":210,"num_write_kb":2248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":165,"ondisk_log_size":165,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364448,"kb_used_data":11024,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577376,"statfs":{"total":1151940427776,"available":1151567233024,"internally_reserved":0,"allocated":11288576,"data_stored":7564385,"data_compressed":390735,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.755913"},"pg_stats":[{"pgid":"2.1f","version":"75'4","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.114370+0000","last_change":"2024-09-21T06:34:30.633366+0000","last_active":"2024-09-21T06:35:05.114370+0000","last_peered":"2024-09-21T06:35:05.114370+0000","last_clean":"2024-09-21T06:35:05.114370+0000","last_became_active":"2024-09-21T06:34:30.632929+0000","last_became_peered":"2024-09-21T06:34:30.632929+0000","last_unstale":"2024-09-21T06:35:05.114370+0000","last_undegraded":"2024-09-21T06:35:05.114370+0000","last_fullsized":"2024-09-21T06:35:05.114370+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:47.722461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.272950+0000","last_change":"2024-09-21T06:34:32.632989+0000","last_active":"2024-09-21T06:35:05.272950+0000","last_peered":"2024-09-21T06:35:05.272950+0000","last_clean":"2024-09-21T06:35:05.272950+0000","last_became_active":"2024-09-21T06:34:32.632024+0000","last_became_peered":"2024-09-21T06:34:32.632024+0000","last_unstale":"2024-09-21T06:35:05.272950+0000","last_undegraded":"2024-09-21T06:35:05.272950+0000","last_fullsized":"2024-09-21T06:35:05.272950+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:13.669613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.274531+0000","last_change":"2024-09-21T06:35:05.274434+0000","last_active":"2024-09-21T06:35:05.274531+0000","last_peered":"2024-09-21T06:35:05.274531+0000","last_clean":"2024-09-21T06:35:05.274531+0000","last_became_active":"2024-09-21T06:35:05.273833+0000","last_became_peered":"2024-09-21T06:35:05.273833+0000","last_unstale":"2024-09-21T06:35:05.274531+0000","last_undegraded":"2024-09-21T06:35:05.274531+0000","last_fullsized":"2024-09-21T06:35:05.274531+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:48.847861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.273266+0000","last_change":"2024-09-21T06:34:30.946309+0000","last_active":"2024-09-21T06:35:05.273266+0000","last_peered":"2024-09-21T06:35:05.273266+0000","last_clean":"2024-09-21T06:35:05.273266+0000","last_became_active":"2024-09-21T06:34:30.945913+0000","last_became_peered":"2024-09-21T06:34:30.945913+0000","last_unstale":"2024-09-21T06:35:05.273266+0000","last_undegraded":"2024-09-21T06:35:05.273266+0000","last_fullsized":"2024-09-21T06:35:05.273266+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:21.676524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000669+0000","last_change":"2024-09-21T06:34:32.637636+0000","last_active":"2024-09-21T06:35:05.000669+0000","last_peered":"2024-09-21T06:35:05.000669+0000","last_clean":"2024-09-21T06:35:05.000669+0000","last_became_active":"2024-09-21T06:34:32.636534+0000","last_became_peered":"2024-09-21T06:34:32.636534+0000","last_unstale":"2024-09-21T06:35:05.000669+0000","last_undegraded":"2024-09-21T06:35:05.000669+0000","last_fullsized":"2024-09-21T06:35:05.000669+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:05.802118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.274538+0000","last_change":"2024-09-21T06:35:05.274419+0000","last_active":"2024-09-21T06:35:05.274538+0000","last_peered":"2024-09-21T06:35:05.274538+0000","last_clean":"2024-09-21T06:35:05.274538+0000","last_became_active":"2024-09-21T06:35:05.273865+0000","last_became_peered":"2024-09-21T06:35:05.273865+0000","last_unstale":"2024-09-21T06:35:05.274538+0000","last_undegraded":"2024-09-21T06:35:05.274538+0000","last_fullsized":"2024-09-21T06:35:05.274538+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:04.490355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.004199+0000","last_change":"2024-09-21T06:34:30.636856+0000","last_active":"2024-09-21T06:35:05.004199+0000","last_peered":"2024-09-21T06:35:05.004199+0000","last_clean":"2024-09-21T06:35:05.004199+0000","last_became_active":"2024-09-21T06:34:30.636460+0000","last_became_peered":"2024-09-21T06:34:30.636460+0000","last_unstale":"2024-09-21T06:35:05.004199+0000","last_undegraded":"2024-09-21T06:35:05.004199+0000","last_fullsized":"2024-09-21T06:35:05.004199+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:58.917851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995796+0000","last_change":"2024-09-21T06:34:32.650303+0000","last_active":"2024-09-21T06:35:04.995796+0000","last_peered":"2024-09-21T06:35:04.995796+0000","last_clean":"2024-09-21T06:35:04.995796+0000","last_became_active":"2024-09-21T06:34:32.647189+0000","last_became_peered":"2024-09-21T06:34:32.647189+0000","last_unstale":"2024-09-21T06:35:04.995796+0000","last_undegraded":"2024-09-21T06:35:04.995796+0000","last_fullsized":"2024-09-21T06:35:04.995796+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:14.889384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.996942+0000","last_change":"2024-09-21T06:34:39.713139+0000","last_active":"2024-09-21T06:35:04.996942+0000","last_peered":"2024-09-21T06:35:04.996942+0000","last_clean":"2024-09-21T06:35:04.996942+0000","last_became_active":"2024-09-21T06:34:39.712379+0000","last_became_peered":"2024-09-21T06:34:39.712379+0000","last_unstale":"2024-09-21T06:35:04.996942+0000","last_undegraded":"2024-09-21T06:35:04.996942+0000","last_fullsized":"2024-09-21T06:35:04.996942+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:53.100529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.004039+0000","last_change":"2024-09-21T06:34:30.633850+0000","last_active":"2024-09-21T06:35:05.004039+0000","last_peered":"2024-09-21T06:35:05.004039+0000","last_clean":"2024-09-21T06:35:05.004039+0000","last_became_active":"2024-09-21T06:34:30.633468+0000","last_became_peered":"2024-09-21T06:34:30.633468+0000","last_unstale":"2024-09-21T06:35:05.004039+0000","last_undegraded":"2024-09-21T06:35:05.004039+0000","last_fullsized":"2024-09-21T06:35:05.004039+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:59.095803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.005361+0000","last_change":"2024-09-21T06:35:05.005240+0000","last_active":"2024-09-21T06:35:05.005361+0000","last_peered":"2024-09-21T06:35:05.005361+0000","last_clean":"2024-09-21T06:35:05.005361+0000","last_became_active":"2024-09-21T06:35:05.004849+0000","last_became_peered":"2024-09-21T06:35:05.004849+0000","last_unstale":"2024-09-21T06:35:05.005361+0000","last_undegraded":"2024-09-21T06:35:05.005361+0000","last_fullsized":"2024-09-21T06:35:05.005361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:28.849604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.274638+0000","last_change":"2024-09-21T06:35:05.274437+0000","last_active":"2024-09-21T06:35:05.274638+0000","last_peered":"2024-09-21T06:35:05.274638+0000","last_clean":"2024-09-21T06:35:05.274638+0000","last_became_active":"2024-09-21T06:35:05.273865+0000","last_became_peered":"2024-09-21T06:35:05.273865+0000","last_unstale":"2024-09-21T06:35:05.274638+0000","last_undegraded":"2024-09-21T06:35:05.274638+0000","last_fullsized":"2024-09-21T06:35:05.274638+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:43.339504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029725+0000","last_change":"2024-09-21T06:34:32.645630+0000","last_active":"2024-09-21T06:35:05.029725+0000","last_peered":"2024-09-21T06:35:05.029725+0000","last_clean":"2024-09-21T06:35:05.029725+0000","last_became_active":"2024-09-21T06:34:32.645237+0000","last_became_peered":"2024-09-21T06:34:32.645237+0000","last_unstale":"2024-09-21T06:35:05.029725+0000","last_undegraded":"2024-09-21T06:35:05.029725+0000","last_fullsized":"2024-09-21T06:35:05.029725+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:25.064318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.003627+0000","last_change":"2024-09-21T06:34:30.636609+0000","last_active":"2024-09-21T06:35:05.003627+0000","last_peered":"2024-09-21T06:35:05.003627+0000","last_clean":"2024-09-21T06:35:05.003627+0000","last_became_active":"2024-09-21T06:34:30.636217+0000","last_became_peered":"2024-09-21T06:34:30.636217+0000","last_unstale":"2024-09-21T06:35:05.003627+0000","last_undegraded":"2024-09-21T06:35:05.003627+0000","last_fullsized":"2024-09-21T06:35:05.003627+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:32.541307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997116+0000","last_change":"2024-09-21T06:34:39.714177+0000","last_active":"2024-09-21T06:35:04.997116+0000","last_peered":"2024-09-21T06:35:04.997116+0000","last_clean":"2024-09-21T06:35:04.997116+0000","last_became_active":"2024-09-21T06:34:39.713713+0000","last_became_peered":"2024-09-21T06:34:39.713713+0000","last_unstale":"2024-09-21T06:35:04.997116+0000","last_undegraded":"2024-09-21T06:35:04.997116+0000","last_fullsized":"2024-09-21T06:35:04.997116+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:07.042240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.272625+0000","last_change":"2024-09-21T06:34:32.636297+0000","last_active":"2024-09-21T06:35:05.272625+0000","last_peered":"2024-09-21T06:35:05.272625+0000","last_clean":"2024-09-21T06:35:05.272625+0000","last_became_active":"2024-09-21T06:34:32.635930+0000","last_became_peered":"2024-09-21T06:34:32.635930+0000","last_unstale":"2024-09-21T06:35:05.272625+0000","last_undegraded":"2024-09-21T06:35:05.272625+0000","last_fullsized":"2024-09-21T06:35:05.272625+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:43.091682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995838+0000","last_change":"2024-09-21T06:34:30.631405+0000","last_active":"2024-09-21T06:35:04.995838+0000","last_peered":"2024-09-21T06:35:04.995838+0000","last_clean":"2024-09-21T06:35:04.995838+0000","last_became_active":"2024-09-21T06:34:30.630512+0000","last_became_peered":"2024-09-21T06:34:30.630512+0000","last_unstale":"2024-09-21T06:35:04.995838+0000","last_undegraded":"2024-09-21T06:35:04.995838+0000","last_fullsized":"2024-09-21T06:35:04.995838+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:43.412695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"83'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997680+0000","last_change":"2024-09-21T06:34:39.710573+0000","last_active":"2024-09-21T06:35:04.997680+0000","last_peered":"2024-09-21T06:35:04.997680+0000","last_clean":"2024-09-21T06:35:04.997680+0000","last_became_active":"2024-09-21T06:34:39.708969+0000","last_became_peered":"2024-09-21T06:34:39.708969+0000","last_unstale":"2024-09-21T06:35:04.997680+0000","last_undegraded":"2024-09-21T06:35:04.997680+0000","last_fullsized":"2024-09-21T06:35:04.997680+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:34.847587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000559+0000","last_change":"2024-09-21T06:34:32.641085+0000","last_active":"2024-09-21T06:35:05.000559+0000","last_peered":"2024-09-21T06:35:05.000559+0000","last_clean":"2024-09-21T06:35:05.000559+0000","last_became_active":"2024-09-21T06:34:32.640723+0000","last_became_peered":"2024-09-21T06:34:32.640723+0000","last_unstale":"2024-09-21T06:35:05.000559+0000","last_undegraded":"2024-09-21T06:35:05.000559+0000","last_fullsized":"2024-09-21T06:35:05.000559+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:21.410262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.273157+0000","last_change":"2024-09-21T06:34:30.626719+0000","last_active":"2024-09-21T06:35:05.273157+0000","last_peered":"2024-09-21T06:35:05.273157+0000","last_clean":"2024-09-21T06:35:05.273157+0000","last_became_active":"2024-09-21T06:34:30.626312+0000","last_became_peered":"2024-09-21T06:34:30.626312+0000","last_unstale":"2024-09-21T06:35:05.273157+0000","last_undegraded":"2024-09-21T06:35:05.273157+0000","last_fullsized":"2024-09-21T06:35:05.273157+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:53.938914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.114020+0000","last_change":"2024-09-21T06:34:39.711172+0000","last_active":"2024-09-21T06:35:05.114020+0000","last_peered":"2024-09-21T06:35:05.114020+0000","last_clean":"2024-09-21T06:35:05.114020+0000","last_became_active":"2024-09-21T06:34:39.708424+0000","last_became_peered":"2024-09-21T06:34:39.708424+0000","last_unstale":"2024-09-21T06:35:05.114020+0000","last_undegraded":"2024-09-21T06:35:05.114020+0000","last_fullsized":"2024-09-21T06:35:05.114020+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:43.918355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.003179+0000","last_change":"2024-09-21T06:34:32.641463+0000","last_active":"2024-09-21T06:35:05.003179+0000","last_peered":"2024-09-21T06:35:05.003179+0000","last_clean":"2024-09-21T06:35:05.003179+0000","last_became_active":"2024-09-21T06:34:32.641086+0000","last_became_peered":"2024-09-21T06:34:32.641086+0000","last_unstale":"2024-09-21T06:35:05.003179+0000","last_undegraded":"2024-09-21T06:35:05.003179+0000","last_fullsized":"2024-09-21T06:35:05.003179+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:58.991374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995541+0000","last_change":"2024-09-21T06:34:30.946698+0000","last_active":"2024-09-21T06:35:04.995541+0000","last_peered":"2024-09-21T06:35:04.995541+0000","last_clean":"2024-09-21T06:35:04.995541+0000","last_became_active":"2024-09-21T06:34:30.946325+0000","last_became_peered":"2024-09-21T06:34:30.946325+0000","last_unstale":"2024-09-21T06:35:04.995541+0000","last_undegraded":"2024-09-21T06:35:04.995541+0000","last_fullsized":"2024-09-21T06:35:04.995541+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:22.639265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997471+0000","last_change":"2024-09-21T06:34:39.712429+0000","last_active":"2024-09-21T06:35:04.997471+0000","last_peered":"2024-09-21T06:35:04.997471+0000","last_clean":"2024-09-21T06:35:04.997471+0000","last_became_active":"2024-09-21T06:34:39.711823+0000","last_became_peered":"2024-09-21T06:34:39.711823+0000","last_unstale":"2024-09-21T06:35:04.997471+0000","last_undegraded":"2024-09-21T06:35:04.997471+0000","last_fullsized":"2024-09-21T06:35:04.997471+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:03:49.513991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.274438+0000","last_change":"2024-09-21T06:35:05.274326+0000","last_active":"2024-09-21T06:35:05.274438+0000","last_peered":"2024-09-21T06:35:05.274438+0000","last_clean":"2024-09-21T06:35:05.274438+0000","last_became_active":"2024-09-21T06:35:05.273884+0000","last_became_peered":"2024-09-21T06:35:05.273884+0000","last_unstale":"2024-09-21T06:35:05.274438+0000","last_undegraded":"2024-09-21T06:35:05.274438+0000","last_fullsized":"2024-09-21T06:35:05.274438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:03.219765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997399+0000","last_change":"2024-09-21T06:34:30.676794+0000","last_active":"2024-09-21T06:35:04.997399+0000","last_peered":"2024-09-21T06:35:04.997399+0000","last_clean":"2024-09-21T06:35:04.997399+0000","last_became_active":"2024-09-21T06:34:30.676400+0000","last_became_peered":"2024-09-21T06:34:30.676400+0000","last_unstale":"2024-09-21T06:35:04.997399+0000","last_undegraded":"2024-09-21T06:35:04.997399+0000","last_fullsized":"2024-09-21T06:35:04.997399+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:23.681363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995836+0000","last_change":"2024-09-21T06:34:39.708980+0000","last_active":"2024-09-21T06:35:04.995836+0000","last_peered":"2024-09-21T06:35:04.995836+0000","last_clean":"2024-09-21T06:35:04.995836+0000","last_became_active":"2024-09-21T06:34:39.707748+0000","last_became_peered":"2024-09-21T06:34:39.707748+0000","last_unstale":"2024-09-21T06:35:04.995836+0000","last_undegraded":"2024-09-21T06:35:04.995836+0000","last_fullsized":"2024-09-21T06:35:04.995836+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:16.812226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.273262+0000","last_change":"2024-09-21T06:34:32.638337+0000","last_active":"2024-09-21T06:35:05.273262+0000","last_peered":"2024-09-21T06:35:05.273262+0000","last_clean":"2024-09-21T06:35:05.273262+0000","last_became_active":"2024-09-21T06:34:32.638013+0000","last_became_peered":"2024-09-21T06:34:32.638013+0000","last_unstale":"2024-09-21T06:35:05.273262+0000","last_undegraded":"2024-09-21T06:35:05.273262+0000","last_fullsized":"2024-09-21T06:35:05.273262+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:26.321190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"83'9","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:23.662935+0000","last_change":"2024-09-21T06:34:30.631086+0000","last_active":"2024-09-21T06:35:23.662935+0000","last_peered":"2024-09-21T06:35:23.662935+0000","last_clean":"2024-09-21T06:35:23.662935+0000","last_became_active":"2024-09-21T06:34:30.630706+0000","last_became_peered":"2024-09-21T06:34:30.630706+0000","last_unstale":"2024-09-21T06:35:23.662935+0000","last_undegraded":"2024-09-21T06:35:23.662935+0000","last_fullsized":"2024-09-21T06:35:23.662935+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:30.260359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.272633+0000","last_change":"2024-09-21T06:34:39.709097+0000","last_active":"2024-09-21T06:35:05.272633+0000","last_peered":"2024-09-21T06:35:05.272633+0000","last_clean":"2024-09-21T06:35:05.272633+0000","last_became_active":"2024-09-21T06:34:39.708609+0000","last_became_peered":"2024-09-21T06:34:39.708609+0000","last_unstale":"2024-09-21T06:35:05.272633+0000","last_undegraded":"2024-09-21T06:35:05.272633+0000","last_fullsized":"2024-09-21T06:35:05.272633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:48.842174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995363+0000","last_change":"2024-09-21T06:34:32.636816+0000","last_active":"2024-09-21T06:35:04.995363+0000","last_peered":"2024-09-21T06:35:04.995363+0000","last_clean":"2024-09-21T06:35:04.995363+0000","last_became_active":"2024-09-21T06:34:32.636429+0000","last_became_peered":"2024-09-21T06:34:32.636429+0000","last_unstale":"2024-09-21T06:35:04.995363+0000","last_undegraded":"2024-09-21T06:35:04.995363+0000","last_fullsized":"2024-09-21T06:35:04.995363+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:49.010491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.004246+0000","last_change":"2024-09-21T06:34:30.947481+0000","last_active":"2024-09-21T06:35:05.004246+0000","last_peered":"2024-09-21T06:35:05.004246+0000","last_clean":"2024-09-21T06:35:05.004246+0000","last_became_active":"2024-09-21T06:34:30.947040+0000","last_became_peered":"2024-09-21T06:34:30.947040+0000","last_unstale":"2024-09-21T06:35:05.004246+0000","last_undegraded":"2024-09-21T06:35:05.004246+0000","last_fullsized":"2024-09-21T06:35:05.004246+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:52.813476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000546+0000","last_change":"2024-09-21T06:34:39.708674+0000","last_active":"2024-09-21T06:35:05.000546+0000","last_peered":"2024-09-21T06:35:05.000546+0000","last_clean":"2024-09-21T06:35:05.000546+0000","last_became_active":"2024-09-21T06:34:39.708176+0000","last_became_peered":"2024-09-21T06:34:39.708176+0000","last_unstale":"2024-09-21T06:35:05.000546+0000","last_undegraded":"2024-09-21T06:35:05.000546+0000","last_fullsized":"2024-09-21T06:35:05.000546+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:57.070792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000844+0000","last_change":"2024-09-21T06:34:32.636381+0000","last_active":"2024-09-21T06:35:05.000844+0000","last_peered":"2024-09-21T06:35:05.000844+0000","last_clean":"2024-09-21T06:35:05.000844+0000","last_became_active":"2024-09-21T06:34:32.635937+0000","last_became_peered":"2024-09-21T06:34:32.635937+0000","last_unstale":"2024-09-21T06:35:05.000844+0000","last_undegraded":"2024-09-21T06:35:05.000844+0000","last_fullsized":"2024-09-21T06:35:05.000844+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:52.870676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.114033+0000","last_change":"2024-09-21T06:34:30.947160+0000","last_active":"2024-09-21T06:35:05.114033+0000","last_peered":"2024-09-21T06:35:05.114033+0000","last_clean":"2024-09-21T06:35:05.114033+0000","last_became_active":"2024-09-21T06:34:30.946736+0000","last_became_peered":"2024-09-21T06:34:30.946736+0000","last_unstale":"2024-09-21T06:35:05.114033+0000","last_undegraded":"2024-09-21T06:35:05.114033+0000","last_fullsized":"2024-09-21T06:35:05.114033+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:47.425481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.006145+0000","last_change":"2024-09-21T06:35:05.006032+0000","last_active":"2024-09-21T06:35:05.006145+0000","last_peered":"2024-09-21T06:35:05.006145+0000","last_clean":"2024-09-21T06:35:05.006145+0000","last_became_active":"2024-09-21T06:35:05.005429+0000","last_became_peered":"2024-09-21T06:35:05.005429+0000","last_unstale":"2024-09-21T06:35:05.006145+0000","last_undegraded":"2024-09-21T06:35:05.006145+0000","last_fullsized":"2024-09-21T06:35:05.006145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:04.138097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000613+0000","last_change":"2024-09-21T06:34:32.648256+0000","last_active":"2024-09-21T06:35:05.000613+0000","last_peered":"2024-09-21T06:35:05.000613+0000","last_clean":"2024-09-21T06:35:05.000613+0000","last_became_active":"2024-09-21T06:34:32.647881+0000","last_became_peered":"2024-09-21T06:34:32.647881+0000","last_unstale":"2024-09-21T06:35:05.000613+0000","last_undegraded":"2024-09-21T06:35:05.000613+0000","last_fullsized":"2024-09-21T06:35:05.000613+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:14.590834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029849+0000","last_change":"2024-09-21T06:34:30.629918+0000","last_active":"2024-09-21T06:35:05.029849+0000","last_peered":"2024-09-21T06:35:05.029849+0000","last_clean":"2024-09-21T06:35:05.029849+0000","last_became_active":"2024-09-21T06:34:30.628785+0000","last_became_peered":"2024-09-21T06:34:30.628785+0000","last_unstale":"2024-09-21T06:35:05.029849+0000","last_undegraded":"2024-09-21T06:35:05.029849+0000","last_fullsized":"2024-09-21T06:35:05.029849+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:05.333406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995102+0000","last_change":"2024-09-21T06:34:39.710073+0000","last_active":"2024-09-21T06:35:04.995102+0000","last_peered":"2024-09-21T06:35:04.995102+0000","last_clean":"2024-09-21T06:35:04.995102+0000","last_became_active":"2024-09-21T06:34:39.708986+0000","last_became_peered":"2024-09-21T06:34:39.708986+0000","last_unstale":"2024-09-21T06:35:04.995102+0000","last_undegraded":"2024-09-21T06:35:04.995102+0000","last_fullsized":"2024-09-21T06:35:04.995102+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:46.379902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.998454+0000","last_change":"2024-09-21T06:34:32.644224+0000","last_active":"2024-09-21T06:35:04.998454+0000","last_peered":"2024-09-21T06:35:04.998454+0000","last_clean":"2024-09-21T06:35:04.998454+0000","last_became_active":"2024-09-21T06:34:32.643850+0000","last_became_peered":"2024-09-21T06:34:32.643850+0000","last_unstale":"2024-09-21T06:35:04.998454+0000","last_undegraded":"2024-09-21T06:35:04.998454+0000","last_fullsized":"2024-09-21T06:35:04.998454+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:24:33.188537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.994875+0000","last_change":"2024-09-21T06:34:30.946334+0000","last_active":"2024-09-21T06:35:04.994875+0000","last_peered":"2024-09-21T06:35:04.994875+0000","last_clean":"2024-09-21T06:35:04.994875+0000","last_became_active":"2024-09-21T06:34:30.945908+0000","last_became_peered":"2024-09-21T06:34:30.945908+0000","last_unstale":"2024-09-21T06:35:04.994875+0000","last_undegraded":"2024-09-21T06:35:04.994875+0000","last_fullsized":"2024-09-21T06:35:04.994875+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:46.746527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995193+0000","last_change":"2024-09-21T06:34:39.707909+0000","last_active":"2024-09-21T06:35:04.995193+0000","last_peered":"2024-09-21T06:35:04.995193+0000","last_clean":"2024-09-21T06:35:04.995193+0000","last_became_active":"2024-09-21T06:34:39.707492+0000","last_became_peered":"2024-09-21T06:34:39.707492+0000","last_unstale":"2024-09-21T06:35:04.995193+0000","last_undegraded":"2024-09-21T06:35:04.995193+0000","last_fullsized":"2024-09-21T06:35:04.995193+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:56.201212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.994774+0000","last_change":"2024-09-21T06:34:32.635870+0000","last_active":"2024-09-21T06:35:04.994774+0000","last_peered":"2024-09-21T06:35:04.994774+0000","last_clean":"2024-09-21T06:35:04.994774+0000","last_became_active":"2024-09-21T06:34:32.635252+0000","last_became_peered":"2024-09-21T06:34:32.635252+0000","last_unstale":"2024-09-21T06:35:04.994774+0000","last_undegraded":"2024-09-21T06:35:04.994774+0000","last_fullsized":"2024-09-21T06:35:04.994774+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:10.613033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029607+0000","last_change":"2024-09-21T06:34:30.634360+0000","last_active":"2024-09-21T06:35:05.029607+0000","last_peered":"2024-09-21T06:35:05.029607+0000","last_clean":"2024-09-21T06:35:05.029607+0000","last_became_active":"2024-09-21T06:34:30.633845+0000","last_became_peered":"2024-09-21T06:34:30.633845+0000","last_unstale":"2024-09-21T06:35:05.029607+0000","last_undegraded":"2024-09-21T06:35:05.029607+0000","last_fullsized":"2024-09-21T06:35:05.029607+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:34:53.388670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999643+0000","last_change":"2024-09-21T06:34:39.719620+0000","last_active":"2024-09-21T06:35:04.999643+0000","last_peered":"2024-09-21T06:35:04.999643+0000","last_clean":"2024-09-21T06:35:04.999643+0000","last_became_active":"2024-09-21T06:34:39.718786+0000","last_became_peered":"2024-09-21T06:34:39.718786+0000","last_unstale":"2024-09-21T06:35:04.999643+0000","last_undegraded":"2024-09-21T06:35:04.999643+0000","last_fullsized":"2024-09-21T06:35:04.999643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:43.524932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.273070+0000","last_change":"2024-09-21T06:34:32.634970+0000","last_active":"2024-09-21T06:35:05.273070+0000","last_peered":"2024-09-21T06:35:05.273070+0000","last_clean":"2024-09-21T06:35:05.273070+0000","last_became_active":"2024-09-21T06:34:32.634611+0000","last_became_peered":"2024-09-21T06:34:32.634611+0000","last_unstale":"2024-09-21T06:35:05.273070+0000","last_undegraded":"2024-09-21T06:35:05.273070+0000","last_fullsized":"2024-09-21T06:35:05.273070+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:37.855849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997918+0000","last_change":"2024-09-21T06:34:30.628123+0000","last_active":"2024-09-21T06:35:04.997918+0000","last_peered":"2024-09-21T06:35:04.997918+0000","last_clean":"2024-09-21T06:35:04.997918+0000","last_became_active":"2024-09-21T06:34:30.627498+0000","last_became_peered":"2024-09-21T06:34:30.627498+0000","last_unstale":"2024-09-21T06:35:04.997918+0000","last_undegraded":"2024-09-21T06:35:04.997918+0000","last_fullsized":"2024-09-21T06:35:04.997918+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:42.576441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":111,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029743+0000","last_change":"2024-09-21T06:33:16.856623+0000","last_active":"2024-09-21T06:35:05.029743+0000","last_peered":"2024-09-21T06:35:05.029743+0000","last_clean":"2024-09-21T06:35:05.029743+0000","last_became_active":"2024-09-21T06:33:16.850787+0000","last_became_peered":"2024-09-21T06:33:16.850787+0000","last_unstale":"2024-09-21T06:35:05.029743+0000","last_undegraded":"2024-09-21T06:35:05.029743+0000","last_fullsized":"2024-09-21T06:35:05.029743+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:50.283252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:50.283252+0000","last_clean_scrub_stamp":"2024-09-21T06:29:50.283252+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:29:20.711208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029992+0000","last_change":"2024-09-21T06:34:39.706472+0000","last_active":"2024-09-21T06:35:05.029992+0000","last_peered":"2024-09-21T06:35:05.029992+0000","last_clean":"2024-09-21T06:35:05.029992+0000","last_became_active":"2024-09-21T06:34:39.705724+0000","last_became_peered":"2024-09-21T06:34:39.705724+0000","last_unstale":"2024-09-21T06:35:05.029992+0000","last_undegraded":"2024-09-21T06:35:05.029992+0000","last_fullsized":"2024-09-21T06:35:05.029992+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:43.238704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.030085+0000","last_change":"2024-09-21T06:34:32.645944+0000","last_active":"2024-09-21T06:35:05.030085+0000","last_peered":"2024-09-21T06:35:05.030085+0000","last_clean":"2024-09-21T06:35:05.030085+0000","last_became_active":"2024-09-21T06:34:32.645134+0000","last_became_peered":"2024-09-21T06:34:32.645134+0000","last_unstale":"2024-09-21T06:35:05.030085+0000","last_undegraded":"2024-09-21T06:35:05.030085+0000","last_fullsized":"2024-09-21T06:35:05.030085+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:45.254164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.996320+0000","last_change":"2024-09-21T06:34:30.946296+0000","last_active":"2024-09-21T06:35:04.996320+0000","last_peered":"2024-09-21T06:35:04.996320+0000","last_clean":"2024-09-21T06:35:04.996320+0000","last_became_active":"2024-09-21T06:34:30.945777+0000","last_became_peered":"2024-09-21T06:34:30.945777+0000","last_unstale":"2024-09-21T06:35:04.996320+0000","last_undegraded":"2024-09-21T06:35:04.996320+0000","last_fullsized":"2024-09-21T06:35:04.996320+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:32:25.988512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995681+0000","last_change":"2024-09-21T06:34:39.705876+0000","last_active":"2024-09-21T06:35:04.995681+0000","last_peered":"2024-09-21T06:35:04.995681+0000","last_clean":"2024-09-21T06:35:04.995681+0000","last_became_active":"2024-09-21T06:34:39.705132+0000","last_became_peered":"2024-09-21T06:34:39.705132+0000","last_unstale":"2024-09-21T06:35:04.995681+0000","last_undegraded":"2024-09-21T06:35:04.995681+0000","last_fullsized":"2024-09-21T06:35:04.995681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:18.180244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029892+0000","last_change":"2024-09-21T06:34:32.638242+0000","last_active":"2024-09-21T06:35:05.029892+0000","last_peered":"2024-09-21T06:35:05.029892+0000","last_clean":"2024-09-21T06:35:05.029892+0000","last_became_active":"2024-09-21T06:34:32.637816+0000","last_became_peered":"2024-09-21T06:34:32.637816+0000","last_unstale":"2024-09-21T06:35:05.029892+0000","last_undegraded":"2024-09-21T06:35:05.029892+0000","last_fullsized":"2024-09-21T06:35:05.029892+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:40.926147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000042+0000","last_change":"2024-09-21T06:34:30.632173+0000","last_active":"2024-09-21T06:35:05.000042+0000","last_peered":"2024-09-21T06:35:05.000042+0000","last_clean":"2024-09-21T06:35:05.000042+0000","last_became_active":"2024-09-21T06:34:30.631638+0000","last_became_peered":"2024-09-21T06:34:30.631638+0000","last_unstale":"2024-09-21T06:35:05.000042+0000","last_undegraded":"2024-09-21T06:35:05.000042+0000","last_fullsized":"2024-09-21T06:35:05.000042+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:29.763272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000157+0000","last_change":"2024-09-21T06:34:39.714620+0000","last_active":"2024-09-21T06:35:05.000157+0000","last_peered":"2024-09-21T06:35:05.000157+0000","last_clean":"2024-09-21T06:35:05.000157+0000","last_became_active":"2024-09-21T06:34:39.714199+0000","last_became_peered":"2024-09-21T06:34:39.714199+0000","last_unstale":"2024-09-21T06:35:05.000157+0000","last_undegraded":"2024-09-21T06:35:05.000157+0000","last_fullsized":"2024-09-21T06:35:05.000157+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:37.041249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.030100+0000","last_change":"2024-09-21T06:34:32.645001+0000","last_active":"2024-09-21T06:35:05.030100+0000","last_peered":"2024-09-21T06:35:05.030100+0000","last_clean":"2024-09-21T06:35:05.030100+0000","last_became_active":"2024-09-21T06:34:32.644584+0000","last_became_peered":"2024-09-21T06:34:32.644584+0000","last_unstale":"2024-09-21T06:35:05.030100+0000","last_undegraded":"2024-09-21T06:35:05.030100+0000","last_fullsized":"2024-09-21T06:35:05.030100+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:47.098850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995420+0000","last_change":"2024-09-21T06:34:30.632593+0000","last_active":"2024-09-21T06:35:04.995420+0000","last_peered":"2024-09-21T06:35:04.995420+0000","last_clean":"2024-09-21T06:35:04.995420+0000","last_became_active":"2024-09-21T06:34:30.632170+0000","last_became_peered":"2024-09-21T06:34:30.632170+0000","last_unstale":"2024-09-21T06:35:04.995420+0000","last_undegraded":"2024-09-21T06:35:04.995420+0000","last_fullsized":"2024-09-21T06:35:04.995420+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:56.170655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.998214+0000","last_change":"2024-09-21T06:34:39.710172+0000","last_active":"2024-09-21T06:35:04.998214+0000","last_peered":"2024-09-21T06:35:04.998214+0000","last_clean":"2024-09-21T06:35:04.998214+0000","last_became_active":"2024-09-21T06:34:39.708876+0000","last_became_peered":"2024-09-21T06:34:39.708876+0000","last_unstale":"2024-09-21T06:35:04.998214+0000","last_undegraded":"2024-09-21T06:35:04.998214+0000","last_fullsized":"2024-09-21T06:35:04.998214+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:32.371873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000997+0000","last_change":"2024-09-21T06:34:32.651056+0000","last_active":"2024-09-21T06:35:05.000997+0000","last_peered":"2024-09-21T06:35:05.000997+0000","last_clean":"2024-09-21T06:35:05.000997+0000","last_became_active":"2024-09-21T06:34:32.650676+0000","last_became_peered":"2024-09-21T06:34:32.650676+0000","last_unstale":"2024-09-21T06:35:05.000997+0000","last_undegraded":"2024-09-21T06:35:05.000997+0000","last_fullsized":"2024-09-21T06:35:05.000997+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:34.599654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.004248+0000","last_change":"2024-09-21T06:34:30.632307+0000","last_active":"2024-09-21T06:35:05.004248+0000","last_peered":"2024-09-21T06:35:05.004248+0000","last_clean":"2024-09-21T06:35:05.004248+0000","last_became_active":"2024-09-21T06:34:30.631920+0000","last_became_peered":"2024-09-21T06:34:30.631920+0000","last_unstale":"2024-09-21T06:35:05.004248+0000","last_undegraded":"2024-09-21T06:35:05.004248+0000","last_fullsized":"2024-09-21T06:35:05.004248+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:46.940898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.998452+0000","last_change":"2024-09-21T06:34:39.715056+0000","last_active":"2024-09-21T06:35:04.998452+0000","last_peered":"2024-09-21T06:35:04.998452+0000","last_clean":"2024-09-21T06:35:04.998452+0000","last_became_active":"2024-09-21T06:34:39.714563+0000","last_became_peered":"2024-09-21T06:34:39.714563+0000","last_unstale":"2024-09-21T06:35:04.998452+0000","last_undegraded":"2024-09-21T06:35:04.998452+0000","last_fullsized":"2024-09-21T06:35:04.998452+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:48.567057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.998840+0000","last_change":"2024-09-21T06:34:32.646934+0000","last_active":"2024-09-21T06:35:04.998840+0000","last_peered":"2024-09-21T06:35:04.998840+0000","last_clean":"2024-09-21T06:35:04.998840+0000","last_became_active":"2024-09-21T06:34:32.646559+0000","last_became_peered":"2024-09-21T06:34:32.646559+0000","last_unstale":"2024-09-21T06:35:04.998840+0000","last_undegraded":"2024-09-21T06:35:04.998840+0000","last_fullsized":"2024-09-21T06:35:04.998840+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:37.802590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.004016+0000","last_change":"2024-09-21T06:34:30.946769+0000","last_active":"2024-09-21T06:35:05.004016+0000","last_peered":"2024-09-21T06:35:05.004016+0000","last_clean":"2024-09-21T06:35:05.004016+0000","last_became_active":"2024-09-21T06:34:30.946327+0000","last_became_peered":"2024-09-21T06:34:30.946327+0000","last_unstale":"2024-09-21T06:35:05.004016+0000","last_undegraded":"2024-09-21T06:35:05.004016+0000","last_fullsized":"2024-09-21T06:35:05.004016+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:52.055398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995322+0000","last_change":"2024-09-21T06:34:39.713398+0000","last_active":"2024-09-21T06:35:04.995322+0000","last_peered":"2024-09-21T06:35:04.995322+0000","last_clean":"2024-09-21T06:35:04.995322+0000","last_became_active":"2024-09-21T06:34:39.712954+0000","last_became_peered":"2024-09-21T06:34:39.712954+0000","last_unstale":"2024-09-21T06:35:04.995322+0000","last_undegraded":"2024-09-21T06:35:04.995322+0000","last_fullsized":"2024-09-21T06:35:04.995322+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:58.881014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029465+0000","last_change":"2024-09-21T06:34:32.640879+0000","last_active":"2024-09-21T06:35:05.029465+0000","last_peered":"2024-09-21T06:35:05.029465+0000","last_clean":"2024-09-21T06:35:05.029465+0000","last_became_active":"2024-09-21T06:34:32.640526+0000","last_became_peered":"2024-09-21T06:34:32.640526+0000","last_unstale":"2024-09-21T06:35:05.029465+0000","last_undegraded":"2024-09-21T06:35:05.029465+0000","last_fullsized":"2024-09-21T06:35:05.029465+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:12.327195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997068+0000","last_change":"2024-09-21T06:34:30.631449+0000","last_active":"2024-09-21T06:35:04.997068+0000","last_peered":"2024-09-21T06:35:04.997068+0000","last_clean":"2024-09-21T06:35:04.997068+0000","last_became_active":"2024-09-21T06:34:30.631081+0000","last_became_peered":"2024-09-21T06:34:30.631081+0000","last_unstale":"2024-09-21T06:35:04.997068+0000","last_undegraded":"2024-09-21T06:35:04.997068+0000","last_fullsized":"2024-09-21T06:35:04.997068+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:09.202696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.114180+0000","last_change":"2024-09-21T06:34:39.707379+0000","last_active":"2024-09-21T06:35:05.114180+0000","last_peered":"2024-09-21T06:35:05.114180+0000","last_clean":"2024-09-21T06:35:05.114180+0000","last_became_active":"2024-09-21T06:34:39.706979+0000","last_became_peered":"2024-09-21T06:34:39.706979+0000","last_unstale":"2024-09-21T06:35:05.114180+0000","last_undegraded":"2024-09-21T06:35:05.114180+0000","last_fullsized":"2024-09-21T06:35:05.114180+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:24.394233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029782+0000","last_change":"2024-09-21T06:34:32.649458+0000","last_active":"2024-09-21T06:35:05.029782+0000","last_peered":"2024-09-21T06:35:05.029782+0000","last_clean":"2024-09-21T06:35:05.029782+0000","last_became_active":"2024-09-21T06:34:32.649051+0000","last_became_peered":"2024-09-21T06:34:32.649051+0000","last_unstale":"2024-09-21T06:35:05.029782+0000","last_undegraded":"2024-09-21T06:35:05.029782+0000","last_fullsized":"2024-09-21T06:35:05.029782+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:12.631446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999624+0000","last_change":"2024-09-21T06:34:30.634592+0000","last_active":"2024-09-21T06:35:04.999624+0000","last_peered":"2024-09-21T06:35:04.999624+0000","last_clean":"2024-09-21T06:35:04.999624+0000","last_became_active":"2024-09-21T06:34:30.634251+0000","last_became_peered":"2024-09-21T06:34:30.634251+0000","last_unstale":"2024-09-21T06:35:04.999624+0000","last_undegraded":"2024-09-21T06:35:04.999624+0000","last_fullsized":"2024-09-21T06:35:04.999624+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:32.817467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000642+0000","last_change":"2024-09-21T06:34:39.719416+0000","last_active":"2024-09-21T06:35:05.000642+0000","last_peered":"2024-09-21T06:35:05.000642+0000","last_clean":"2024-09-21T06:35:05.000642+0000","last_became_active":"2024-09-21T06:34:39.718910+0000","last_became_peered":"2024-09-21T06:34:39.718910+0000","last_unstale":"2024-09-21T06:35:05.000642+0000","last_undegraded":"2024-09-21T06:35:05.000642+0000","last_fullsized":"2024-09-21T06:35:05.000642+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:41.413250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.994958+0000","last_change":"2024-09-21T06:34:32.640727+0000","last_active":"2024-09-21T06:35:04.994958+0000","last_peered":"2024-09-21T06:35:04.994958+0000","last_clean":"2024-09-21T06:35:04.994958+0000","last_became_active":"2024-09-21T06:34:32.640296+0000","last_became_peered":"2024-09-21T06:34:32.640296+0000","last_unstale":"2024-09-21T06:35:04.994958+0000","last_undegraded":"2024-09-21T06:35:04.994958+0000","last_fullsized":"2024-09-21T06:35:04.994958+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:35.124059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.996682+0000","last_change":"2024-09-21T06:34:30.633332+0000","last_active":"2024-09-21T06:35:04.996682+0000","last_peered":"2024-09-21T06:35:04.996682+0000","last_clean":"2024-09-21T06:35:04.996682+0000","last_became_active":"2024-09-21T06:34:30.632641+0000","last_became_peered":"2024-09-21T06:34:30.632641+0000","last_unstale":"2024-09-21T06:35:04.996682+0000","last_undegraded":"2024-09-21T06:35:04.996682+0000","last_fullsized":"2024-09-21T06:35:04.996682+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:30:23.930313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999766+0000","last_change":"2024-09-21T06:34:39.712504+0000","last_active":"2024-09-21T06:35:04.999766+0000","last_peered":"2024-09-21T06:35:04.999766+0000","last_clean":"2024-09-21T06:35:04.999766+0000","last_became_active":"2024-09-21T06:34:39.711638+0000","last_became_peered":"2024-09-21T06:34:39.711638+0000","last_unstale":"2024-09-21T06:35:04.999766+0000","last_undegraded":"2024-09-21T06:35:04.999766+0000","last_fullsized":"2024-09-21T06:35:04.999766+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:26:09.003367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.004007+0000","last_change":"2024-09-21T06:34:32.639582+0000","last_active":"2024-09-21T06:35:05.004007+0000","last_peered":"2024-09-21T06:35:05.004007+0000","last_clean":"2024-09-21T06:35:05.004007+0000","last_became_active":"2024-09-21T06:34:32.639023+0000","last_became_peered":"2024-09-21T06:34:32.639023+0000","last_unstale":"2024-09-21T06:35:05.004007+0000","last_undegraded":"2024-09-21T06:35:05.004007+0000","last_fullsized":"2024-09-21T06:35:05.004007+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:26.773389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"83'30","reported_seq":178,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:36:00.119601+0000","last_change":"2024-09-21T06:34:30.626280+0000","last_active":"2024-09-21T06:36:00.119601+0000","last_peered":"2024-09-21T06:36:00.119601+0000","last_clean":"2024-09-21T06:36:00.119601+0000","last_became_active":"2024-09-21T06:34:30.625662+0000","last_became_peered":"2024-09-21T06:34:30.625662+0000","last_unstale":"2024-09-21T06:36:00.119601+0000","last_undegraded":"2024-09-21T06:36:00.119601+0000","last_fullsized":"2024-09-21T06:36:00.119601+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:10.541881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68785,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":113,"num_read_kb":124,"num_write":30,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.113723+0000","last_change":"2024-09-21T06:34:39.708999+0000","last_active":"2024-09-21T06:35:05.113723+0000","last_peered":"2024-09-21T06:35:05.113723+0000","last_clean":"2024-09-21T06:35:05.113723+0000","last_became_active":"2024-09-21T06:34:39.708580+0000","last_became_peered":"2024-09-21T06:34:39.708580+0000","last_unstale":"2024-09-21T06:35:05.113723+0000","last_undegraded":"2024-09-21T06:35:05.113723+0000","last_fullsized":"2024-09-21T06:35:05.113723+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:07.648358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995798+0000","last_change":"2024-09-21T06:34:32.651157+0000","last_active":"2024-09-21T06:35:04.995798+0000","last_peered":"2024-09-21T06:35:04.995798+0000","last_clean":"2024-09-21T06:35:04.995798+0000","last_became_active":"2024-09-21T06:34:32.650804+0000","last_became_peered":"2024-09-21T06:34:32.650804+0000","last_unstale":"2024-09-21T06:35:04.995798+0000","last_undegraded":"2024-09-21T06:35:04.995798+0000","last_fullsized":"2024-09-21T06:35:04.995798+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:49.758496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999901+0000","last_change":"2024-09-21T06:34:30.947142+0000","last_active":"2024-09-21T06:35:04.999901+0000","last_peered":"2024-09-21T06:35:04.999901+0000","last_clean":"2024-09-21T06:35:04.999901+0000","last_became_active":"2024-09-21T06:34:30.946748+0000","last_became_peered":"2024-09-21T06:34:30.946748+0000","last_unstale":"2024-09-21T06:35:04.999901+0000","last_undegraded":"2024-09-21T06:35:04.999901+0000","last_fullsized":"2024-09-21T06:35:04.999901+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:39.222286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997754+0000","last_change":"2024-09-21T06:34:39.709266+0000","last_active":"2024-09-21T06:35:04.997754+0000","last_peered":"2024-09-21T06:35:04.997754+0000","last_clean":"2024-09-21T06:35:04.997754+0000","last_became_active":"2024-09-21T06:34:39.708753+0000","last_became_peered":"2024-09-21T06:34:39.708753+0000","last_unstale":"2024-09-21T06:35:04.997754+0000","last_undegraded":"2024-09-21T06:35:04.997754+0000","last_fullsized":"2024-09-21T06:35:04.997754+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:48.711681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000708+0000","last_change":"2024-09-21T06:34:32.637976+0000","last_active":"2024-09-21T06:35:05.000708+0000","last_peered":"2024-09-21T06:35:05.000708+0000","last_clean":"2024-09-21T06:35:05.000708+0000","last_became_active":"2024-09-21T06:34:32.637267+0000","last_became_peered":"2024-09-21T06:34:32.637267+0000","last_unstale":"2024-09-21T06:35:05.000708+0000","last_undegraded":"2024-09-21T06:35:05.000708+0000","last_fullsized":"2024-09-21T06:35:05.000708+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:31.340583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.030056+0000","last_change":"2024-09-21T06:34:30.631059+0000","last_active":"2024-09-21T06:35:05.030056+0000","last_peered":"2024-09-21T06:35:05.030056+0000","last_clean":"2024-09-21T06:35:05.030056+0000","last_became_active":"2024-09-21T06:34:30.630633+0000","last_became_peered":"2024-09-21T06:34:30.630633+0000","last_unstale":"2024-09-21T06:35:05.030056+0000","last_undegraded":"2024-09-21T06:35:05.030056+0000","last_fullsized":"2024-09-21T06:35:05.030056+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:09.049438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.113457+0000","last_change":"2024-09-21T06:34:39.713831+0000","last_active":"2024-09-21T06:35:05.113457+0000","last_peered":"2024-09-21T06:35:05.113457+0000","last_clean":"2024-09-21T06:35:05.113457+0000","last_became_active":"2024-09-21T06:34:39.713403+0000","last_became_peered":"2024-09-21T06:34:39.713403+0000","last_unstale":"2024-09-21T06:35:05.113457+0000","last_undegraded":"2024-09-21T06:35:05.113457+0000","last_fullsized":"2024-09-21T06:35:05.113457+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:22.201063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.996367+0000","last_change":"2024-09-21T06:34:32.646924+0000","last_active":"2024-09-21T06:35:04.996367+0000","last_peered":"2024-09-21T06:35:04.996367+0000","last_clean":"2024-09-21T06:35:04.996367+0000","last_became_active":"2024-09-21T06:34:32.646531+0000","last_became_peered":"2024-09-21T06:34:32.646531+0000","last_unstale":"2024-09-21T06:35:04.996367+0000","last_undegraded":"2024-09-21T06:35:04.996367+0000","last_fullsized":"2024-09-21T06:35:04.996367+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:48.550193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.001234+0000","last_change":"2024-09-21T06:34:30.631901+0000","last_active":"2024-09-21T06:35:05.001234+0000","last_peered":"2024-09-21T06:35:05.001234+0000","last_clean":"2024-09-21T06:35:05.001234+0000","last_became_active":"2024-09-21T06:34:30.631450+0000","last_became_peered":"2024-09-21T06:34:30.631450+0000","last_unstale":"2024-09-21T06:35:05.001234+0000","last_undegraded":"2024-09-21T06:35:05.001234+0000","last_fullsized":"2024-09-21T06:35:05.001234+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:20.709456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.272748+0000","last_change":"2024-09-21T06:34:39.711951+0000","last_active":"2024-09-21T06:35:05.272748+0000","last_peered":"2024-09-21T06:35:05.272748+0000","last_clean":"2024-09-21T06:35:05.272748+0000","last_became_active":"2024-09-21T06:34:39.710783+0000","last_became_peered":"2024-09-21T06:34:39.710783+0000","last_unstale":"2024-09-21T06:35:05.272748+0000","last_undegraded":"2024-09-21T06:35:05.272748+0000","last_fullsized":"2024-09-21T06:35:05.272748+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:27:58.501155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000355+0000","last_change":"2024-09-21T06:34:39.709639+0000","last_active":"2024-09-21T06:35:05.000355+0000","last_peered":"2024-09-21T06:35:05.000355+0000","last_clean":"2024-09-21T06:35:05.000355+0000","last_became_active":"2024-09-21T06:34:39.709241+0000","last_became_peered":"2024-09-21T06:34:39.709241+0000","last_unstale":"2024-09-21T06:35:05.000355+0000","last_undegraded":"2024-09-21T06:35:05.000355+0000","last_fullsized":"2024-09-21T06:35:05.000355+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.357399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.113674+0000","last_change":"2024-09-21T06:34:32.635365+0000","last_active":"2024-09-21T06:35:05.113674+0000","last_peered":"2024-09-21T06:35:05.113674+0000","last_clean":"2024-09-21T06:35:05.113674+0000","last_became_active":"2024-09-21T06:34:32.635012+0000","last_became_peered":"2024-09-21T06:34:32.635012+0000","last_unstale":"2024-09-21T06:35:05.113674+0000","last_undegraded":"2024-09-21T06:35:05.113674+0000","last_fullsized":"2024-09-21T06:35:05.113674+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:54.671074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.998620+0000","last_change":"2024-09-21T06:34:30.633347+0000","last_active":"2024-09-21T06:35:04.998620+0000","last_peered":"2024-09-21T06:35:04.998620+0000","last_clean":"2024-09-21T06:35:04.998620+0000","last_became_active":"2024-09-21T06:34:30.632513+0000","last_became_peered":"2024-09-21T06:34:30.632513+0000","last_unstale":"2024-09-21T06:35:04.998620+0000","last_undegraded":"2024-09-21T06:35:04.998620+0000","last_fullsized":"2024-09-21T06:35:04.998620+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:12.439109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.001150+0000","last_change":"2024-09-21T06:34:39.710204+0000","last_active":"2024-09-21T06:35:05.001150+0000","last_peered":"2024-09-21T06:35:05.001150+0000","last_clean":"2024-09-21T06:35:05.001150+0000","last_became_active":"2024-09-21T06:34:39.709744+0000","last_became_peered":"2024-09-21T06:34:39.709744+0000","last_unstale":"2024-09-21T06:35:05.001150+0000","last_undegraded":"2024-09-21T06:35:05.001150+0000","last_fullsized":"2024-09-21T06:35:05.001150+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:45.484088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.272833+0000","last_change":"2024-09-21T06:34:32.634170+0000","last_active":"2024-09-21T06:35:05.272833+0000","last_peered":"2024-09-21T06:35:05.272833+0000","last_clean":"2024-09-21T06:35:05.272833+0000","last_became_active":"2024-09-21T06:34:32.633831+0000","last_became_peered":"2024-09-21T06:34:32.633831+0000","last_unstale":"2024-09-21T06:35:05.272833+0000","last_undegraded":"2024-09-21T06:35:05.272833+0000","last_fullsized":"2024-09-21T06:35:05.272833+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:36.738279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.994732+0000","last_change":"2024-09-21T06:34:30.629422+0000","last_active":"2024-09-21T06:35:04.994732+0000","last_peered":"2024-09-21T06:35:04.994732+0000","last_clean":"2024-09-21T06:35:04.994732+0000","last_became_active":"2024-09-21T06:34:30.629025+0000","last_became_peered":"2024-09-21T06:34:30.629025+0000","last_unstale":"2024-09-21T06:35:04.994732+0000","last_undegraded":"2024-09-21T06:35:04.994732+0000","last_fullsized":"2024-09-21T06:35:04.994732+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:05.068816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000955+0000","last_change":"2024-09-21T06:34:30.634562+0000","last_active":"2024-09-21T06:35:05.000955+0000","last_peered":"2024-09-21T06:35:05.000955+0000","last_clean":"2024-09-21T06:35:05.000955+0000","last_became_active":"2024-09-21T06:34:30.634170+0000","last_became_peered":"2024-09-21T06:34:30.634170+0000","last_unstale":"2024-09-21T06:35:05.000955+0000","last_undegraded":"2024-09-21T06:35:05.000955+0000","last_fullsized":"2024-09-21T06:35:05.000955+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:15.243103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999225+0000","last_change":"2024-09-21T06:34:32.645718+0000","last_active":"2024-09-21T06:35:04.999225+0000","last_peered":"2024-09-21T06:35:04.999225+0000","last_clean":"2024-09-21T06:35:04.999225+0000","last_became_active":"2024-09-21T06:34:32.645072+0000","last_became_peered":"2024-09-21T06:34:32.645072+0000","last_unstale":"2024-09-21T06:35:04.999225+0000","last_undegraded":"2024-09-21T06:35:04.999225+0000","last_fullsized":"2024-09-21T06:35:04.999225+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:54.109246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.996127+0000","last_change":"2024-09-21T06:34:39.716947+0000","last_active":"2024-09-21T06:35:04.996127+0000","last_peered":"2024-09-21T06:35:04.996127+0000","last_clean":"2024-09-21T06:35:04.996127+0000","last_became_active":"2024-09-21T06:34:39.716540+0000","last_became_peered":"2024-09-21T06:34:39.716540+0000","last_unstale":"2024-09-21T06:35:04.996127+0000","last_undegraded":"2024-09-21T06:35:04.996127+0000","last_fullsized":"2024-09-21T06:35:04.996127+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:12.748444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999360+0000","last_change":"2024-09-21T06:34:30.636389+0000","last_active":"2024-09-21T06:35:04.999360+0000","last_peered":"2024-09-21T06:35:04.999360+0000","last_clean":"2024-09-21T06:35:04.999360+0000","last_became_active":"2024-09-21T06:34:30.636008+0000","last_became_peered":"2024-09-21T06:34:30.636008+0000","last_unstale":"2024-09-21T06:35:04.999360+0000","last_undegraded":"2024-09-21T06:35:04.999360+0000","last_fullsized":"2024-09-21T06:35:04.999360+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:45.476267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.003221+0000","last_change":"2024-09-21T06:34:32.635644+0000","last_active":"2024-09-21T06:35:05.003221+0000","last_peered":"2024-09-21T06:35:05.003221+0000","last_clean":"2024-09-21T06:35:05.003221+0000","last_became_active":"2024-09-21T06:34:32.635253+0000","last_became_peered":"2024-09-21T06:34:32.635253+0000","last_unstale":"2024-09-21T06:35:05.003221+0000","last_undegraded":"2024-09-21T06:35:05.003221+0000","last_fullsized":"2024-09-21T06:35:05.003221+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:34.713656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.996076+0000","last_change":"2024-09-21T06:34:39.710382+0000","last_active":"2024-09-21T06:35:04.996076+0000","last_peered":"2024-09-21T06:35:04.996076+0000","last_clean":"2024-09-21T06:35:04.996076+0000","last_became_active":"2024-09-21T06:34:39.709884+0000","last_became_peered":"2024-09-21T06:34:39.709884+0000","last_unstale":"2024-09-21T06:35:04.996076+0000","last_undegraded":"2024-09-21T06:35:04.996076+0000","last_fullsized":"2024-09-21T06:35:04.996076+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:01.602366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000463+0000","last_change":"2024-09-21T06:34:32.640025+0000","last_active":"2024-09-21T06:35:05.000463+0000","last_peered":"2024-09-21T06:35:05.000463+0000","last_clean":"2024-09-21T06:35:05.000463+0000","last_became_active":"2024-09-21T06:34:32.639648+0000","last_became_peered":"2024-09-21T06:34:32.639648+0000","last_unstale":"2024-09-21T06:35:05.000463+0000","last_undegraded":"2024-09-21T06:35:05.000463+0000","last_fullsized":"2024-09-21T06:35:05.000463+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:05.833065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.115450+0000","last_change":"2024-09-21T06:35:05.115344+0000","last_active":"2024-09-21T06:35:05.115450+0000","last_peered":"2024-09-21T06:35:05.115450+0000","last_clean":"2024-09-21T06:35:05.115450+0000","last_became_active":"2024-09-21T06:35:05.114678+0000","last_became_peered":"2024-09-21T06:35:05.114678+0000","last_unstale":"2024-09-21T06:35:05.115450+0000","last_undegraded":"2024-09-21T06:35:05.115450+0000","last_fullsized":"2024-09-21T06:35:05.115450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:24.593145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.115415+0000","last_change":"2024-09-21T06:35:05.115301+0000","last_active":"2024-09-21T06:35:05.115415+0000","last_peered":"2024-09-21T06:35:05.115415+0000","last_clean":"2024-09-21T06:35:05.115415+0000","last_became_active":"2024-09-21T06:35:05.114706+0000","last_became_peered":"2024-09-21T06:35:05.114706+0000","last_unstale":"2024-09-21T06:35:05.115415+0000","last_undegraded":"2024-09-21T06:35:05.115415+0000","last_fullsized":"2024-09-21T06:35:05.115415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:09.513594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000261+0000","last_change":"2024-09-21T06:34:32.634679+0000","last_active":"2024-09-21T06:35:05.000261+0000","last_peered":"2024-09-21T06:35:05.000261+0000","last_clean":"2024-09-21T06:35:05.000261+0000","last_became_active":"2024-09-21T06:34:32.634285+0000","last_became_peered":"2024-09-21T06:34:32.634285+0000","last_unstale":"2024-09-21T06:35:05.000261+0000","last_undegraded":"2024-09-21T06:35:05.000261+0000","last_fullsized":"2024-09-21T06:35:05.000261+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:15.185453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995303+0000","last_change":"2024-09-21T06:34:32.639037+0000","last_active":"2024-09-21T06:35:04.995303+0000","last_peered":"2024-09-21T06:35:04.995303+0000","last_clean":"2024-09-21T06:35:04.995303+0000","last_became_active":"2024-09-21T06:34:32.638689+0000","last_became_peered":"2024-09-21T06:34:32.638689+0000","last_unstale":"2024-09-21T06:35:04.995303+0000","last_undegraded":"2024-09-21T06:35:04.995303+0000","last_fullsized":"2024-09-21T06:35:04.995303+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:13.609191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.115157+0000","last_change":"2024-09-21T06:35:05.115051+0000","last_active":"2024-09-21T06:35:05.115157+0000","last_peered":"2024-09-21T06:35:05.115157+0000","last_clean":"2024-09-21T06:35:05.115157+0000","last_became_active":"2024-09-21T06:35:05.114602+0000","last_became_peered":"2024-09-21T06:35:05.114602+0000","last_unstale":"2024-09-21T06:35:05.115157+0000","last_undegraded":"2024-09-21T06:35:05.115157+0000","last_fullsized":"2024-09-21T06:35:05.115157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:48.778188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.996472+0000","last_change":"2024-09-21T06:34:32.637237+0000","last_active":"2024-09-21T06:35:04.996472+0000","last_peered":"2024-09-21T06:35:04.996472+0000","last_clean":"2024-09-21T06:35:04.996472+0000","last_became_active":"2024-09-21T06:34:32.634105+0000","last_became_peered":"2024-09-21T06:34:32.634105+0000","last_unstale":"2024-09-21T06:35:04.996472+0000","last_undegraded":"2024-09-21T06:35:04.996472+0000","last_fullsized":"2024-09-21T06:35:04.996472+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:52.530461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.273180+0000","last_change":"2024-09-21T06:34:32.624441+0000","last_active":"2024-09-21T06:35:05.273180+0000","last_peered":"2024-09-21T06:35:05.273180+0000","last_clean":"2024-09-21T06:35:05.273180+0000","last_became_active":"2024-09-21T06:34:32.623974+0000","last_became_peered":"2024-09-21T06:34:32.623974+0000","last_unstale":"2024-09-21T06:35:05.273180+0000","last_undegraded":"2024-09-21T06:35:05.273180+0000","last_fullsized":"2024-09-21T06:35:05.273180+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:12.378113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.998387+0000","last_change":"2024-09-21T06:34:32.633879+0000","last_active":"2024-09-21T06:35:04.998387+0000","last_peered":"2024-09-21T06:35:04.998387+0000","last_clean":"2024-09-21T06:35:04.998387+0000","last_became_active":"2024-09-21T06:34:32.631791+0000","last_became_peered":"2024-09-21T06:34:32.631791+0000","last_unstale":"2024-09-21T06:35:04.998387+0000","last_undegraded":"2024-09-21T06:35:04.998387+0000","last_fullsized":"2024-09-21T06:35:04.998387+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:46.246892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.273005+0000","last_change":"2024-09-21T06:34:32.639974+0000","last_active":"2024-09-21T06:35:05.273005+0000","last_peered":"2024-09-21T06:35:05.273005+0000","last_clean":"2024-09-21T06:35:05.273005+0000","last_became_active":"2024-09-21T06:34:32.639631+0000","last_became_peered":"2024-09-21T06:34:32.639631+0000","last_unstale":"2024-09-21T06:35:05.273005+0000","last_undegraded":"2024-09-21T06:35:05.273005+0000","last_fullsized":"2024-09-21T06:35:05.273005+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:50.069378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995887+0000","last_change":"2024-09-21T06:34:32.645403+0000","last_active":"2024-09-21T06:35:04.995887+0000","last_peered":"2024-09-21T06:35:04.995887+0000","last_clean":"2024-09-21T06:35:04.995887+0000","last_became_active":"2024-09-21T06:34:32.644986+0000","last_became_peered":"2024-09-21T06:34:32.644986+0000","last_unstale":"2024-09-21T06:35:04.995887+0000","last_undegraded":"2024-09-21T06:35:04.995887+0000","last_fullsized":"2024-09-21T06:35:04.995887+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:43.640107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995183+0000","last_change":"2024-09-21T06:34:32.639933+0000","last_active":"2024-09-21T06:35:04.995183+0000","last_peered":"2024-09-21T06:35:04.995183+0000","last_clean":"2024-09-21T06:35:04.995183+0000","last_became_active":"2024-09-21T06:34:32.639438+0000","last_became_peered":"2024-09-21T06:34:32.639438+0000","last_unstale":"2024-09-21T06:35:04.995183+0000","last_undegraded":"2024-09-21T06:35:04.995183+0000","last_fullsized":"2024-09-21T06:35:04.995183+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:18.860219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000816+0000","last_change":"2024-09-21T06:34:32.648580+0000","last_active":"2024-09-21T06:35:05.000816+0000","last_peered":"2024-09-21T06:35:05.000816+0000","last_clean":"2024-09-21T06:35:05.000816+0000","last_became_active":"2024-09-21T06:34:32.648016+0000","last_became_peered":"2024-09-21T06:34:32.648016+0000","last_unstale":"2024-09-21T06:35:05.000816+0000","last_undegraded":"2024-09-21T06:35:05.000816+0000","last_fullsized":"2024-09-21T06:35:05.000816+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:52.858069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.996429+0000","last_change":"2024-09-21T06:34:32.633376+0000","last_active":"2024-09-21T06:35:04.996429+0000","last_peered":"2024-09-21T06:35:04.996429+0000","last_clean":"2024-09-21T06:35:04.996429+0000","last_became_active":"2024-09-21T06:34:32.631052+0000","last_became_peered":"2024-09-21T06:34:32.631052+0000","last_unstale":"2024-09-21T06:35:04.996429+0000","last_undegraded":"2024-09-21T06:35:04.996429+0000","last_fullsized":"2024-09-21T06:35:04.996429+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:46.229599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.114206+0000","last_change":"2024-09-21T06:34:32.636966+0000","last_active":"2024-09-21T06:35:05.114206+0000","last_peered":"2024-09-21T06:35:05.114206+0000","last_clean":"2024-09-21T06:35:05.114206+0000","last_became_active":"2024-09-21T06:34:32.636565+0000","last_became_peered":"2024-09-21T06:34:32.636565+0000","last_unstale":"2024-09-21T06:35:05.114206+0000","last_undegraded":"2024-09-21T06:35:05.114206+0000","last_fullsized":"2024-09-21T06:35:05.114206+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:36.233082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.001311+0000","last_change":"2024-09-21T06:34:32.646895+0000","last_active":"2024-09-21T06:35:05.001311+0000","last_peered":"2024-09-21T06:35:05.001311+0000","last_clean":"2024-09-21T06:35:05.001311+0000","last_became_active":"2024-09-21T06:34:32.646505+0000","last_became_peered":"2024-09-21T06:34:32.646505+0000","last_unstale":"2024-09-21T06:35:05.001311+0000","last_undegraded":"2024-09-21T06:35:05.001311+0000","last_fullsized":"2024-09-21T06:35:05.001311+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:43.441337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995353+0000","last_change":"2024-09-21T06:34:32.645402+0000","last_active":"2024-09-21T06:35:04.995353+0000","last_peered":"2024-09-21T06:35:04.995353+0000","last_clean":"2024-09-21T06:35:04.995353+0000","last_became_active":"2024-09-21T06:34:32.645013+0000","last_became_peered":"2024-09-21T06:34:32.645013+0000","last_unstale":"2024-09-21T06:35:04.995353+0000","last_undegraded":"2024-09-21T06:35:04.995353+0000","last_fullsized":"2024-09-21T06:35:04.995353+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:33.475847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999814+0000","last_change":"2024-09-21T06:34:32.643234+0000","last_active":"2024-09-21T06:35:04.999814+0000","last_peered":"2024-09-21T06:35:04.999814+0000","last_clean":"2024-09-21T06:35:04.999814+0000","last_became_active":"2024-09-21T06:34:32.642614+0000","last_became_peered":"2024-09-21T06:34:32.642614+0000","last_unstale":"2024-09-21T06:35:04.999814+0000","last_undegraded":"2024-09-21T06:35:04.999814+0000","last_fullsized":"2024-09-21T06:35:04.999814+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:50.903500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999942+0000","last_change":"2024-09-21T06:34:32.641887+0000","last_active":"2024-09-21T06:35:04.999942+0000","last_peered":"2024-09-21T06:35:04.999942+0000","last_clean":"2024-09-21T06:35:04.999942+0000","last_became_active":"2024-09-21T06:34:32.641490+0000","last_became_peered":"2024-09-21T06:34:32.641490+0000","last_unstale":"2024-09-21T06:35:04.999942+0000","last_undegraded":"2024-09-21T06:35:04.999942+0000","last_fullsized":"2024-09-21T06:35:04.999942+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:34.550970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995289+0000","last_change":"2024-09-21T06:34:32.641865+0000","last_active":"2024-09-21T06:35:04.995289+0000","last_peered":"2024-09-21T06:35:04.995289+0000","last_clean":"2024-09-21T06:35:04.995289+0000","last_became_active":"2024-09-21T06:34:32.641252+0000","last_became_peered":"2024-09-21T06:34:32.641252+0000","last_unstale":"2024-09-21T06:35:04.995289+0000","last_undegraded":"2024-09-21T06:35:04.995289+0000","last_fullsized":"2024-09-21T06:35:04.995289+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:44.459202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997920+0000","last_change":"2024-09-21T06:34:32.641259+0000","last_active":"2024-09-21T06:35:04.997920+0000","last_peered":"2024-09-21T06:35:04.997920+0000","last_clean":"2024-09-21T06:35:04.997920+0000","last_became_active":"2024-09-21T06:34:32.640907+0000","last_became_peered":"2024-09-21T06:34:32.640907+0000","last_unstale":"2024-09-21T06:35:04.997920+0000","last_undegraded":"2024-09-21T06:35:04.997920+0000","last_fullsized":"2024-09-21T06:35:04.997920+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:59.966857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.030054+0000","last_change":"2024-09-21T06:34:32.646095+0000","last_active":"2024-09-21T06:35:05.030054+0000","last_peered":"2024-09-21T06:35:05.030054+0000","last_clean":"2024-09-21T06:35:05.030054+0000","last_became_active":"2024-09-21T06:34:32.645604+0000","last_became_peered":"2024-09-21T06:34:32.645604+0000","last_unstale":"2024-09-21T06:35:05.030054+0000","last_undegraded":"2024-09-21T06:35:05.030054+0000","last_fullsized":"2024-09-21T06:35:05.030054+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:30.872428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.001219+0000","last_change":"2024-09-21T06:34:32.649303+0000","last_active":"2024-09-21T06:35:05.001219+0000","last_peered":"2024-09-21T06:35:05.001219+0000","last_clean":"2024-09-21T06:35:05.001219+0000","last_became_active":"2024-09-21T06:34:32.648965+0000","last_became_peered":"2024-09-21T06:34:32.648965+0000","last_unstale":"2024-09-21T06:35:05.001219+0000","last_undegraded":"2024-09-21T06:35:05.001219+0000","last_fullsized":"2024-09-21T06:35:05.001219+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:51.335949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.996335+0000","last_change":"2024-09-21T06:34:32.637261+0000","last_active":"2024-09-21T06:35:04.996335+0000","last_peered":"2024-09-21T06:35:04.996335+0000","last_clean":"2024-09-21T06:35:04.996335+0000","last_became_active":"2024-09-21T06:34:32.636861+0000","last_became_peered":"2024-09-21T06:34:32.636861+0000","last_unstale":"2024-09-21T06:35:04.996335+0000","last_undegraded":"2024-09-21T06:35:04.996335+0000","last_fullsized":"2024-09-21T06:35:04.996335+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:55.271027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.272766+0000","last_change":"2024-09-21T06:34:32.635156+0000","last_active":"2024-09-21T06:35:05.272766+0000","last_peered":"2024-09-21T06:35:05.272766+0000","last_clean":"2024-09-21T06:35:05.272766+0000","last_became_active":"2024-09-21T06:34:32.634802+0000","last_became_peered":"2024-09-21T06:34:32.634802+0000","last_unstale":"2024-09-21T06:35:05.272766+0000","last_undegraded":"2024-09-21T06:35:05.272766+0000","last_fullsized":"2024-09-21T06:35:05.272766+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:16.139829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029506+0000","last_change":"2024-09-21T06:34:32.637509+0000","last_active":"2024-09-21T06:35:05.029506+0000","last_peered":"2024-09-21T06:35:05.029506+0000","last_clean":"2024-09-21T06:35:05.029506+0000","last_became_active":"2024-09-21T06:34:32.637136+0000","last_became_peered":"2024-09-21T06:34:32.637136+0000","last_unstale":"2024-09-21T06:35:05.029506+0000","last_undegraded":"2024-09-21T06:35:05.029506+0000","last_fullsized":"2024-09-21T06:35:05.029506+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:48.088988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.114942+0000","last_change":"2024-09-21T06:35:05.114819+0000","last_active":"2024-09-21T06:35:05.114942+0000","last_peered":"2024-09-21T06:35:05.114942+0000","last_clean":"2024-09-21T06:35:05.114942+0000","last_became_active":"2024-09-21T06:35:05.114424+0000","last_became_peered":"2024-09-21T06:35:05.114424+0000","last_unstale":"2024-09-21T06:35:05.114942+0000","last_undegraded":"2024-09-21T06:35:05.114942+0000","last_fullsized":"2024-09-21T06:35:05.114942+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:59.323643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.003387+0000","last_change":"2024-09-21T06:34:32.636103+0000","last_active":"2024-09-21T06:35:05.003387+0000","last_peered":"2024-09-21T06:35:05.003387+0000","last_clean":"2024-09-21T06:35:05.003387+0000","last_became_active":"2024-09-21T06:34:32.635728+0000","last_became_peered":"2024-09-21T06:34:32.635728+0000","last_unstale":"2024-09-21T06:35:05.003387+0000","last_undegraded":"2024-09-21T06:35:05.003387+0000","last_fullsized":"2024-09-21T06:35:05.003387+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:16.731402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997848+0000","last_change":"2024-09-21T06:34:32.635654+0000","last_active":"2024-09-21T06:35:04.997848+0000","last_peered":"2024-09-21T06:35:04.997848+0000","last_clean":"2024-09-21T06:35:04.997848+0000","last_became_active":"2024-09-21T06:34:32.635265+0000","last_became_peered":"2024-09-21T06:34:32.635265+0000","last_unstale":"2024-09-21T06:35:04.997848+0000","last_undegraded":"2024-09-21T06:35:04.997848+0000","last_fullsized":"2024-09-21T06:35:04.997848+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:28.169625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000238+0000","last_change":"2024-09-21T06:34:32.644640+0000","last_active":"2024-09-21T06:35:05.000238+0000","last_peered":"2024-09-21T06:35:05.000238+0000","last_clean":"2024-09-21T06:35:05.000238+0000","last_became_active":"2024-09-21T06:34:32.644334+0000","last_became_peered":"2024-09-21T06:34:32.644334+0000","last_unstale":"2024-09-21T06:35:05.000238+0000","last_undegraded":"2024-09-21T06:35:05.000238+0000","last_fullsized":"2024-09-21T06:35:05.000238+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:25.572178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000396+0000","last_change":"2024-09-21T06:34:32.643705+0000","last_active":"2024-09-21T06:35:05.000396+0000","last_peered":"2024-09-21T06:35:05.000396+0000","last_clean":"2024-09-21T06:35:05.000396+0000","last_became_active":"2024-09-21T06:34:32.642705+0000","last_became_peered":"2024-09-21T06:34:32.642705+0000","last_unstale":"2024-09-21T06:35:05.000396+0000","last_undegraded":"2024-09-21T06:35:05.000396+0000","last_fullsized":"2024-09-21T06:35:05.000396+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:22.542559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.001285+0000","last_change":"2024-09-21T06:34:32.646539+0000","last_active":"2024-09-21T06:35:05.001285+0000","last_peered":"2024-09-21T06:35:05.001285+0000","last_clean":"2024-09-21T06:35:05.001285+0000","last_became_active":"2024-09-21T06:34:32.645992+0000","last_became_peered":"2024-09-21T06:34:32.645992+0000","last_unstale":"2024-09-21T06:35:05.001285+0000","last_undegraded":"2024-09-21T06:35:05.001285+0000","last_fullsized":"2024-09-21T06:35:05.001285+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:38.856001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69583,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":119,"num_read_kb":130,"num_write":85,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":208749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71,"ondisk_log_size":71,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282016,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743456,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":466743,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939684,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":1121994,"data_compressed":59457,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103210,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":471421,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299439,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":467347,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495668,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":1061789,"data_compressed":59457,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659194,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":901120,"data_stored":462647,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855423,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":1057293,"data_compressed":59457,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018948,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":466765,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215176,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":462751,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":531466,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":531432,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":905216,"data_stored":462737,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":68819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:36:05.954 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T06:36:05.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:05 smithi203 ceph-mon[33735]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:36:05.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:05 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-21T06:36:05.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:05 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-09-21T06:36:05.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:05 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-21T06:36:05.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:05 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-21T06:36:05.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:05 smithi203 ceph-mon[33735]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T06:36:06.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:05 smithi100 ceph-mon[33401]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:36:06.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:05 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-21T06:36:06.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:05 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-09-21T06:36:06.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:05 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-21T06:36:06.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:05 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-21T06:36:06.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:05 smithi100 ceph-mon[33401]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T06:36:06.213 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:36:06.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.804+0000 7fb0b759e640 1 -- 172.21.15.67:0/827010715 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0b81092d0 msgr2=0x7fb0b81096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:06.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.804+0000 7fb0b759e640 1 --2- 172.21.15.67:0/827010715 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0b81092d0 0x7fb0b81096e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac009900 tx=0x7fb0ac02a410 comp rx=0 tx=0).stop 2024-09-21T06:36:06.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.804+0000 7fb0b759e640 1 -- 172.21.15.67:0/827010715 shutdown_connections 2024-09-21T06:36:06.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.804+0000 7fb0b759e640 1 --2- 172.21.15.67:0/827010715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0b8109c20 0x7fb0b8110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:06.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.804+0000 7fb0b759e640 1 --2- 172.21.15.67:0/827010715 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0b81092d0 0x7fb0b81096e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:06.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.804+0000 7fb0b759e640 1 --2- 172.21.15.67:0/827010715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b8103990 0x7fb0b8103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:06.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.804+0000 7fb0b759e640 1 -- 172.21.15.67:0/827010715 >> 172.21.15.67:0/827010715 conn(0x7fb0b80ff3e0 msgr2=0x7fb0b8101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:06.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.804+0000 7fb0b759e640 1 -- 172.21.15.67:0/827010715 shutdown_connections 2024-09-21T06:36:06.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.804+0000 7fb0b759e640 1 -- 172.21.15.67:0/827010715 wait complete. 2024-09-21T06:36:06.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.805+0000 7fb0b759e640 1 Processor -- start 2024-09-21T06:36:06.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.805+0000 7fb0b759e640 1 -- start start 2024-09-21T06:36:06.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.805+0000 7fb0b759e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0b8103990 0x7fb0b819a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:06.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.805+0000 7fb0b759e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0b81092d0 0x7fb0b819a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:06.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.805+0000 7fb0b759e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b8109c20 0x7fb0b819faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:06.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.805+0000 7fb0b759e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b81137b0 con 0x7fb0b8103990 2024-09-21T06:36:06.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.805+0000 7fb0b759e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b8113630 con 0x7fb0b8109c20 2024-09-21T06:36:06.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.805+0000 7fb0b759e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b8113930 con 0x7fb0b81092d0 2024-09-21T06:36:06.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.805+0000 7fb0b659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0b8103990 0x7fb0b819a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:06.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.805+0000 7fb0b659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0b8103990 0x7fb0b819a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44622/0 (socket says 172.21.15.67:44622) 2024-09-21T06:36:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.805+0000 7fb0b659c640 1 -- 172.21.15.67:0/499693401 learned_addr learned my addr 172.21.15.67:0/499693401 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.806+0000 7fb0b6d9d640 1 --2- 172.21.15.67:0/499693401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b8109c20 0x7fb0b819faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.806+0000 7fb0b5d9b640 1 --2- 172.21.15.67:0/499693401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0b81092d0 0x7fb0b819a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.806+0000 7fb0b659c640 1 -- 172.21.15.67:0/499693401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0b81092d0 msgr2=0x7fb0b819a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.806+0000 7fb0b659c640 1 --2- 172.21.15.67:0/499693401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0b81092d0 0x7fb0b819a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.806+0000 7fb0b659c640 1 -- 172.21.15.67:0/499693401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b8109c20 msgr2=0x7fb0b819faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.806+0000 7fb0b659c640 1 --2- 172.21.15.67:0/499693401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b8109c20 0x7fb0b819faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.806+0000 7fb0b659c640 1 -- 172.21.15.67:0/499693401 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b81a01d0 con 0x7fb0b8103990 2024-09-21T06:36:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.806+0000 7fb0b659c640 1 --2- 172.21.15.67:0/499693401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0b8103990 0x7fb0b819a260 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800d950 tx=0x7fb0a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.806+0000 7fb0a77fe640 1 -- 172.21.15.67:0/499693401 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8017070 con 0x7fb0b8103990 2024-09-21T06:36:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.806+0000 7fb0a77fe640 1 -- 172.21.15.67:0/499693401 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0a8009c30 con 0x7fb0b8103990 2024-09-21T06:36:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.806+0000 7fb0b759e640 1 -- 172.21.15.67:0/499693401 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b81a04c0 con 0x7fb0b8103990 2024-09-21T06:36:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.806+0000 7fb0a77fe640 1 -- 172.21.15.67:0/499693401 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a800f620 con 0x7fb0b8103990 2024-09-21T06:36:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.806+0000 7fb0b759e640 1 -- 172.21.15.67:0/499693401 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b81a6c20 con 0x7fb0b8103990 2024-09-21T06:36:06.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.807+0000 7fb0b759e640 1 -- 172.21.15.67:0/499693401 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0b8103da0 con 0x7fb0b8103990 2024-09-21T06:36:06.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.809+0000 7fb0a77fe640 1 -- 172.21.15.67:0/499693401 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fb0a8005030 con 0x7fb0b8103990 2024-09-21T06:36:06.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.810+0000 7fb0a77fe640 1 --2- 172.21.15.67:0/499693401 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb0a00b6620 0x7fb0a00b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:06.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.810+0000 7fb0b5d9b640 1 --2- 172.21.15.67:0/499693401 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb0a00b6620 0x7fb0a00b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:06.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.810+0000 7fb0a77fe640 1 -- 172.21.15.67:0/499693401 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fb0a80e73e0 con 0x7fb0b8103990 2024-09-21T06:36:06.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.811+0000 7fb0b5d9b640 1 --2- 172.21.15.67:0/499693401 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb0a00b6620 0x7fb0a00b8b30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0057c0 tx=0x7fb0ac000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:06.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.813+0000 7fb0a77fe640 1 -- 172.21.15.67:0/499693401 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb0a80ad7c0 con 0x7fb0b8103990 2024-09-21T06:36:06.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:06 smithi203 ceph-mon[33735]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:36:06.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:06 smithi203 ceph-mon[33735]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T06:36:07.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:06.998+0000 7fb0b759e640 1 -- 172.21.15.67:0/499693401 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb0b8063800 con 0x7fb0a00b6620 2024-09-21T06:36:07.000 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:06 smithi067 ceph-mon[31232]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:36:07.000 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:06 smithi067 ceph-mon[31232]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T06:36:07.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:07.004+0000 7fb0a77fe640 1 -- 172.21.15.67:0/499693401 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340525 (secure 0 0 0) 0x7fb0b8063800 con 0x7fb0a00b6620 2024-09-21T06:36:07.009 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:36:07.010 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-09-21T06:36:07.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:07.007+0000 7fb0b759e640 1 -- 172.21.15.67:0/499693401 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb0a00b6620 msgr2=0x7fb0a00b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:07.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:07.007+0000 7fb0b759e640 1 --2- 172.21.15.67:0/499693401 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb0a00b6620 0x7fb0a00b8b30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0057c0 tx=0x7fb0ac000bc0 comp rx=0 tx=0).stop 2024-09-21T06:36:07.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:07.007+0000 7fb0b759e640 1 -- 172.21.15.67:0/499693401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0b8103990 msgr2=0x7fb0b819a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:07.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:07.007+0000 7fb0b759e640 1 --2- 172.21.15.67:0/499693401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0b8103990 0x7fb0b819a260 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800d950 tx=0x7fb0a800de10 comp rx=0 tx=0).stop 2024-09-21T06:36:07.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:07.007+0000 7fb0b759e640 1 -- 172.21.15.67:0/499693401 shutdown_connections 2024-09-21T06:36:07.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:07.007+0000 7fb0b759e640 1 --2- 172.21.15.67:0/499693401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b8109c20 0x7fb0b819faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:07.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:07.007+0000 7fb0b759e640 1 --2- 172.21.15.67:0/499693401 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb0a00b6620 0x7fb0a00b8b30 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:07.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:07.007+0000 7fb0b759e640 1 --2- 172.21.15.67:0/499693401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0b81092d0 0x7fb0b819a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:07.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:07.007+0000 7fb0b759e640 1 --2- 172.21.15.67:0/499693401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0b8103990 0x7fb0b819a260 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:07.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:07.007+0000 7fb0b759e640 1 -- 172.21.15.67:0/499693401 >> 172.21.15.67:0/499693401 conn(0x7fb0b80ff3e0 msgr2=0x7fb0b8101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:07.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:07.007+0000 7fb0b759e640 1 -- 172.21.15.67:0/499693401 shutdown_connections 2024-09-21T06:36:07.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:07.007+0000 7fb0b759e640 1 -- 172.21.15.67:0/499693401 wait complete. 2024-09-21T06:36:07.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:06 smithi100 ceph-mon[33401]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:36:07.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:06 smithi100 ceph-mon[33401]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T06:36:07.798 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-09-21T06:36:05.867817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660185,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":205,"num_read_kb":199,"num_write":210,"num_write_kb":2248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":165,"ondisk_log_size":165,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364496,"kb_used_data":11060,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577328,"statfs":{"total":1151940427776,"available":1151567183872,"internally_reserved":0,"allocated":11325440,"data_stored":7592957,"data_compressed":399729,"data_compressed_allocated":540672,"data_compressed_original":2580480,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002594"},"pg_stats":[{"pgid":"2.1f","version":"75'4","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.114370+0000","last_change":"2024-09-21T06:34:30.633366+0000","last_active":"2024-09-21T06:35:05.114370+0000","last_peered":"2024-09-21T06:35:05.114370+0000","last_clean":"2024-09-21T06:35:05.114370+0000","last_became_active":"2024-09-21T06:34:30.632929+0000","last_became_peered":"2024-09-21T06:34:30.632929+0000","last_unstale":"2024-09-21T06:35:05.114370+0000","last_undegraded":"2024-09-21T06:35:05.114370+0000","last_fullsized":"2024-09-21T06:35:05.114370+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:47.722461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.834454+0000","last_change":"2024-09-21T06:34:32.632989+0000","last_active":"2024-09-21T06:36:04.834454+0000","last_peered":"2024-09-21T06:36:04.834454+0000","last_clean":"2024-09-21T06:36:04.834454+0000","last_became_active":"2024-09-21T06:34:32.632024+0000","last_became_peered":"2024-09-21T06:34:32.632024+0000","last_unstale":"2024-09-21T06:36:04.834454+0000","last_undegraded":"2024-09-21T06:36:04.834454+0000","last_fullsized":"2024-09-21T06:36:04.834454+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:13.669613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.834068+0000","last_change":"2024-09-21T06:35:05.274434+0000","last_active":"2024-09-21T06:36:04.834068+0000","last_peered":"2024-09-21T06:36:04.834068+0000","last_clean":"2024-09-21T06:36:04.834068+0000","last_became_active":"2024-09-21T06:35:05.273833+0000","last_became_peered":"2024-09-21T06:35:05.273833+0000","last_unstale":"2024-09-21T06:36:04.834068+0000","last_undegraded":"2024-09-21T06:36:04.834068+0000","last_fullsized":"2024-09-21T06:36:04.834068+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:48.847861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.834817+0000","last_change":"2024-09-21T06:34:30.946309+0000","last_active":"2024-09-21T06:36:04.834817+0000","last_peered":"2024-09-21T06:36:04.834817+0000","last_clean":"2024-09-21T06:36:04.834817+0000","last_became_active":"2024-09-21T06:34:30.945913+0000","last_became_peered":"2024-09-21T06:34:30.945913+0000","last_unstale":"2024-09-21T06:36:04.834817+0000","last_undegraded":"2024-09-21T06:36:04.834817+0000","last_fullsized":"2024-09-21T06:36:04.834817+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:21.676524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000669+0000","last_change":"2024-09-21T06:34:32.637636+0000","last_active":"2024-09-21T06:35:05.000669+0000","last_peered":"2024-09-21T06:35:05.000669+0000","last_clean":"2024-09-21T06:35:05.000669+0000","last_became_active":"2024-09-21T06:34:32.636534+0000","last_became_peered":"2024-09-21T06:34:32.636534+0000","last_unstale":"2024-09-21T06:35:05.000669+0000","last_undegraded":"2024-09-21T06:35:05.000669+0000","last_fullsized":"2024-09-21T06:35:05.000669+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:05.802118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.274538+0000","last_change":"2024-09-21T06:35:05.274419+0000","last_active":"2024-09-21T06:35:05.274538+0000","last_peered":"2024-09-21T06:35:05.274538+0000","last_clean":"2024-09-21T06:35:05.274538+0000","last_became_active":"2024-09-21T06:35:05.273865+0000","last_became_peered":"2024-09-21T06:35:05.273865+0000","last_unstale":"2024-09-21T06:35:05.274538+0000","last_undegraded":"2024-09-21T06:35:05.274538+0000","last_fullsized":"2024-09-21T06:35:05.274538+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:04.490355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.004199+0000","last_change":"2024-09-21T06:34:30.636856+0000","last_active":"2024-09-21T06:35:05.004199+0000","last_peered":"2024-09-21T06:35:05.004199+0000","last_clean":"2024-09-21T06:35:05.004199+0000","last_became_active":"2024-09-21T06:34:30.636460+0000","last_became_peered":"2024-09-21T06:34:30.636460+0000","last_unstale":"2024-09-21T06:35:05.004199+0000","last_undegraded":"2024-09-21T06:35:05.004199+0000","last_fullsized":"2024-09-21T06:35:05.004199+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:58.917851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995796+0000","last_change":"2024-09-21T06:34:32.650303+0000","last_active":"2024-09-21T06:35:04.995796+0000","last_peered":"2024-09-21T06:35:04.995796+0000","last_clean":"2024-09-21T06:35:04.995796+0000","last_became_active":"2024-09-21T06:34:32.647189+0000","last_became_peered":"2024-09-21T06:34:32.647189+0000","last_unstale":"2024-09-21T06:35:04.995796+0000","last_undegraded":"2024-09-21T06:35:04.995796+0000","last_fullsized":"2024-09-21T06:35:04.995796+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:14.889384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.996942+0000","last_change":"2024-09-21T06:34:39.713139+0000","last_active":"2024-09-21T06:35:04.996942+0000","last_peered":"2024-09-21T06:35:04.996942+0000","last_clean":"2024-09-21T06:35:04.996942+0000","last_became_active":"2024-09-21T06:34:39.712379+0000","last_became_peered":"2024-09-21T06:34:39.712379+0000","last_unstale":"2024-09-21T06:35:04.996942+0000","last_undegraded":"2024-09-21T06:35:04.996942+0000","last_fullsized":"2024-09-21T06:35:04.996942+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:53.100529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.835790+0000","last_change":"2024-09-21T06:34:30.633850+0000","last_active":"2024-09-21T06:36:04.835790+0000","last_peered":"2024-09-21T06:36:04.835790+0000","last_clean":"2024-09-21T06:36:04.835790+0000","last_became_active":"2024-09-21T06:34:30.633468+0000","last_became_peered":"2024-09-21T06:34:30.633468+0000","last_unstale":"2024-09-21T06:36:04.835790+0000","last_undegraded":"2024-09-21T06:36:04.835790+0000","last_fullsized":"2024-09-21T06:36:04.835790+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:59.095803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.005361+0000","last_change":"2024-09-21T06:35:05.005240+0000","last_active":"2024-09-21T06:35:05.005361+0000","last_peered":"2024-09-21T06:35:05.005361+0000","last_clean":"2024-09-21T06:35:05.005361+0000","last_became_active":"2024-09-21T06:35:05.004849+0000","last_became_peered":"2024-09-21T06:35:05.004849+0000","last_unstale":"2024-09-21T06:35:05.005361+0000","last_undegraded":"2024-09-21T06:35:05.005361+0000","last_fullsized":"2024-09-21T06:35:05.005361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:28.849604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.274638+0000","last_change":"2024-09-21T06:35:05.274437+0000","last_active":"2024-09-21T06:35:05.274638+0000","last_peered":"2024-09-21T06:35:05.274638+0000","last_clean":"2024-09-21T06:35:05.274638+0000","last_became_active":"2024-09-21T06:35:05.273865+0000","last_became_peered":"2024-09-21T06:35:05.273865+0000","last_unstale":"2024-09-21T06:35:05.274638+0000","last_undegraded":"2024-09-21T06:35:05.274638+0000","last_fullsized":"2024-09-21T06:35:05.274638+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:43.339504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029725+0000","last_change":"2024-09-21T06:34:32.645630+0000","last_active":"2024-09-21T06:35:05.029725+0000","last_peered":"2024-09-21T06:35:05.029725+0000","last_clean":"2024-09-21T06:35:05.029725+0000","last_became_active":"2024-09-21T06:34:32.645237+0000","last_became_peered":"2024-09-21T06:34:32.645237+0000","last_unstale":"2024-09-21T06:35:05.029725+0000","last_undegraded":"2024-09-21T06:35:05.029725+0000","last_fullsized":"2024-09-21T06:35:05.029725+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:25.064318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.003627+0000","last_change":"2024-09-21T06:34:30.636609+0000","last_active":"2024-09-21T06:35:05.003627+0000","last_peered":"2024-09-21T06:35:05.003627+0000","last_clean":"2024-09-21T06:35:05.003627+0000","last_became_active":"2024-09-21T06:34:30.636217+0000","last_became_peered":"2024-09-21T06:34:30.636217+0000","last_unstale":"2024-09-21T06:35:05.003627+0000","last_undegraded":"2024-09-21T06:35:05.003627+0000","last_fullsized":"2024-09-21T06:35:05.003627+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:32.541307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997116+0000","last_change":"2024-09-21T06:34:39.714177+0000","last_active":"2024-09-21T06:35:04.997116+0000","last_peered":"2024-09-21T06:35:04.997116+0000","last_clean":"2024-09-21T06:35:04.997116+0000","last_became_active":"2024-09-21T06:34:39.713713+0000","last_became_peered":"2024-09-21T06:34:39.713713+0000","last_unstale":"2024-09-21T06:35:04.997116+0000","last_undegraded":"2024-09-21T06:35:04.997116+0000","last_fullsized":"2024-09-21T06:35:04.997116+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:07.042240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.834755+0000","last_change":"2024-09-21T06:34:32.636297+0000","last_active":"2024-09-21T06:36:04.834755+0000","last_peered":"2024-09-21T06:36:04.834755+0000","last_clean":"2024-09-21T06:36:04.834755+0000","last_became_active":"2024-09-21T06:34:32.635930+0000","last_became_peered":"2024-09-21T06:34:32.635930+0000","last_unstale":"2024-09-21T06:36:04.834755+0000","last_undegraded":"2024-09-21T06:36:04.834755+0000","last_fullsized":"2024-09-21T06:36:04.834755+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:43.091682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.834445+0000","last_change":"2024-09-21T06:34:30.631405+0000","last_active":"2024-09-21T06:36:04.834445+0000","last_peered":"2024-09-21T06:36:04.834445+0000","last_clean":"2024-09-21T06:36:04.834445+0000","last_became_active":"2024-09-21T06:34:30.630512+0000","last_became_peered":"2024-09-21T06:34:30.630512+0000","last_unstale":"2024-09-21T06:36:04.834445+0000","last_undegraded":"2024-09-21T06:36:04.834445+0000","last_fullsized":"2024-09-21T06:36:04.834445+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:43.412695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"83'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997680+0000","last_change":"2024-09-21T06:34:39.710573+0000","last_active":"2024-09-21T06:35:04.997680+0000","last_peered":"2024-09-21T06:35:04.997680+0000","last_clean":"2024-09-21T06:35:04.997680+0000","last_became_active":"2024-09-21T06:34:39.708969+0000","last_became_peered":"2024-09-21T06:34:39.708969+0000","last_unstale":"2024-09-21T06:35:04.997680+0000","last_undegraded":"2024-09-21T06:35:04.997680+0000","last_fullsized":"2024-09-21T06:35:04.997680+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:34.847587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000559+0000","last_change":"2024-09-21T06:34:32.641085+0000","last_active":"2024-09-21T06:35:05.000559+0000","last_peered":"2024-09-21T06:35:05.000559+0000","last_clean":"2024-09-21T06:35:05.000559+0000","last_became_active":"2024-09-21T06:34:32.640723+0000","last_became_peered":"2024-09-21T06:34:32.640723+0000","last_unstale":"2024-09-21T06:35:05.000559+0000","last_undegraded":"2024-09-21T06:35:05.000559+0000","last_fullsized":"2024-09-21T06:35:05.000559+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:21.410262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.834234+0000","last_change":"2024-09-21T06:34:30.626719+0000","last_active":"2024-09-21T06:36:04.834234+0000","last_peered":"2024-09-21T06:36:04.834234+0000","last_clean":"2024-09-21T06:36:04.834234+0000","last_became_active":"2024-09-21T06:34:30.626312+0000","last_became_peered":"2024-09-21T06:34:30.626312+0000","last_unstale":"2024-09-21T06:36:04.834234+0000","last_undegraded":"2024-09-21T06:36:04.834234+0000","last_fullsized":"2024-09-21T06:36:04.834234+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:53.938914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.114020+0000","last_change":"2024-09-21T06:34:39.711172+0000","last_active":"2024-09-21T06:35:05.114020+0000","last_peered":"2024-09-21T06:35:05.114020+0000","last_clean":"2024-09-21T06:35:05.114020+0000","last_became_active":"2024-09-21T06:34:39.708424+0000","last_became_peered":"2024-09-21T06:34:39.708424+0000","last_unstale":"2024-09-21T06:35:05.114020+0000","last_undegraded":"2024-09-21T06:35:05.114020+0000","last_fullsized":"2024-09-21T06:35:05.114020+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:43.918355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.833657+0000","last_change":"2024-09-21T06:34:32.641463+0000","last_active":"2024-09-21T06:36:04.833657+0000","last_peered":"2024-09-21T06:36:04.833657+0000","last_clean":"2024-09-21T06:36:04.833657+0000","last_became_active":"2024-09-21T06:34:32.641086+0000","last_became_peered":"2024-09-21T06:34:32.641086+0000","last_unstale":"2024-09-21T06:36:04.833657+0000","last_undegraded":"2024-09-21T06:36:04.833657+0000","last_fullsized":"2024-09-21T06:36:04.833657+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:58.991374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995541+0000","last_change":"2024-09-21T06:34:30.946698+0000","last_active":"2024-09-21T06:35:04.995541+0000","last_peered":"2024-09-21T06:35:04.995541+0000","last_clean":"2024-09-21T06:35:04.995541+0000","last_became_active":"2024-09-21T06:34:30.946325+0000","last_became_peered":"2024-09-21T06:34:30.946325+0000","last_unstale":"2024-09-21T06:35:04.995541+0000","last_undegraded":"2024-09-21T06:35:04.995541+0000","last_fullsized":"2024-09-21T06:35:04.995541+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:22.639265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997471+0000","last_change":"2024-09-21T06:34:39.712429+0000","last_active":"2024-09-21T06:35:04.997471+0000","last_peered":"2024-09-21T06:35:04.997471+0000","last_clean":"2024-09-21T06:35:04.997471+0000","last_became_active":"2024-09-21T06:34:39.711823+0000","last_became_peered":"2024-09-21T06:34:39.711823+0000","last_unstale":"2024-09-21T06:35:04.997471+0000","last_undegraded":"2024-09-21T06:35:04.997471+0000","last_fullsized":"2024-09-21T06:35:04.997471+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:03:49.513991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.274438+0000","last_change":"2024-09-21T06:35:05.274326+0000","last_active":"2024-09-21T06:35:05.274438+0000","last_peered":"2024-09-21T06:35:05.274438+0000","last_clean":"2024-09-21T06:35:05.274438+0000","last_became_active":"2024-09-21T06:35:05.273884+0000","last_became_peered":"2024-09-21T06:35:05.273884+0000","last_unstale":"2024-09-21T06:35:05.274438+0000","last_undegraded":"2024-09-21T06:35:05.274438+0000","last_fullsized":"2024-09-21T06:35:05.274438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:03.219765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997399+0000","last_change":"2024-09-21T06:34:30.676794+0000","last_active":"2024-09-21T06:35:04.997399+0000","last_peered":"2024-09-21T06:35:04.997399+0000","last_clean":"2024-09-21T06:35:04.997399+0000","last_became_active":"2024-09-21T06:34:30.676400+0000","last_became_peered":"2024-09-21T06:34:30.676400+0000","last_unstale":"2024-09-21T06:35:04.997399+0000","last_undegraded":"2024-09-21T06:35:04.997399+0000","last_fullsized":"2024-09-21T06:35:04.997399+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:23.681363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.835574+0000","last_change":"2024-09-21T06:34:39.708980+0000","last_active":"2024-09-21T06:36:04.835574+0000","last_peered":"2024-09-21T06:36:04.835574+0000","last_clean":"2024-09-21T06:36:04.835574+0000","last_became_active":"2024-09-21T06:34:39.707748+0000","last_became_peered":"2024-09-21T06:34:39.707748+0000","last_unstale":"2024-09-21T06:36:04.835574+0000","last_undegraded":"2024-09-21T06:36:04.835574+0000","last_fullsized":"2024-09-21T06:36:04.835574+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:16.812226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.835284+0000","last_change":"2024-09-21T06:34:32.638337+0000","last_active":"2024-09-21T06:36:04.835284+0000","last_peered":"2024-09-21T06:36:04.835284+0000","last_clean":"2024-09-21T06:36:04.835284+0000","last_became_active":"2024-09-21T06:34:32.638013+0000","last_became_peered":"2024-09-21T06:34:32.638013+0000","last_unstale":"2024-09-21T06:36:04.835284+0000","last_undegraded":"2024-09-21T06:36:04.835284+0000","last_fullsized":"2024-09-21T06:36:04.835284+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:26.321190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"83'9","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:23.662935+0000","last_change":"2024-09-21T06:34:30.631086+0000","last_active":"2024-09-21T06:35:23.662935+0000","last_peered":"2024-09-21T06:35:23.662935+0000","last_clean":"2024-09-21T06:35:23.662935+0000","last_became_active":"2024-09-21T06:34:30.630706+0000","last_became_peered":"2024-09-21T06:34:30.630706+0000","last_unstale":"2024-09-21T06:35:23.662935+0000","last_undegraded":"2024-09-21T06:35:23.662935+0000","last_fullsized":"2024-09-21T06:35:23.662935+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:30.260359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.833830+0000","last_change":"2024-09-21T06:34:39.709097+0000","last_active":"2024-09-21T06:36:04.833830+0000","last_peered":"2024-09-21T06:36:04.833830+0000","last_clean":"2024-09-21T06:36:04.833830+0000","last_became_active":"2024-09-21T06:34:39.708609+0000","last_became_peered":"2024-09-21T06:34:39.708609+0000","last_unstale":"2024-09-21T06:36:04.833830+0000","last_undegraded":"2024-09-21T06:36:04.833830+0000","last_fullsized":"2024-09-21T06:36:04.833830+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:48.842174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995363+0000","last_change":"2024-09-21T06:34:32.636816+0000","last_active":"2024-09-21T06:35:04.995363+0000","last_peered":"2024-09-21T06:35:04.995363+0000","last_clean":"2024-09-21T06:35:04.995363+0000","last_became_active":"2024-09-21T06:34:32.636429+0000","last_became_peered":"2024-09-21T06:34:32.636429+0000","last_unstale":"2024-09-21T06:35:04.995363+0000","last_undegraded":"2024-09-21T06:35:04.995363+0000","last_fullsized":"2024-09-21T06:35:04.995363+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:49.010491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.836011+0000","last_change":"2024-09-21T06:34:30.947481+0000","last_active":"2024-09-21T06:36:04.836011+0000","last_peered":"2024-09-21T06:36:04.836011+0000","last_clean":"2024-09-21T06:36:04.836011+0000","last_became_active":"2024-09-21T06:34:30.947040+0000","last_became_peered":"2024-09-21T06:34:30.947040+0000","last_unstale":"2024-09-21T06:36:04.836011+0000","last_undegraded":"2024-09-21T06:36:04.836011+0000","last_fullsized":"2024-09-21T06:36:04.836011+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:52.813476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000546+0000","last_change":"2024-09-21T06:34:39.708674+0000","last_active":"2024-09-21T06:35:05.000546+0000","last_peered":"2024-09-21T06:35:05.000546+0000","last_clean":"2024-09-21T06:35:05.000546+0000","last_became_active":"2024-09-21T06:34:39.708176+0000","last_became_peered":"2024-09-21T06:34:39.708176+0000","last_unstale":"2024-09-21T06:35:05.000546+0000","last_undegraded":"2024-09-21T06:35:05.000546+0000","last_fullsized":"2024-09-21T06:35:05.000546+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:57.070792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.833860+0000","last_change":"2024-09-21T06:34:32.636381+0000","last_active":"2024-09-21T06:36:04.833860+0000","last_peered":"2024-09-21T06:36:04.833860+0000","last_clean":"2024-09-21T06:36:04.833860+0000","last_became_active":"2024-09-21T06:34:32.635937+0000","last_became_peered":"2024-09-21T06:34:32.635937+0000","last_unstale":"2024-09-21T06:36:04.833860+0000","last_undegraded":"2024-09-21T06:36:04.833860+0000","last_fullsized":"2024-09-21T06:36:04.833860+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:52.870676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.114033+0000","last_change":"2024-09-21T06:34:30.947160+0000","last_active":"2024-09-21T06:35:05.114033+0000","last_peered":"2024-09-21T06:35:05.114033+0000","last_clean":"2024-09-21T06:35:05.114033+0000","last_became_active":"2024-09-21T06:34:30.946736+0000","last_became_peered":"2024-09-21T06:34:30.946736+0000","last_unstale":"2024-09-21T06:35:05.114033+0000","last_undegraded":"2024-09-21T06:35:05.114033+0000","last_fullsized":"2024-09-21T06:35:05.114033+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:47.425481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.833575+0000","last_change":"2024-09-21T06:35:05.006032+0000","last_active":"2024-09-21T06:36:04.833575+0000","last_peered":"2024-09-21T06:36:04.833575+0000","last_clean":"2024-09-21T06:36:04.833575+0000","last_became_active":"2024-09-21T06:35:05.005429+0000","last_became_peered":"2024-09-21T06:35:05.005429+0000","last_unstale":"2024-09-21T06:36:04.833575+0000","last_undegraded":"2024-09-21T06:36:04.833575+0000","last_fullsized":"2024-09-21T06:36:04.833575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:04.138097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.833789+0000","last_change":"2024-09-21T06:34:32.648256+0000","last_active":"2024-09-21T06:36:04.833789+0000","last_peered":"2024-09-21T06:36:04.833789+0000","last_clean":"2024-09-21T06:36:04.833789+0000","last_became_active":"2024-09-21T06:34:32.647881+0000","last_became_peered":"2024-09-21T06:34:32.647881+0000","last_unstale":"2024-09-21T06:36:04.833789+0000","last_undegraded":"2024-09-21T06:36:04.833789+0000","last_fullsized":"2024-09-21T06:36:04.833789+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:14.590834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029849+0000","last_change":"2024-09-21T06:34:30.629918+0000","last_active":"2024-09-21T06:35:05.029849+0000","last_peered":"2024-09-21T06:35:05.029849+0000","last_clean":"2024-09-21T06:35:05.029849+0000","last_became_active":"2024-09-21T06:34:30.628785+0000","last_became_peered":"2024-09-21T06:34:30.628785+0000","last_unstale":"2024-09-21T06:35:05.029849+0000","last_undegraded":"2024-09-21T06:35:05.029849+0000","last_fullsized":"2024-09-21T06:35:05.029849+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:05.333406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995102+0000","last_change":"2024-09-21T06:34:39.710073+0000","last_active":"2024-09-21T06:35:04.995102+0000","last_peered":"2024-09-21T06:35:04.995102+0000","last_clean":"2024-09-21T06:35:04.995102+0000","last_became_active":"2024-09-21T06:34:39.708986+0000","last_became_peered":"2024-09-21T06:34:39.708986+0000","last_unstale":"2024-09-21T06:35:04.995102+0000","last_undegraded":"2024-09-21T06:35:04.995102+0000","last_fullsized":"2024-09-21T06:35:04.995102+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:46.379902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.998454+0000","last_change":"2024-09-21T06:34:32.644224+0000","last_active":"2024-09-21T06:35:04.998454+0000","last_peered":"2024-09-21T06:35:04.998454+0000","last_clean":"2024-09-21T06:35:04.998454+0000","last_became_active":"2024-09-21T06:34:32.643850+0000","last_became_peered":"2024-09-21T06:34:32.643850+0000","last_unstale":"2024-09-21T06:35:04.998454+0000","last_undegraded":"2024-09-21T06:35:04.998454+0000","last_fullsized":"2024-09-21T06:35:04.998454+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:24:33.188537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.994875+0000","last_change":"2024-09-21T06:34:30.946334+0000","last_active":"2024-09-21T06:35:04.994875+0000","last_peered":"2024-09-21T06:35:04.994875+0000","last_clean":"2024-09-21T06:35:04.994875+0000","last_became_active":"2024-09-21T06:34:30.945908+0000","last_became_peered":"2024-09-21T06:34:30.945908+0000","last_unstale":"2024-09-21T06:35:04.994875+0000","last_undegraded":"2024-09-21T06:35:04.994875+0000","last_fullsized":"2024-09-21T06:35:04.994875+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:46.746527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995193+0000","last_change":"2024-09-21T06:34:39.707909+0000","last_active":"2024-09-21T06:35:04.995193+0000","last_peered":"2024-09-21T06:35:04.995193+0000","last_clean":"2024-09-21T06:35:04.995193+0000","last_became_active":"2024-09-21T06:34:39.707492+0000","last_became_peered":"2024-09-21T06:34:39.707492+0000","last_unstale":"2024-09-21T06:35:04.995193+0000","last_undegraded":"2024-09-21T06:35:04.995193+0000","last_fullsized":"2024-09-21T06:35:04.995193+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:56.201212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.994774+0000","last_change":"2024-09-21T06:34:32.635870+0000","last_active":"2024-09-21T06:35:04.994774+0000","last_peered":"2024-09-21T06:35:04.994774+0000","last_clean":"2024-09-21T06:35:04.994774+0000","last_became_active":"2024-09-21T06:34:32.635252+0000","last_became_peered":"2024-09-21T06:34:32.635252+0000","last_unstale":"2024-09-21T06:35:04.994774+0000","last_undegraded":"2024-09-21T06:35:04.994774+0000","last_fullsized":"2024-09-21T06:35:04.994774+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:10.613033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029607+0000","last_change":"2024-09-21T06:34:30.634360+0000","last_active":"2024-09-21T06:35:05.029607+0000","last_peered":"2024-09-21T06:35:05.029607+0000","last_clean":"2024-09-21T06:35:05.029607+0000","last_became_active":"2024-09-21T06:34:30.633845+0000","last_became_peered":"2024-09-21T06:34:30.633845+0000","last_unstale":"2024-09-21T06:35:05.029607+0000","last_undegraded":"2024-09-21T06:35:05.029607+0000","last_fullsized":"2024-09-21T06:35:05.029607+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:34:53.388670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999643+0000","last_change":"2024-09-21T06:34:39.719620+0000","last_active":"2024-09-21T06:35:04.999643+0000","last_peered":"2024-09-21T06:35:04.999643+0000","last_clean":"2024-09-21T06:35:04.999643+0000","last_became_active":"2024-09-21T06:34:39.718786+0000","last_became_peered":"2024-09-21T06:34:39.718786+0000","last_unstale":"2024-09-21T06:35:04.999643+0000","last_undegraded":"2024-09-21T06:35:04.999643+0000","last_fullsized":"2024-09-21T06:35:04.999643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:43.524932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.834429+0000","last_change":"2024-09-21T06:34:32.634970+0000","last_active":"2024-09-21T06:36:04.834429+0000","last_peered":"2024-09-21T06:36:04.834429+0000","last_clean":"2024-09-21T06:36:04.834429+0000","last_became_active":"2024-09-21T06:34:32.634611+0000","last_became_peered":"2024-09-21T06:34:32.634611+0000","last_unstale":"2024-09-21T06:36:04.834429+0000","last_undegraded":"2024-09-21T06:36:04.834429+0000","last_fullsized":"2024-09-21T06:36:04.834429+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:37.855849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997918+0000","last_change":"2024-09-21T06:34:30.628123+0000","last_active":"2024-09-21T06:35:04.997918+0000","last_peered":"2024-09-21T06:35:04.997918+0000","last_clean":"2024-09-21T06:35:04.997918+0000","last_became_active":"2024-09-21T06:34:30.627498+0000","last_became_peered":"2024-09-21T06:34:30.627498+0000","last_unstale":"2024-09-21T06:35:04.997918+0000","last_undegraded":"2024-09-21T06:35:04.997918+0000","last_fullsized":"2024-09-21T06:35:04.997918+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:42.576441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":111,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029743+0000","last_change":"2024-09-21T06:33:16.856623+0000","last_active":"2024-09-21T06:35:05.029743+0000","last_peered":"2024-09-21T06:35:05.029743+0000","last_clean":"2024-09-21T06:35:05.029743+0000","last_became_active":"2024-09-21T06:33:16.850787+0000","last_became_peered":"2024-09-21T06:33:16.850787+0000","last_unstale":"2024-09-21T06:35:05.029743+0000","last_undegraded":"2024-09-21T06:35:05.029743+0000","last_fullsized":"2024-09-21T06:35:05.029743+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:50.283252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:50.283252+0000","last_clean_scrub_stamp":"2024-09-21T06:29:50.283252+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:29:20.711208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029992+0000","last_change":"2024-09-21T06:34:39.706472+0000","last_active":"2024-09-21T06:35:05.029992+0000","last_peered":"2024-09-21T06:35:05.029992+0000","last_clean":"2024-09-21T06:35:05.029992+0000","last_became_active":"2024-09-21T06:34:39.705724+0000","last_became_peered":"2024-09-21T06:34:39.705724+0000","last_unstale":"2024-09-21T06:35:05.029992+0000","last_undegraded":"2024-09-21T06:35:05.029992+0000","last_fullsized":"2024-09-21T06:35:05.029992+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:43.238704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.030085+0000","last_change":"2024-09-21T06:34:32.645944+0000","last_active":"2024-09-21T06:35:05.030085+0000","last_peered":"2024-09-21T06:35:05.030085+0000","last_clean":"2024-09-21T06:35:05.030085+0000","last_became_active":"2024-09-21T06:34:32.645134+0000","last_became_peered":"2024-09-21T06:34:32.645134+0000","last_unstale":"2024-09-21T06:35:05.030085+0000","last_undegraded":"2024-09-21T06:35:05.030085+0000","last_fullsized":"2024-09-21T06:35:05.030085+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:45.254164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.835407+0000","last_change":"2024-09-21T06:34:30.946296+0000","last_active":"2024-09-21T06:36:04.835407+0000","last_peered":"2024-09-21T06:36:04.835407+0000","last_clean":"2024-09-21T06:36:04.835407+0000","last_became_active":"2024-09-21T06:34:30.945777+0000","last_became_peered":"2024-09-21T06:34:30.945777+0000","last_unstale":"2024-09-21T06:36:04.835407+0000","last_undegraded":"2024-09-21T06:36:04.835407+0000","last_fullsized":"2024-09-21T06:36:04.835407+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:32:25.988512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995681+0000","last_change":"2024-09-21T06:34:39.705876+0000","last_active":"2024-09-21T06:35:04.995681+0000","last_peered":"2024-09-21T06:35:04.995681+0000","last_clean":"2024-09-21T06:35:04.995681+0000","last_became_active":"2024-09-21T06:34:39.705132+0000","last_became_peered":"2024-09-21T06:34:39.705132+0000","last_unstale":"2024-09-21T06:35:04.995681+0000","last_undegraded":"2024-09-21T06:35:04.995681+0000","last_fullsized":"2024-09-21T06:35:04.995681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:18.180244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029892+0000","last_change":"2024-09-21T06:34:32.638242+0000","last_active":"2024-09-21T06:35:05.029892+0000","last_peered":"2024-09-21T06:35:05.029892+0000","last_clean":"2024-09-21T06:35:05.029892+0000","last_became_active":"2024-09-21T06:34:32.637816+0000","last_became_peered":"2024-09-21T06:34:32.637816+0000","last_unstale":"2024-09-21T06:35:05.029892+0000","last_undegraded":"2024-09-21T06:35:05.029892+0000","last_fullsized":"2024-09-21T06:35:05.029892+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:40.926147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000042+0000","last_change":"2024-09-21T06:34:30.632173+0000","last_active":"2024-09-21T06:35:05.000042+0000","last_peered":"2024-09-21T06:35:05.000042+0000","last_clean":"2024-09-21T06:35:05.000042+0000","last_became_active":"2024-09-21T06:34:30.631638+0000","last_became_peered":"2024-09-21T06:34:30.631638+0000","last_unstale":"2024-09-21T06:35:05.000042+0000","last_undegraded":"2024-09-21T06:35:05.000042+0000","last_fullsized":"2024-09-21T06:35:05.000042+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:29.763272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000157+0000","last_change":"2024-09-21T06:34:39.714620+0000","last_active":"2024-09-21T06:35:05.000157+0000","last_peered":"2024-09-21T06:35:05.000157+0000","last_clean":"2024-09-21T06:35:05.000157+0000","last_became_active":"2024-09-21T06:34:39.714199+0000","last_became_peered":"2024-09-21T06:34:39.714199+0000","last_unstale":"2024-09-21T06:35:05.000157+0000","last_undegraded":"2024-09-21T06:35:05.000157+0000","last_fullsized":"2024-09-21T06:35:05.000157+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:37.041249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.030100+0000","last_change":"2024-09-21T06:34:32.645001+0000","last_active":"2024-09-21T06:35:05.030100+0000","last_peered":"2024-09-21T06:35:05.030100+0000","last_clean":"2024-09-21T06:35:05.030100+0000","last_became_active":"2024-09-21T06:34:32.644584+0000","last_became_peered":"2024-09-21T06:34:32.644584+0000","last_unstale":"2024-09-21T06:35:05.030100+0000","last_undegraded":"2024-09-21T06:35:05.030100+0000","last_fullsized":"2024-09-21T06:35:05.030100+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:47.098850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995420+0000","last_change":"2024-09-21T06:34:30.632593+0000","last_active":"2024-09-21T06:35:04.995420+0000","last_peered":"2024-09-21T06:35:04.995420+0000","last_clean":"2024-09-21T06:35:04.995420+0000","last_became_active":"2024-09-21T06:34:30.632170+0000","last_became_peered":"2024-09-21T06:34:30.632170+0000","last_unstale":"2024-09-21T06:35:04.995420+0000","last_undegraded":"2024-09-21T06:35:04.995420+0000","last_fullsized":"2024-09-21T06:35:04.995420+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:56.170655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.998214+0000","last_change":"2024-09-21T06:34:39.710172+0000","last_active":"2024-09-21T06:35:04.998214+0000","last_peered":"2024-09-21T06:35:04.998214+0000","last_clean":"2024-09-21T06:35:04.998214+0000","last_became_active":"2024-09-21T06:34:39.708876+0000","last_became_peered":"2024-09-21T06:34:39.708876+0000","last_unstale":"2024-09-21T06:35:04.998214+0000","last_undegraded":"2024-09-21T06:35:04.998214+0000","last_fullsized":"2024-09-21T06:35:04.998214+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:32.371873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000997+0000","last_change":"2024-09-21T06:34:32.651056+0000","last_active":"2024-09-21T06:35:05.000997+0000","last_peered":"2024-09-21T06:35:05.000997+0000","last_clean":"2024-09-21T06:35:05.000997+0000","last_became_active":"2024-09-21T06:34:32.650676+0000","last_became_peered":"2024-09-21T06:34:32.650676+0000","last_unstale":"2024-09-21T06:35:05.000997+0000","last_undegraded":"2024-09-21T06:35:05.000997+0000","last_fullsized":"2024-09-21T06:35:05.000997+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:34.599654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.835778+0000","last_change":"2024-09-21T06:34:30.632307+0000","last_active":"2024-09-21T06:36:04.835778+0000","last_peered":"2024-09-21T06:36:04.835778+0000","last_clean":"2024-09-21T06:36:04.835778+0000","last_became_active":"2024-09-21T06:34:30.631920+0000","last_became_peered":"2024-09-21T06:34:30.631920+0000","last_unstale":"2024-09-21T06:36:04.835778+0000","last_undegraded":"2024-09-21T06:36:04.835778+0000","last_fullsized":"2024-09-21T06:36:04.835778+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:46.940898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.998452+0000","last_change":"2024-09-21T06:34:39.715056+0000","last_active":"2024-09-21T06:35:04.998452+0000","last_peered":"2024-09-21T06:35:04.998452+0000","last_clean":"2024-09-21T06:35:04.998452+0000","last_became_active":"2024-09-21T06:34:39.714563+0000","last_became_peered":"2024-09-21T06:34:39.714563+0000","last_unstale":"2024-09-21T06:35:04.998452+0000","last_undegraded":"2024-09-21T06:35:04.998452+0000","last_fullsized":"2024-09-21T06:35:04.998452+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:48.567057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.998840+0000","last_change":"2024-09-21T06:34:32.646934+0000","last_active":"2024-09-21T06:35:04.998840+0000","last_peered":"2024-09-21T06:35:04.998840+0000","last_clean":"2024-09-21T06:35:04.998840+0000","last_became_active":"2024-09-21T06:34:32.646559+0000","last_became_peered":"2024-09-21T06:34:32.646559+0000","last_unstale":"2024-09-21T06:35:04.998840+0000","last_undegraded":"2024-09-21T06:35:04.998840+0000","last_fullsized":"2024-09-21T06:35:04.998840+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:37.802590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":44,"reported_epoch":83,"state":"peering","last_fresh":"2024-09-21T06:35:05.004087+0000","last_change":"2024-09-21T06:36:04.834224+0000","last_active":"2024-09-21T06:36:04.834224+0000","last_peered":"2024-09-21T06:35:05.004087+0000","last_clean":"2024-09-21T06:35:05.004087+0000","last_became_active":"2024-09-21T06:34:30.946327+0000","last_became_peered":"2024-09-21T06:34:30.946327+0000","last_unstale":"2024-09-21T06:35:05.004087+0000","last_undegraded":"2024-09-21T06:35:05.004087+0000","last_fullsized":"2024-09-21T06:35:05.004087+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1,2],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995322+0000","last_change":"2024-09-21T06:34:39.713398+0000","last_active":"2024-09-21T06:35:04.995322+0000","last_peered":"2024-09-21T06:35:04.995322+0000","last_clean":"2024-09-21T06:35:04.995322+0000","last_became_active":"2024-09-21T06:34:39.712954+0000","last_became_peered":"2024-09-21T06:34:39.712954+0000","last_unstale":"2024-09-21T06:35:04.995322+0000","last_undegraded":"2024-09-21T06:35:04.995322+0000","last_fullsized":"2024-09-21T06:35:04.995322+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:58.881014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029465+0000","last_change":"2024-09-21T06:34:32.640879+0000","last_active":"2024-09-21T06:35:05.029465+0000","last_peered":"2024-09-21T06:35:05.029465+0000","last_clean":"2024-09-21T06:35:05.029465+0000","last_became_active":"2024-09-21T06:34:32.640526+0000","last_became_peered":"2024-09-21T06:34:32.640526+0000","last_unstale":"2024-09-21T06:35:05.029465+0000","last_undegraded":"2024-09-21T06:35:05.029465+0000","last_fullsized":"2024-09-21T06:35:05.029465+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:12.327195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997068+0000","last_change":"2024-09-21T06:34:30.631449+0000","last_active":"2024-09-21T06:35:04.997068+0000","last_peered":"2024-09-21T06:35:04.997068+0000","last_clean":"2024-09-21T06:35:04.997068+0000","last_became_active":"2024-09-21T06:34:30.631081+0000","last_became_peered":"2024-09-21T06:34:30.631081+0000","last_unstale":"2024-09-21T06:35:04.997068+0000","last_undegraded":"2024-09-21T06:35:04.997068+0000","last_fullsized":"2024-09-21T06:35:04.997068+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:09.202696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.114180+0000","last_change":"2024-09-21T06:34:39.707379+0000","last_active":"2024-09-21T06:35:05.114180+0000","last_peered":"2024-09-21T06:35:05.114180+0000","last_clean":"2024-09-21T06:35:05.114180+0000","last_became_active":"2024-09-21T06:34:39.706979+0000","last_became_peered":"2024-09-21T06:34:39.706979+0000","last_unstale":"2024-09-21T06:35:05.114180+0000","last_undegraded":"2024-09-21T06:35:05.114180+0000","last_fullsized":"2024-09-21T06:35:05.114180+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:24.394233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029782+0000","last_change":"2024-09-21T06:34:32.649458+0000","last_active":"2024-09-21T06:35:05.029782+0000","last_peered":"2024-09-21T06:35:05.029782+0000","last_clean":"2024-09-21T06:35:05.029782+0000","last_became_active":"2024-09-21T06:34:32.649051+0000","last_became_peered":"2024-09-21T06:34:32.649051+0000","last_unstale":"2024-09-21T06:35:05.029782+0000","last_undegraded":"2024-09-21T06:35:05.029782+0000","last_fullsized":"2024-09-21T06:35:05.029782+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:12.631446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999624+0000","last_change":"2024-09-21T06:34:30.634592+0000","last_active":"2024-09-21T06:35:04.999624+0000","last_peered":"2024-09-21T06:35:04.999624+0000","last_clean":"2024-09-21T06:35:04.999624+0000","last_became_active":"2024-09-21T06:34:30.634251+0000","last_became_peered":"2024-09-21T06:34:30.634251+0000","last_unstale":"2024-09-21T06:35:04.999624+0000","last_undegraded":"2024-09-21T06:35:04.999624+0000","last_fullsized":"2024-09-21T06:35:04.999624+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:32.817467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000642+0000","last_change":"2024-09-21T06:34:39.719416+0000","last_active":"2024-09-21T06:35:05.000642+0000","last_peered":"2024-09-21T06:35:05.000642+0000","last_clean":"2024-09-21T06:35:05.000642+0000","last_became_active":"2024-09-21T06:34:39.718910+0000","last_became_peered":"2024-09-21T06:34:39.718910+0000","last_unstale":"2024-09-21T06:35:05.000642+0000","last_undegraded":"2024-09-21T06:35:05.000642+0000","last_fullsized":"2024-09-21T06:35:05.000642+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:41.413250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.994958+0000","last_change":"2024-09-21T06:34:32.640727+0000","last_active":"2024-09-21T06:35:04.994958+0000","last_peered":"2024-09-21T06:35:04.994958+0000","last_clean":"2024-09-21T06:35:04.994958+0000","last_became_active":"2024-09-21T06:34:32.640296+0000","last_became_peered":"2024-09-21T06:34:32.640296+0000","last_unstale":"2024-09-21T06:35:04.994958+0000","last_undegraded":"2024-09-21T06:35:04.994958+0000","last_fullsized":"2024-09-21T06:35:04.994958+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:35.124059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.834648+0000","last_change":"2024-09-21T06:34:30.633332+0000","last_active":"2024-09-21T06:36:04.834648+0000","last_peered":"2024-09-21T06:36:04.834648+0000","last_clean":"2024-09-21T06:36:04.834648+0000","last_became_active":"2024-09-21T06:34:30.632641+0000","last_became_peered":"2024-09-21T06:34:30.632641+0000","last_unstale":"2024-09-21T06:36:04.834648+0000","last_undegraded":"2024-09-21T06:36:04.834648+0000","last_fullsized":"2024-09-21T06:36:04.834648+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:30:23.930313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999766+0000","last_change":"2024-09-21T06:34:39.712504+0000","last_active":"2024-09-21T06:35:04.999766+0000","last_peered":"2024-09-21T06:35:04.999766+0000","last_clean":"2024-09-21T06:35:04.999766+0000","last_became_active":"2024-09-21T06:34:39.711638+0000","last_became_peered":"2024-09-21T06:34:39.711638+0000","last_unstale":"2024-09-21T06:35:04.999766+0000","last_undegraded":"2024-09-21T06:35:04.999766+0000","last_fullsized":"2024-09-21T06:35:04.999766+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:26:09.003367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.835429+0000","last_change":"2024-09-21T06:34:32.639582+0000","last_active":"2024-09-21T06:36:04.835429+0000","last_peered":"2024-09-21T06:36:04.835429+0000","last_clean":"2024-09-21T06:36:04.835429+0000","last_became_active":"2024-09-21T06:34:32.639023+0000","last_became_peered":"2024-09-21T06:34:32.639023+0000","last_unstale":"2024-09-21T06:36:04.835429+0000","last_undegraded":"2024-09-21T06:36:04.835429+0000","last_fullsized":"2024-09-21T06:36:04.835429+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:26.773389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"83'30","reported_seq":190,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.127841+0000","last_change":"2024-09-21T06:34:30.626280+0000","last_active":"2024-09-21T06:36:05.127841+0000","last_peered":"2024-09-21T06:36:05.127841+0000","last_clean":"2024-09-21T06:36:05.127841+0000","last_became_active":"2024-09-21T06:34:30.625662+0000","last_became_peered":"2024-09-21T06:34:30.625662+0000","last_unstale":"2024-09-21T06:36:05.127841+0000","last_undegraded":"2024-09-21T06:36:05.127841+0000","last_fullsized":"2024-09-21T06:36:05.127841+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:10.541881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68785,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":123,"num_read_kb":129,"num_write":30,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.113723+0000","last_change":"2024-09-21T06:34:39.708999+0000","last_active":"2024-09-21T06:35:05.113723+0000","last_peered":"2024-09-21T06:35:05.113723+0000","last_clean":"2024-09-21T06:35:05.113723+0000","last_became_active":"2024-09-21T06:34:39.708580+0000","last_became_peered":"2024-09-21T06:34:39.708580+0000","last_unstale":"2024-09-21T06:35:05.113723+0000","last_undegraded":"2024-09-21T06:35:05.113723+0000","last_fullsized":"2024-09-21T06:35:05.113723+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:07.648358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995798+0000","last_change":"2024-09-21T06:34:32.651157+0000","last_active":"2024-09-21T06:35:04.995798+0000","last_peered":"2024-09-21T06:35:04.995798+0000","last_clean":"2024-09-21T06:35:04.995798+0000","last_became_active":"2024-09-21T06:34:32.650804+0000","last_became_peered":"2024-09-21T06:34:32.650804+0000","last_unstale":"2024-09-21T06:35:04.995798+0000","last_undegraded":"2024-09-21T06:35:04.995798+0000","last_fullsized":"2024-09-21T06:35:04.995798+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:49.758496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999901+0000","last_change":"2024-09-21T06:34:30.947142+0000","last_active":"2024-09-21T06:35:04.999901+0000","last_peered":"2024-09-21T06:35:04.999901+0000","last_clean":"2024-09-21T06:35:04.999901+0000","last_became_active":"2024-09-21T06:34:30.946748+0000","last_became_peered":"2024-09-21T06:34:30.946748+0000","last_unstale":"2024-09-21T06:35:04.999901+0000","last_undegraded":"2024-09-21T06:35:04.999901+0000","last_fullsized":"2024-09-21T06:35:04.999901+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:39.222286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997754+0000","last_change":"2024-09-21T06:34:39.709266+0000","last_active":"2024-09-21T06:35:04.997754+0000","last_peered":"2024-09-21T06:35:04.997754+0000","last_clean":"2024-09-21T06:35:04.997754+0000","last_became_active":"2024-09-21T06:34:39.708753+0000","last_became_peered":"2024-09-21T06:34:39.708753+0000","last_unstale":"2024-09-21T06:35:04.997754+0000","last_undegraded":"2024-09-21T06:35:04.997754+0000","last_fullsized":"2024-09-21T06:35:04.997754+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:48.711681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000708+0000","last_change":"2024-09-21T06:34:32.637976+0000","last_active":"2024-09-21T06:35:05.000708+0000","last_peered":"2024-09-21T06:35:05.000708+0000","last_clean":"2024-09-21T06:35:05.000708+0000","last_became_active":"2024-09-21T06:34:32.637267+0000","last_became_peered":"2024-09-21T06:34:32.637267+0000","last_unstale":"2024-09-21T06:35:05.000708+0000","last_undegraded":"2024-09-21T06:35:05.000708+0000","last_fullsized":"2024-09-21T06:35:05.000708+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:31.340583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.030056+0000","last_change":"2024-09-21T06:34:30.631059+0000","last_active":"2024-09-21T06:35:05.030056+0000","last_peered":"2024-09-21T06:35:05.030056+0000","last_clean":"2024-09-21T06:35:05.030056+0000","last_became_active":"2024-09-21T06:34:30.630633+0000","last_became_peered":"2024-09-21T06:34:30.630633+0000","last_unstale":"2024-09-21T06:35:05.030056+0000","last_undegraded":"2024-09-21T06:35:05.030056+0000","last_fullsized":"2024-09-21T06:35:05.030056+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:09.049438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.113457+0000","last_change":"2024-09-21T06:34:39.713831+0000","last_active":"2024-09-21T06:35:05.113457+0000","last_peered":"2024-09-21T06:35:05.113457+0000","last_clean":"2024-09-21T06:35:05.113457+0000","last_became_active":"2024-09-21T06:34:39.713403+0000","last_became_peered":"2024-09-21T06:34:39.713403+0000","last_unstale":"2024-09-21T06:35:05.113457+0000","last_undegraded":"2024-09-21T06:35:05.113457+0000","last_fullsized":"2024-09-21T06:35:05.113457+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:22.201063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.996367+0000","last_change":"2024-09-21T06:34:32.646924+0000","last_active":"2024-09-21T06:35:04.996367+0000","last_peered":"2024-09-21T06:35:04.996367+0000","last_clean":"2024-09-21T06:35:04.996367+0000","last_became_active":"2024-09-21T06:34:32.646531+0000","last_became_peered":"2024-09-21T06:34:32.646531+0000","last_unstale":"2024-09-21T06:35:04.996367+0000","last_undegraded":"2024-09-21T06:35:04.996367+0000","last_fullsized":"2024-09-21T06:35:04.996367+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:48.550193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.833967+0000","last_change":"2024-09-21T06:34:30.631901+0000","last_active":"2024-09-21T06:36:04.833967+0000","last_peered":"2024-09-21T06:36:04.833967+0000","last_clean":"2024-09-21T06:36:04.833967+0000","last_became_active":"2024-09-21T06:34:30.631450+0000","last_became_peered":"2024-09-21T06:34:30.631450+0000","last_unstale":"2024-09-21T06:36:04.833967+0000","last_undegraded":"2024-09-21T06:36:04.833967+0000","last_fullsized":"2024-09-21T06:36:04.833967+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:20.709456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.833960+0000","last_change":"2024-09-21T06:34:39.711951+0000","last_active":"2024-09-21T06:36:04.833960+0000","last_peered":"2024-09-21T06:36:04.833960+0000","last_clean":"2024-09-21T06:36:04.833960+0000","last_became_active":"2024-09-21T06:34:39.710783+0000","last_became_peered":"2024-09-21T06:34:39.710783+0000","last_unstale":"2024-09-21T06:36:04.833960+0000","last_undegraded":"2024-09-21T06:36:04.833960+0000","last_fullsized":"2024-09-21T06:36:04.833960+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:27:58.501155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000355+0000","last_change":"2024-09-21T06:34:39.709639+0000","last_active":"2024-09-21T06:35:05.000355+0000","last_peered":"2024-09-21T06:35:05.000355+0000","last_clean":"2024-09-21T06:35:05.000355+0000","last_became_active":"2024-09-21T06:34:39.709241+0000","last_became_peered":"2024-09-21T06:34:39.709241+0000","last_unstale":"2024-09-21T06:35:05.000355+0000","last_undegraded":"2024-09-21T06:35:05.000355+0000","last_fullsized":"2024-09-21T06:35:05.000355+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.357399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.113674+0000","last_change":"2024-09-21T06:34:32.635365+0000","last_active":"2024-09-21T06:35:05.113674+0000","last_peered":"2024-09-21T06:35:05.113674+0000","last_clean":"2024-09-21T06:35:05.113674+0000","last_became_active":"2024-09-21T06:34:32.635012+0000","last_became_peered":"2024-09-21T06:34:32.635012+0000","last_unstale":"2024-09-21T06:35:05.113674+0000","last_undegraded":"2024-09-21T06:35:05.113674+0000","last_fullsized":"2024-09-21T06:35:05.113674+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:54.671074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.998620+0000","last_change":"2024-09-21T06:34:30.633347+0000","last_active":"2024-09-21T06:35:04.998620+0000","last_peered":"2024-09-21T06:35:04.998620+0000","last_clean":"2024-09-21T06:35:04.998620+0000","last_became_active":"2024-09-21T06:34:30.632513+0000","last_became_peered":"2024-09-21T06:34:30.632513+0000","last_unstale":"2024-09-21T06:35:04.998620+0000","last_undegraded":"2024-09-21T06:35:04.998620+0000","last_fullsized":"2024-09-21T06:35:04.998620+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:12.439109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.001150+0000","last_change":"2024-09-21T06:34:39.710204+0000","last_active":"2024-09-21T06:35:05.001150+0000","last_peered":"2024-09-21T06:35:05.001150+0000","last_clean":"2024-09-21T06:35:05.001150+0000","last_became_active":"2024-09-21T06:34:39.709744+0000","last_became_peered":"2024-09-21T06:34:39.709744+0000","last_unstale":"2024-09-21T06:35:05.001150+0000","last_undegraded":"2024-09-21T06:35:05.001150+0000","last_fullsized":"2024-09-21T06:35:05.001150+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:45.484088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.833992+0000","last_change":"2024-09-21T06:34:32.634170+0000","last_active":"2024-09-21T06:36:04.833992+0000","last_peered":"2024-09-21T06:36:04.833992+0000","last_clean":"2024-09-21T06:36:04.833992+0000","last_became_active":"2024-09-21T06:34:32.633831+0000","last_became_peered":"2024-09-21T06:34:32.633831+0000","last_unstale":"2024-09-21T06:36:04.833992+0000","last_undegraded":"2024-09-21T06:36:04.833992+0000","last_fullsized":"2024-09-21T06:36:04.833992+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:36.738279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.994732+0000","last_change":"2024-09-21T06:34:30.629422+0000","last_active":"2024-09-21T06:35:04.994732+0000","last_peered":"2024-09-21T06:35:04.994732+0000","last_clean":"2024-09-21T06:35:04.994732+0000","last_became_active":"2024-09-21T06:34:30.629025+0000","last_became_peered":"2024-09-21T06:34:30.629025+0000","last_unstale":"2024-09-21T06:35:04.994732+0000","last_undegraded":"2024-09-21T06:35:04.994732+0000","last_fullsized":"2024-09-21T06:35:04.994732+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:05.068816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.834375+0000","last_change":"2024-09-21T06:34:30.634562+0000","last_active":"2024-09-21T06:36:04.834375+0000","last_peered":"2024-09-21T06:36:04.834375+0000","last_clean":"2024-09-21T06:36:04.834375+0000","last_became_active":"2024-09-21T06:34:30.634170+0000","last_became_peered":"2024-09-21T06:34:30.634170+0000","last_unstale":"2024-09-21T06:36:04.834375+0000","last_undegraded":"2024-09-21T06:36:04.834375+0000","last_fullsized":"2024-09-21T06:36:04.834375+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:15.243103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999225+0000","last_change":"2024-09-21T06:34:32.645718+0000","last_active":"2024-09-21T06:35:04.999225+0000","last_peered":"2024-09-21T06:35:04.999225+0000","last_clean":"2024-09-21T06:35:04.999225+0000","last_became_active":"2024-09-21T06:34:32.645072+0000","last_became_peered":"2024-09-21T06:34:32.645072+0000","last_unstale":"2024-09-21T06:35:04.999225+0000","last_undegraded":"2024-09-21T06:35:04.999225+0000","last_fullsized":"2024-09-21T06:35:04.999225+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:54.109246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.834864+0000","last_change":"2024-09-21T06:34:39.716947+0000","last_active":"2024-09-21T06:36:04.834864+0000","last_peered":"2024-09-21T06:36:04.834864+0000","last_clean":"2024-09-21T06:36:04.834864+0000","last_became_active":"2024-09-21T06:34:39.716540+0000","last_became_peered":"2024-09-21T06:34:39.716540+0000","last_unstale":"2024-09-21T06:36:04.834864+0000","last_undegraded":"2024-09-21T06:36:04.834864+0000","last_fullsized":"2024-09-21T06:36:04.834864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:12.748444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999360+0000","last_change":"2024-09-21T06:34:30.636389+0000","last_active":"2024-09-21T06:35:04.999360+0000","last_peered":"2024-09-21T06:35:04.999360+0000","last_clean":"2024-09-21T06:35:04.999360+0000","last_became_active":"2024-09-21T06:34:30.636008+0000","last_became_peered":"2024-09-21T06:34:30.636008+0000","last_unstale":"2024-09-21T06:35:04.999360+0000","last_undegraded":"2024-09-21T06:35:04.999360+0000","last_fullsized":"2024-09-21T06:35:04.999360+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:45.476267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.833807+0000","last_change":"2024-09-21T06:34:32.635644+0000","last_active":"2024-09-21T06:36:04.833807+0000","last_peered":"2024-09-21T06:36:04.833807+0000","last_clean":"2024-09-21T06:36:04.833807+0000","last_became_active":"2024-09-21T06:34:32.635253+0000","last_became_peered":"2024-09-21T06:34:32.635253+0000","last_unstale":"2024-09-21T06:36:04.833807+0000","last_undegraded":"2024-09-21T06:36:04.833807+0000","last_fullsized":"2024-09-21T06:36:04.833807+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:34.713656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.835922+0000","last_change":"2024-09-21T06:34:39.710382+0000","last_active":"2024-09-21T06:36:04.835922+0000","last_peered":"2024-09-21T06:36:04.835922+0000","last_clean":"2024-09-21T06:36:04.835922+0000","last_became_active":"2024-09-21T06:34:39.709884+0000","last_became_peered":"2024-09-21T06:34:39.709884+0000","last_unstale":"2024-09-21T06:36:04.835922+0000","last_undegraded":"2024-09-21T06:36:04.835922+0000","last_fullsized":"2024-09-21T06:36:04.835922+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:01.602366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000463+0000","last_change":"2024-09-21T06:34:32.640025+0000","last_active":"2024-09-21T06:35:05.000463+0000","last_peered":"2024-09-21T06:35:05.000463+0000","last_clean":"2024-09-21T06:35:05.000463+0000","last_became_active":"2024-09-21T06:34:32.639648+0000","last_became_peered":"2024-09-21T06:34:32.639648+0000","last_unstale":"2024-09-21T06:35:05.000463+0000","last_undegraded":"2024-09-21T06:35:05.000463+0000","last_fullsized":"2024-09-21T06:35:05.000463+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:05.833065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.836918+0000","last_change":"2024-09-21T06:35:05.115344+0000","last_active":"2024-09-21T06:36:04.836918+0000","last_peered":"2024-09-21T06:36:04.836918+0000","last_clean":"2024-09-21T06:36:04.836918+0000","last_became_active":"2024-09-21T06:35:05.114678+0000","last_became_peered":"2024-09-21T06:35:05.114678+0000","last_unstale":"2024-09-21T06:36:04.836918+0000","last_undegraded":"2024-09-21T06:36:04.836918+0000","last_fullsized":"2024-09-21T06:36:04.836918+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:24.593145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.834295+0000","last_change":"2024-09-21T06:35:05.115301+0000","last_active":"2024-09-21T06:36:04.834295+0000","last_peered":"2024-09-21T06:36:04.834295+0000","last_clean":"2024-09-21T06:36:04.834295+0000","last_became_active":"2024-09-21T06:35:05.114706+0000","last_became_peered":"2024-09-21T06:35:05.114706+0000","last_unstale":"2024-09-21T06:36:04.834295+0000","last_undegraded":"2024-09-21T06:36:04.834295+0000","last_fullsized":"2024-09-21T06:36:04.834295+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:09.513594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000261+0000","last_change":"2024-09-21T06:34:32.634679+0000","last_active":"2024-09-21T06:35:05.000261+0000","last_peered":"2024-09-21T06:35:05.000261+0000","last_clean":"2024-09-21T06:35:05.000261+0000","last_became_active":"2024-09-21T06:34:32.634285+0000","last_became_peered":"2024-09-21T06:34:32.634285+0000","last_unstale":"2024-09-21T06:35:05.000261+0000","last_undegraded":"2024-09-21T06:35:05.000261+0000","last_fullsized":"2024-09-21T06:35:05.000261+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:15.185453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995303+0000","last_change":"2024-09-21T06:34:32.639037+0000","last_active":"2024-09-21T06:35:04.995303+0000","last_peered":"2024-09-21T06:35:04.995303+0000","last_clean":"2024-09-21T06:35:04.995303+0000","last_became_active":"2024-09-21T06:34:32.638689+0000","last_became_peered":"2024-09-21T06:34:32.638689+0000","last_unstale":"2024-09-21T06:35:04.995303+0000","last_undegraded":"2024-09-21T06:35:04.995303+0000","last_fullsized":"2024-09-21T06:35:04.995303+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:13.609191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.115157+0000","last_change":"2024-09-21T06:35:05.115051+0000","last_active":"2024-09-21T06:35:05.115157+0000","last_peered":"2024-09-21T06:35:05.115157+0000","last_clean":"2024-09-21T06:35:05.115157+0000","last_became_active":"2024-09-21T06:35:05.114602+0000","last_became_peered":"2024-09-21T06:35:05.114602+0000","last_unstale":"2024-09-21T06:35:05.115157+0000","last_undegraded":"2024-09-21T06:35:05.115157+0000","last_fullsized":"2024-09-21T06:35:05.115157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:48.778188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.835155+0000","last_change":"2024-09-21T06:34:32.637237+0000","last_active":"2024-09-21T06:36:04.835155+0000","last_peered":"2024-09-21T06:36:04.835155+0000","last_clean":"2024-09-21T06:36:04.835155+0000","last_became_active":"2024-09-21T06:34:32.634105+0000","last_became_peered":"2024-09-21T06:34:32.634105+0000","last_unstale":"2024-09-21T06:36:04.835155+0000","last_undegraded":"2024-09-21T06:36:04.835155+0000","last_fullsized":"2024-09-21T06:36:04.835155+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:52.530461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.835249+0000","last_change":"2024-09-21T06:34:32.624441+0000","last_active":"2024-09-21T06:36:04.835249+0000","last_peered":"2024-09-21T06:36:04.835249+0000","last_clean":"2024-09-21T06:36:04.835249+0000","last_became_active":"2024-09-21T06:34:32.623974+0000","last_became_peered":"2024-09-21T06:34:32.623974+0000","last_unstale":"2024-09-21T06:36:04.835249+0000","last_undegraded":"2024-09-21T06:36:04.835249+0000","last_fullsized":"2024-09-21T06:36:04.835249+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:12.378113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.998387+0000","last_change":"2024-09-21T06:34:32.633879+0000","last_active":"2024-09-21T06:35:04.998387+0000","last_peered":"2024-09-21T06:35:04.998387+0000","last_clean":"2024-09-21T06:35:04.998387+0000","last_became_active":"2024-09-21T06:34:32.631791+0000","last_became_peered":"2024-09-21T06:34:32.631791+0000","last_unstale":"2024-09-21T06:35:04.998387+0000","last_undegraded":"2024-09-21T06:35:04.998387+0000","last_fullsized":"2024-09-21T06:35:04.998387+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:46.246892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.834118+0000","last_change":"2024-09-21T06:34:32.639974+0000","last_active":"2024-09-21T06:36:04.834118+0000","last_peered":"2024-09-21T06:36:04.834118+0000","last_clean":"2024-09-21T06:36:04.834118+0000","last_became_active":"2024-09-21T06:34:32.639631+0000","last_became_peered":"2024-09-21T06:34:32.639631+0000","last_unstale":"2024-09-21T06:36:04.834118+0000","last_undegraded":"2024-09-21T06:36:04.834118+0000","last_fullsized":"2024-09-21T06:36:04.834118+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:50.069378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995887+0000","last_change":"2024-09-21T06:34:32.645403+0000","last_active":"2024-09-21T06:35:04.995887+0000","last_peered":"2024-09-21T06:35:04.995887+0000","last_clean":"2024-09-21T06:35:04.995887+0000","last_became_active":"2024-09-21T06:34:32.644986+0000","last_became_peered":"2024-09-21T06:34:32.644986+0000","last_unstale":"2024-09-21T06:35:04.995887+0000","last_undegraded":"2024-09-21T06:35:04.995887+0000","last_fullsized":"2024-09-21T06:35:04.995887+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:43.640107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995183+0000","last_change":"2024-09-21T06:34:32.639933+0000","last_active":"2024-09-21T06:35:04.995183+0000","last_peered":"2024-09-21T06:35:04.995183+0000","last_clean":"2024-09-21T06:35:04.995183+0000","last_became_active":"2024-09-21T06:34:32.639438+0000","last_became_peered":"2024-09-21T06:34:32.639438+0000","last_unstale":"2024-09-21T06:35:04.995183+0000","last_undegraded":"2024-09-21T06:35:04.995183+0000","last_fullsized":"2024-09-21T06:35:04.995183+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:18.860219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000816+0000","last_change":"2024-09-21T06:34:32.648580+0000","last_active":"2024-09-21T06:35:05.000816+0000","last_peered":"2024-09-21T06:35:05.000816+0000","last_clean":"2024-09-21T06:35:05.000816+0000","last_became_active":"2024-09-21T06:34:32.648016+0000","last_became_peered":"2024-09-21T06:34:32.648016+0000","last_unstale":"2024-09-21T06:35:05.000816+0000","last_undegraded":"2024-09-21T06:35:05.000816+0000","last_fullsized":"2024-09-21T06:35:05.000816+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:52.858069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.836427+0000","last_change":"2024-09-21T06:34:32.633376+0000","last_active":"2024-09-21T06:36:04.836427+0000","last_peered":"2024-09-21T06:36:04.836427+0000","last_clean":"2024-09-21T06:36:04.836427+0000","last_became_active":"2024-09-21T06:34:32.631052+0000","last_became_peered":"2024-09-21T06:34:32.631052+0000","last_unstale":"2024-09-21T06:36:04.836427+0000","last_undegraded":"2024-09-21T06:36:04.836427+0000","last_fullsized":"2024-09-21T06:36:04.836427+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:46.229599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.114206+0000","last_change":"2024-09-21T06:34:32.636966+0000","last_active":"2024-09-21T06:35:05.114206+0000","last_peered":"2024-09-21T06:35:05.114206+0000","last_clean":"2024-09-21T06:35:05.114206+0000","last_became_active":"2024-09-21T06:34:32.636565+0000","last_became_peered":"2024-09-21T06:34:32.636565+0000","last_unstale":"2024-09-21T06:35:05.114206+0000","last_undegraded":"2024-09-21T06:35:05.114206+0000","last_fullsized":"2024-09-21T06:35:05.114206+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:36.233082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.001311+0000","last_change":"2024-09-21T06:34:32.646895+0000","last_active":"2024-09-21T06:35:05.001311+0000","last_peered":"2024-09-21T06:35:05.001311+0000","last_clean":"2024-09-21T06:35:05.001311+0000","last_became_active":"2024-09-21T06:34:32.646505+0000","last_became_peered":"2024-09-21T06:34:32.646505+0000","last_unstale":"2024-09-21T06:35:05.001311+0000","last_undegraded":"2024-09-21T06:35:05.001311+0000","last_fullsized":"2024-09-21T06:35:05.001311+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:43.441337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995353+0000","last_change":"2024-09-21T06:34:32.645402+0000","last_active":"2024-09-21T06:35:04.995353+0000","last_peered":"2024-09-21T06:35:04.995353+0000","last_clean":"2024-09-21T06:35:04.995353+0000","last_became_active":"2024-09-21T06:34:32.645013+0000","last_became_peered":"2024-09-21T06:34:32.645013+0000","last_unstale":"2024-09-21T06:35:04.995353+0000","last_undegraded":"2024-09-21T06:35:04.995353+0000","last_fullsized":"2024-09-21T06:35:04.995353+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:33.475847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999814+0000","last_change":"2024-09-21T06:34:32.643234+0000","last_active":"2024-09-21T06:35:04.999814+0000","last_peered":"2024-09-21T06:35:04.999814+0000","last_clean":"2024-09-21T06:35:04.999814+0000","last_became_active":"2024-09-21T06:34:32.642614+0000","last_became_peered":"2024-09-21T06:34:32.642614+0000","last_unstale":"2024-09-21T06:35:04.999814+0000","last_undegraded":"2024-09-21T06:35:04.999814+0000","last_fullsized":"2024-09-21T06:35:04.999814+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:50.903500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.999942+0000","last_change":"2024-09-21T06:34:32.641887+0000","last_active":"2024-09-21T06:35:04.999942+0000","last_peered":"2024-09-21T06:35:04.999942+0000","last_clean":"2024-09-21T06:35:04.999942+0000","last_became_active":"2024-09-21T06:34:32.641490+0000","last_became_peered":"2024-09-21T06:34:32.641490+0000","last_unstale":"2024-09-21T06:35:04.999942+0000","last_undegraded":"2024-09-21T06:35:04.999942+0000","last_fullsized":"2024-09-21T06:35:04.999942+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:34.550970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995289+0000","last_change":"2024-09-21T06:34:32.641865+0000","last_active":"2024-09-21T06:35:04.995289+0000","last_peered":"2024-09-21T06:35:04.995289+0000","last_clean":"2024-09-21T06:35:04.995289+0000","last_became_active":"2024-09-21T06:34:32.641252+0000","last_became_peered":"2024-09-21T06:34:32.641252+0000","last_unstale":"2024-09-21T06:35:04.995289+0000","last_undegraded":"2024-09-21T06:35:04.995289+0000","last_fullsized":"2024-09-21T06:35:04.995289+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:44.459202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997920+0000","last_change":"2024-09-21T06:34:32.641259+0000","last_active":"2024-09-21T06:35:04.997920+0000","last_peered":"2024-09-21T06:35:04.997920+0000","last_clean":"2024-09-21T06:35:04.997920+0000","last_became_active":"2024-09-21T06:34:32.640907+0000","last_became_peered":"2024-09-21T06:34:32.640907+0000","last_unstale":"2024-09-21T06:35:04.997920+0000","last_undegraded":"2024-09-21T06:35:04.997920+0000","last_fullsized":"2024-09-21T06:35:04.997920+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:59.966857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.030054+0000","last_change":"2024-09-21T06:34:32.646095+0000","last_active":"2024-09-21T06:35:05.030054+0000","last_peered":"2024-09-21T06:35:05.030054+0000","last_clean":"2024-09-21T06:35:05.030054+0000","last_became_active":"2024-09-21T06:34:32.645604+0000","last_became_peered":"2024-09-21T06:34:32.645604+0000","last_unstale":"2024-09-21T06:35:05.030054+0000","last_undegraded":"2024-09-21T06:35:05.030054+0000","last_fullsized":"2024-09-21T06:35:05.030054+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:30.872428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.834243+0000","last_change":"2024-09-21T06:34:32.649303+0000","last_active":"2024-09-21T06:36:04.834243+0000","last_peered":"2024-09-21T06:36:04.834243+0000","last_clean":"2024-09-21T06:36:04.834243+0000","last_became_active":"2024-09-21T06:34:32.648965+0000","last_became_peered":"2024-09-21T06:34:32.648965+0000","last_unstale":"2024-09-21T06:36:04.834243+0000","last_undegraded":"2024-09-21T06:36:04.834243+0000","last_fullsized":"2024-09-21T06:36:04.834243+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:51.335949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.996335+0000","last_change":"2024-09-21T06:34:32.637261+0000","last_active":"2024-09-21T06:35:04.996335+0000","last_peered":"2024-09-21T06:35:04.996335+0000","last_clean":"2024-09-21T06:35:04.996335+0000","last_became_active":"2024-09-21T06:34:32.636861+0000","last_became_peered":"2024-09-21T06:34:32.636861+0000","last_unstale":"2024-09-21T06:35:04.996335+0000","last_undegraded":"2024-09-21T06:35:04.996335+0000","last_fullsized":"2024-09-21T06:35:04.996335+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:55.271027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.835379+0000","last_change":"2024-09-21T06:34:32.635156+0000","last_active":"2024-09-21T06:36:04.835379+0000","last_peered":"2024-09-21T06:36:04.835379+0000","last_clean":"2024-09-21T06:36:04.835379+0000","last_became_active":"2024-09-21T06:34:32.634802+0000","last_became_peered":"2024-09-21T06:34:32.634802+0000","last_unstale":"2024-09-21T06:36:04.835379+0000","last_undegraded":"2024-09-21T06:36:04.835379+0000","last_fullsized":"2024-09-21T06:36:04.835379+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:16.139829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.029506+0000","last_change":"2024-09-21T06:34:32.637509+0000","last_active":"2024-09-21T06:35:05.029506+0000","last_peered":"2024-09-21T06:35:05.029506+0000","last_clean":"2024-09-21T06:35:05.029506+0000","last_became_active":"2024-09-21T06:34:32.637136+0000","last_became_peered":"2024-09-21T06:34:32.637136+0000","last_unstale":"2024-09-21T06:35:05.029506+0000","last_undegraded":"2024-09-21T06:35:05.029506+0000","last_fullsized":"2024-09-21T06:35:05.029506+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:48.088988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.834258+0000","last_change":"2024-09-21T06:35:05.114819+0000","last_active":"2024-09-21T06:36:04.834258+0000","last_peered":"2024-09-21T06:36:04.834258+0000","last_clean":"2024-09-21T06:36:04.834258+0000","last_became_active":"2024-09-21T06:35:05.114424+0000","last_became_peered":"2024-09-21T06:35:05.114424+0000","last_unstale":"2024-09-21T06:36:04.834258+0000","last_undegraded":"2024-09-21T06:36:04.834258+0000","last_fullsized":"2024-09-21T06:36:04.834258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:59.323643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.834338+0000","last_change":"2024-09-21T06:34:32.636103+0000","last_active":"2024-09-21T06:36:04.834338+0000","last_peered":"2024-09-21T06:36:04.834338+0000","last_clean":"2024-09-21T06:36:04.834338+0000","last_became_active":"2024-09-21T06:34:32.635728+0000","last_became_peered":"2024-09-21T06:34:32.635728+0000","last_unstale":"2024-09-21T06:36:04.834338+0000","last_undegraded":"2024-09-21T06:36:04.834338+0000","last_fullsized":"2024-09-21T06:36:04.834338+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:16.731402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.997848+0000","last_change":"2024-09-21T06:34:32.635654+0000","last_active":"2024-09-21T06:35:04.997848+0000","last_peered":"2024-09-21T06:35:04.997848+0000","last_clean":"2024-09-21T06:35:04.997848+0000","last_became_active":"2024-09-21T06:34:32.635265+0000","last_became_peered":"2024-09-21T06:34:32.635265+0000","last_unstale":"2024-09-21T06:35:04.997848+0000","last_undegraded":"2024-09-21T06:35:04.997848+0000","last_fullsized":"2024-09-21T06:35:04.997848+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:28.169625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000238+0000","last_change":"2024-09-21T06:34:32.644640+0000","last_active":"2024-09-21T06:35:05.000238+0000","last_peered":"2024-09-21T06:35:05.000238+0000","last_clean":"2024-09-21T06:35:05.000238+0000","last_became_active":"2024-09-21T06:34:32.644334+0000","last_became_peered":"2024-09-21T06:34:32.644334+0000","last_unstale":"2024-09-21T06:35:05.000238+0000","last_undegraded":"2024-09-21T06:35:05.000238+0000","last_fullsized":"2024-09-21T06:35:05.000238+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:25.572178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.000396+0000","last_change":"2024-09-21T06:34:32.643705+0000","last_active":"2024-09-21T06:35:05.000396+0000","last_peered":"2024-09-21T06:35:05.000396+0000","last_clean":"2024-09-21T06:35:05.000396+0000","last_became_active":"2024-09-21T06:34:32.642705+0000","last_became_peered":"2024-09-21T06:34:32.642705+0000","last_unstale":"2024-09-21T06:35:05.000396+0000","last_undegraded":"2024-09-21T06:35:05.000396+0000","last_fullsized":"2024-09-21T06:35:05.000396+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:22.542559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:05.001285+0000","last_change":"2024-09-21T06:34:32.646539+0000","last_active":"2024-09-21T06:35:05.001285+0000","last_peered":"2024-09-21T06:35:05.001285+0000","last_clean":"2024-09-21T06:35:05.001285+0000","last_became_active":"2024-09-21T06:34:32.645992+0000","last_became_peered":"2024-09-21T06:34:32.645992+0000","last_unstale":"2024-09-21T06:35:05.001285+0000","last_undegraded":"2024-09-21T06:35:05.001285+0000","last_fullsized":"2024-09-21T06:35:05.001285+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:38.856001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69583,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":129,"num_read_kb":135,"num_write":85,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":208749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71,"ondisk_log_size":71,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282016,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743456,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":466743,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939685,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1052672,"data_stored":1131518,"data_compressed":62455,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103210,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":471421,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299440,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":467347,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495668,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":1061789,"data_compressed":59457,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659194,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":901120,"data_stored":462647,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855423,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":1057293,"data_compressed":59457,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018948,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":466765,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215177,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":921600,"data_stored":472275,"data_compressed":26594,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":995328,"data_stored":540990,"data_compressed":26594,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":531432,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":905216,"data_stored":462737,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":68819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:36:07.803 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T06:36:07.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:07 smithi203 ceph-mon[33735]: pgmap v346: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:36:07.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:07 smithi203 ceph-mon[33735]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-21T06:36:08.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:07 smithi067 ceph-mon[31232]: pgmap v346: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:36:08.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:07 smithi067 ceph-mon[31232]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-21T06:36:08.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:07 smithi100 ceph-mon[33401]: pgmap v346: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:36:08.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:07 smithi100 ceph-mon[33401]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-21T06:36:08.067 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:36:08.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.598+0000 7fe4eeb50640 1 -- 172.21.15.67:0/574987495 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4e8102730 msgr2=0x7fe4e8110560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:08.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.598+0000 7fe4eeb50640 1 --2- 172.21.15.67:0/574987495 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4e8102730 0x7fe4e8110560 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0009900 tx=0x7fe4d002a410 comp rx=0 tx=0).stop 2024-09-21T06:36:08.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.598+0000 7fe4eeb50640 1 -- 172.21.15.67:0/574987495 shutdown_connections 2024-09-21T06:36:08.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.598+0000 7fe4eeb50640 1 --2- 172.21.15.67:0/574987495 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4e8102730 0x7fe4e8110560 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:08.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.598+0000 7fe4eeb50640 1 --2- 172.21.15.67:0/574987495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4e8101de0 0x7fe4e81021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:08.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.598+0000 7fe4eeb50640 1 --2- 172.21.15.67:0/574987495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4e81086f0 0x7fe4e8108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:08.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.598+0000 7fe4eeb50640 1 -- 172.21.15.67:0/574987495 >> 172.21.15.67:0/574987495 conn(0x7fe4e8077c60 msgr2=0x7fe4e80780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:08.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.598+0000 7fe4eeb50640 1 -- 172.21.15.67:0/574987495 shutdown_connections 2024-09-21T06:36:08.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.598+0000 7fe4eeb50640 1 -- 172.21.15.67:0/574987495 wait complete. 2024-09-21T06:36:08.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.599+0000 7fe4eeb50640 1 Processor -- start 2024-09-21T06:36:08.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.599+0000 7fe4eeb50640 1 -- start start 2024-09-21T06:36:08.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.599+0000 7fe4eeb50640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4e8101de0 0x7fe4e819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:08.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.599+0000 7fe4eeb50640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4e8102730 0x7fe4e819a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:08.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.599+0000 7fe4eeb50640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4e81086f0 0x7fe4e819fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:08.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.599+0000 7fe4eeb50640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e8113950 con 0x7fe4e81086f0 2024-09-21T06:36:08.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.599+0000 7fe4eeb50640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e81137d0 con 0x7fe4e8102730 2024-09-21T06:36:08.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.599+0000 7fe4eeb50640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e8113ad0 con 0x7fe4e8101de0 2024-09-21T06:36:08.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.600+0000 7fe4ed0ed640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4e81086f0 0x7fe4e819fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:08.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.600+0000 7fe4ed0ed640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4e81086f0 0x7fe4e819fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44602/0 (socket says 172.21.15.67:44602) 2024-09-21T06:36:08.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.600+0000 7fe4ed0ed640 1 -- 172.21.15.67:0/3424636992 learned_addr learned my addr 172.21.15.67:0/3424636992 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:08.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.600+0000 7fe4e7fff640 1 --2- 172.21.15.67:0/3424636992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4e8102730 0x7fe4e819a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:08.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.600+0000 7fe4ec8ec640 1 --2- 172.21.15.67:0/3424636992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4e8101de0 0x7fe4e819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:08.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.600+0000 7fe4ed0ed640 1 -- 172.21.15.67:0/3424636992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4e8101de0 msgr2=0x7fe4e819a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:08.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.600+0000 7fe4ed0ed640 1 --2- 172.21.15.67:0/3424636992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4e8101de0 0x7fe4e819a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:08.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.600+0000 7fe4ed0ed640 1 -- 172.21.15.67:0/3424636992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4e8102730 msgr2=0x7fe4e819a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:08.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.600+0000 7fe4ed0ed640 1 --2- 172.21.15.67:0/3424636992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4e8102730 0x7fe4e819a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:08.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.600+0000 7fe4ed0ed640 1 -- 172.21.15.67:0/3424636992 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e81a02b0 con 0x7fe4e81086f0 2024-09-21T06:36:08.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.600+0000 7fe4e7fff640 1 --2- 172.21.15.67:0/3424636992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4e8102730 0x7fe4e819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:36:08.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.600+0000 7fe4ec8ec640 1 --2- 172.21.15.67:0/3424636992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4e8101de0 0x7fe4e819a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:36:08.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.600+0000 7fe4ed0ed640 1 --2- 172.21.15.67:0/3424636992 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4e81086f0 0x7fe4e819fb80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0005ab0 tx=0x7fe4d00040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:08.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.600+0000 7fe4e5ffb640 1 -- 172.21.15.67:0/3424636992 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d003f030 con 0x7fe4e81086f0 2024-09-21T06:36:08.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.600+0000 7fe4e5ffb640 1 -- 172.21.15.67:0/3424636992 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4d0003a80 con 0x7fe4e81086f0 2024-09-21T06:36:08.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.600+0000 7fe4eeb50640 1 -- 172.21.15.67:0/3424636992 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4e81a0540 con 0x7fe4e81086f0 2024-09-21T06:36:08.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.600+0000 7fe4e5ffb640 1 -- 172.21.15.67:0/3424636992 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0035070 con 0x7fe4e81086f0 2024-09-21T06:36:08.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.601+0000 7fe4eeb50640 1 -- 172.21.15.67:0/3424636992 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4e81a6d00 con 0x7fe4e81086f0 2024-09-21T06:36:08.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.602+0000 7fe4eeb50640 1 -- 172.21.15.67:0/3424636992 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4e81021f0 con 0x7fe4e81086f0 2024-09-21T06:36:08.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.603+0000 7fe4e5ffb640 1 -- 172.21.15.67:0/3424636992 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fe4d0042030 con 0x7fe4e81086f0 2024-09-21T06:36:08.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.604+0000 7fe4e5ffb640 1 --2- 172.21.15.67:0/3424636992 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe4d40b65a0 0x7fe4d40b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:08.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.604+0000 7fe4ec8ec640 1 --2- 172.21.15.67:0/3424636992 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe4d40b65a0 0x7fe4d40b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:08.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.604+0000 7fe4e5ffb640 1 -- 172.21.15.67:0/3424636992 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fe4d01082e0 con 0x7fe4e81086f0 2024-09-21T06:36:08.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.605+0000 7fe4ec8ec640 1 --2- 172.21.15.67:0/3424636992 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe4d40b65a0 0x7fe4d40b8ab0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0005fd0 tx=0x7fe4e00094d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:08.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.607+0000 7fe4e5ffb640 1 -- 172.21.15.67:0/3424636992 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe4d00ce5e0 con 0x7fe4e81086f0 2024-09-21T06:36:08.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:08 smithi067 ceph-mon[31232]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:36:08.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:08 smithi067 ceph-mon[31232]: pgmap v347: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T06:36:08.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.801+0000 7fe4eeb50640 1 -- 172.21.15.67:0/3424636992 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe4e8063800 con 0x7fe4d40b65a0 2024-09-21T06:36:08.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.807+0000 7fe4e5ffb640 1 -- 172.21.15.67:0/3424636992 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340544 (secure 0 0 0) 0x7fe4e8063800 con 0x7fe4d40b65a0 2024-09-21T06:36:08.812 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:36:08.813 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-09-21T06:36:08.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.810+0000 7fe4eeb50640 1 -- 172.21.15.67:0/3424636992 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe4d40b65a0 msgr2=0x7fe4d40b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:08.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.810+0000 7fe4eeb50640 1 --2- 172.21.15.67:0/3424636992 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe4d40b65a0 0x7fe4d40b8ab0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0005fd0 tx=0x7fe4e00094d0 comp rx=0 tx=0).stop 2024-09-21T06:36:08.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.810+0000 7fe4eeb50640 1 -- 172.21.15.67:0/3424636992 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4e81086f0 msgr2=0x7fe4e819fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:08.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.810+0000 7fe4eeb50640 1 --2- 172.21.15.67:0/3424636992 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4e81086f0 0x7fe4e819fb80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0005ab0 tx=0x7fe4d00040e0 comp rx=0 tx=0).stop 2024-09-21T06:36:08.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.810+0000 7fe4eeb50640 1 -- 172.21.15.67:0/3424636992 shutdown_connections 2024-09-21T06:36:08.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.810+0000 7fe4eeb50640 1 --2- 172.21.15.67:0/3424636992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4e8102730 0x7fe4e819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:08.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.810+0000 7fe4eeb50640 1 --2- 172.21.15.67:0/3424636992 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe4d40b65a0 0x7fe4d40b8ab0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:08.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.810+0000 7fe4eeb50640 1 --2- 172.21.15.67:0/3424636992 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4e81086f0 0x7fe4e819fb80 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:08.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.810+0000 7fe4eeb50640 1 --2- 172.21.15.67:0/3424636992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4e8101de0 0x7fe4e819a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:08.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.810+0000 7fe4eeb50640 1 -- 172.21.15.67:0/3424636992 >> 172.21.15.67:0/3424636992 conn(0x7fe4e8077c60 msgr2=0x7fe4e8106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:08.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.810+0000 7fe4eeb50640 1 -- 172.21.15.67:0/3424636992 shutdown_connections 2024-09-21T06:36:08.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:08.810+0000 7fe4eeb50640 1 -- 172.21.15.67:0/3424636992 wait complete. 2024-09-21T06:36:08.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:08 smithi203 ceph-mon[33735]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:36:08.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:08 smithi203 ceph-mon[33735]: pgmap v347: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T06:36:09.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:08 smithi100 ceph-mon[33401]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:36:09.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:08 smithi100 ceph-mon[33401]: pgmap v347: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T06:36:09.481 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-09-21T06:36:07.868267+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660185,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":203,"num_read_kb":197,"num_write":210,"num_write_kb":2248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":165,"ondisk_log_size":165,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":420,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364660,"kb_used_data":11192,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577164,"statfs":{"total":1151940427776,"available":1151567015936,"internally_reserved":0,"allocated":11460608,"data_stored":7697353,"data_compressed":432735,"data_compressed_allocated":585728,"data_compressed_original":2670592,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":8,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002430"},"pg_stats":[{"pgid":"2.1f","version":"75'4","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621781+0000","last_change":"2024-09-21T06:34:30.633366+0000","last_active":"2024-09-21T06:36:05.621781+0000","last_peered":"2024-09-21T06:36:05.621781+0000","last_clean":"2024-09-21T06:36:05.621781+0000","last_became_active":"2024-09-21T06:34:30.632929+0000","last_became_peered":"2024-09-21T06:34:30.632929+0000","last_unstale":"2024-09-21T06:36:05.621781+0000","last_undegraded":"2024-09-21T06:36:05.621781+0000","last_fullsized":"2024-09-21T06:36:05.621781+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:47.722461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.834454+0000","last_change":"2024-09-21T06:34:32.632989+0000","last_active":"2024-09-21T06:36:04.834454+0000","last_peered":"2024-09-21T06:36:04.834454+0000","last_clean":"2024-09-21T06:36:04.834454+0000","last_became_active":"2024-09-21T06:34:32.632024+0000","last_became_peered":"2024-09-21T06:34:32.632024+0000","last_unstale":"2024-09-21T06:36:04.834454+0000","last_undegraded":"2024-09-21T06:36:04.834454+0000","last_fullsized":"2024-09-21T06:36:04.834454+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:13.669613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.834068+0000","last_change":"2024-09-21T06:35:05.274434+0000","last_active":"2024-09-21T06:36:04.834068+0000","last_peered":"2024-09-21T06:36:04.834068+0000","last_clean":"2024-09-21T06:36:04.834068+0000","last_became_active":"2024-09-21T06:35:05.273833+0000","last_became_peered":"2024-09-21T06:35:05.273833+0000","last_unstale":"2024-09-21T06:36:04.834068+0000","last_undegraded":"2024-09-21T06:36:04.834068+0000","last_fullsized":"2024-09-21T06:36:04.834068+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:48.847861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.834817+0000","last_change":"2024-09-21T06:34:30.946309+0000","last_active":"2024-09-21T06:36:04.834817+0000","last_peered":"2024-09-21T06:36:04.834817+0000","last_clean":"2024-09-21T06:36:04.834817+0000","last_became_active":"2024-09-21T06:34:30.945913+0000","last_became_peered":"2024-09-21T06:34:30.945913+0000","last_unstale":"2024-09-21T06:36:04.834817+0000","last_undegraded":"2024-09-21T06:36:04.834817+0000","last_fullsized":"2024-09-21T06:36:04.834817+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:21.676524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.629308+0000","last_change":"2024-09-21T06:34:32.637636+0000","last_active":"2024-09-21T06:36:05.629308+0000","last_peered":"2024-09-21T06:36:05.629308+0000","last_clean":"2024-09-21T06:36:05.629308+0000","last_became_active":"2024-09-21T06:34:32.636534+0000","last_became_peered":"2024-09-21T06:34:32.636534+0000","last_unstale":"2024-09-21T06:36:05.629308+0000","last_undegraded":"2024-09-21T06:36:05.629308+0000","last_fullsized":"2024-09-21T06:36:05.629308+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:05.802118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771498+0000","last_change":"2024-09-21T06:35:05.274419+0000","last_active":"2024-09-21T06:36:05.771498+0000","last_peered":"2024-09-21T06:36:05.771498+0000","last_clean":"2024-09-21T06:36:05.771498+0000","last_became_active":"2024-09-21T06:35:05.273865+0000","last_became_peered":"2024-09-21T06:35:05.273865+0000","last_unstale":"2024-09-21T06:36:05.771498+0000","last_undegraded":"2024-09-21T06:36:05.771498+0000","last_fullsized":"2024-09-21T06:36:05.771498+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:04.490355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.631709+0000","last_change":"2024-09-21T06:36:05.631098+0000","last_active":"2024-09-21T06:36:05.631709+0000","last_peered":"2024-09-21T06:36:05.631709+0000","last_clean":"2024-09-21T06:36:05.631709+0000","last_became_active":"2024-09-21T06:36:05.627521+0000","last_became_peered":"2024-09-21T06:36:05.627521+0000","last_unstale":"2024-09-21T06:36:05.631709+0000","last_undegraded":"2024-09-21T06:36:05.631709+0000","last_fullsized":"2024-09-21T06:36:05.631709+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:02.467758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834863+0000","last_change":"2024-09-21T06:34:32.650303+0000","last_active":"2024-09-21T06:36:05.834863+0000","last_peered":"2024-09-21T06:36:05.834863+0000","last_clean":"2024-09-21T06:36:05.834863+0000","last_became_active":"2024-09-21T06:34:32.647189+0000","last_became_peered":"2024-09-21T06:34:32.647189+0000","last_unstale":"2024-09-21T06:36:05.834863+0000","last_undegraded":"2024-09-21T06:36:05.834863+0000","last_fullsized":"2024-09-21T06:36:05.834863+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:14.889384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771714+0000","last_change":"2024-09-21T06:34:39.713139+0000","last_active":"2024-09-21T06:36:05.771714+0000","last_peered":"2024-09-21T06:36:05.771714+0000","last_clean":"2024-09-21T06:36:05.771714+0000","last_became_active":"2024-09-21T06:34:39.712379+0000","last_became_peered":"2024-09-21T06:34:39.712379+0000","last_unstale":"2024-09-21T06:36:05.771714+0000","last_undegraded":"2024-09-21T06:36:05.771714+0000","last_fullsized":"2024-09-21T06:36:05.771714+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:53.100529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.835790+0000","last_change":"2024-09-21T06:34:30.633850+0000","last_active":"2024-09-21T06:36:04.835790+0000","last_peered":"2024-09-21T06:36:04.835790+0000","last_clean":"2024-09-21T06:36:04.835790+0000","last_became_active":"2024-09-21T06:34:30.633468+0000","last_became_peered":"2024-09-21T06:34:30.633468+0000","last_unstale":"2024-09-21T06:36:04.835790+0000","last_undegraded":"2024-09-21T06:36:04.835790+0000","last_fullsized":"2024-09-21T06:36:04.835790+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:59.095803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834771+0000","last_change":"2024-09-21T06:35:05.005240+0000","last_active":"2024-09-21T06:36:05.834771+0000","last_peered":"2024-09-21T06:36:05.834771+0000","last_clean":"2024-09-21T06:36:05.834771+0000","last_became_active":"2024-09-21T06:35:05.004849+0000","last_became_peered":"2024-09-21T06:35:05.004849+0000","last_unstale":"2024-09-21T06:36:05.834771+0000","last_undegraded":"2024-09-21T06:36:05.834771+0000","last_fullsized":"2024-09-21T06:36:05.834771+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:28.849604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771556+0000","last_change":"2024-09-21T06:35:05.274437+0000","last_active":"2024-09-21T06:36:05.771556+0000","last_peered":"2024-09-21T06:36:05.771556+0000","last_clean":"2024-09-21T06:36:05.771556+0000","last_became_active":"2024-09-21T06:35:05.273865+0000","last_became_peered":"2024-09-21T06:35:05.273865+0000","last_unstale":"2024-09-21T06:36:05.771556+0000","last_undegraded":"2024-09-21T06:36:05.771556+0000","last_fullsized":"2024-09-21T06:36:05.771556+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:43.339504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622649+0000","last_change":"2024-09-21T06:34:32.645630+0000","last_active":"2024-09-21T06:36:05.622649+0000","last_peered":"2024-09-21T06:36:05.622649+0000","last_clean":"2024-09-21T06:36:05.622649+0000","last_became_active":"2024-09-21T06:34:32.645237+0000","last_became_peered":"2024-09-21T06:34:32.645237+0000","last_unstale":"2024-09-21T06:36:05.622649+0000","last_undegraded":"2024-09-21T06:36:05.622649+0000","last_fullsized":"2024-09-21T06:36:05.622649+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:25.064318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.633983+0000","last_change":"2024-09-21T06:36:05.633484+0000","last_active":"2024-09-21T06:36:05.633983+0000","last_peered":"2024-09-21T06:36:05.633983+0000","last_clean":"2024-09-21T06:36:05.633983+0000","last_became_active":"2024-09-21T06:36:05.627637+0000","last_became_peered":"2024-09-21T06:36:05.627637+0000","last_unstale":"2024-09-21T06:36:05.633983+0000","last_undegraded":"2024-09-21T06:36:05.633983+0000","last_fullsized":"2024-09-21T06:36:05.633983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:50:00.626538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771936+0000","last_change":"2024-09-21T06:34:39.714177+0000","last_active":"2024-09-21T06:36:05.771936+0000","last_peered":"2024-09-21T06:36:05.771936+0000","last_clean":"2024-09-21T06:36:05.771936+0000","last_became_active":"2024-09-21T06:34:39.713713+0000","last_became_peered":"2024-09-21T06:34:39.713713+0000","last_unstale":"2024-09-21T06:36:05.771936+0000","last_undegraded":"2024-09-21T06:36:05.771936+0000","last_fullsized":"2024-09-21T06:36:05.771936+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:07.042240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.834755+0000","last_change":"2024-09-21T06:34:32.636297+0000","last_active":"2024-09-21T06:36:04.834755+0000","last_peered":"2024-09-21T06:36:04.834755+0000","last_clean":"2024-09-21T06:36:04.834755+0000","last_became_active":"2024-09-21T06:34:32.635930+0000","last_became_peered":"2024-09-21T06:34:32.635930+0000","last_unstale":"2024-09-21T06:36:04.834755+0000","last_undegraded":"2024-09-21T06:36:04.834755+0000","last_fullsized":"2024-09-21T06:36:04.834755+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:43.091682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.834445+0000","last_change":"2024-09-21T06:34:30.631405+0000","last_active":"2024-09-21T06:36:04.834445+0000","last_peered":"2024-09-21T06:36:04.834445+0000","last_clean":"2024-09-21T06:36:04.834445+0000","last_became_active":"2024-09-21T06:34:30.630512+0000","last_became_peered":"2024-09-21T06:34:30.630512+0000","last_unstale":"2024-09-21T06:36:04.834445+0000","last_undegraded":"2024-09-21T06:36:04.834445+0000","last_fullsized":"2024-09-21T06:36:04.834445+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:43.412695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"83'1","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771897+0000","last_change":"2024-09-21T06:34:39.710573+0000","last_active":"2024-09-21T06:36:05.771897+0000","last_peered":"2024-09-21T06:36:05.771897+0000","last_clean":"2024-09-21T06:36:05.771897+0000","last_became_active":"2024-09-21T06:34:39.708969+0000","last_became_peered":"2024-09-21T06:34:39.708969+0000","last_unstale":"2024-09-21T06:36:05.771897+0000","last_undegraded":"2024-09-21T06:36:05.771897+0000","last_fullsized":"2024-09-21T06:36:05.771897+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:34.847587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624019+0000","last_change":"2024-09-21T06:34:32.641085+0000","last_active":"2024-09-21T06:36:05.624019+0000","last_peered":"2024-09-21T06:36:05.624019+0000","last_clean":"2024-09-21T06:36:05.624019+0000","last_became_active":"2024-09-21T06:34:32.640723+0000","last_became_peered":"2024-09-21T06:34:32.640723+0000","last_unstale":"2024-09-21T06:36:05.624019+0000","last_undegraded":"2024-09-21T06:36:05.624019+0000","last_fullsized":"2024-09-21T06:36:05.624019+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:21.410262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.834234+0000","last_change":"2024-09-21T06:34:30.626719+0000","last_active":"2024-09-21T06:36:04.834234+0000","last_peered":"2024-09-21T06:36:04.834234+0000","last_clean":"2024-09-21T06:36:04.834234+0000","last_became_active":"2024-09-21T06:34:30.626312+0000","last_became_peered":"2024-09-21T06:34:30.626312+0000","last_unstale":"2024-09-21T06:36:04.834234+0000","last_undegraded":"2024-09-21T06:36:04.834234+0000","last_fullsized":"2024-09-21T06:36:04.834234+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:53.938914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621617+0000","last_change":"2024-09-21T06:34:39.711172+0000","last_active":"2024-09-21T06:36:05.621617+0000","last_peered":"2024-09-21T06:36:05.621617+0000","last_clean":"2024-09-21T06:36:05.621617+0000","last_became_active":"2024-09-21T06:34:39.708424+0000","last_became_peered":"2024-09-21T06:34:39.708424+0000","last_unstale":"2024-09-21T06:36:05.621617+0000","last_undegraded":"2024-09-21T06:36:05.621617+0000","last_fullsized":"2024-09-21T06:36:05.621617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:43.918355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.833657+0000","last_change":"2024-09-21T06:34:32.641463+0000","last_active":"2024-09-21T06:36:04.833657+0000","last_peered":"2024-09-21T06:36:04.833657+0000","last_clean":"2024-09-21T06:36:04.833657+0000","last_became_active":"2024-09-21T06:34:32.641086+0000","last_became_peered":"2024-09-21T06:34:32.641086+0000","last_unstale":"2024-09-21T06:36:04.833657+0000","last_undegraded":"2024-09-21T06:36:04.833657+0000","last_fullsized":"2024-09-21T06:36:04.833657+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:58.991374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835670+0000","last_change":"2024-09-21T06:34:30.946698+0000","last_active":"2024-09-21T06:36:05.835670+0000","last_peered":"2024-09-21T06:36:05.835670+0000","last_clean":"2024-09-21T06:36:05.835670+0000","last_became_active":"2024-09-21T06:34:30.946325+0000","last_became_peered":"2024-09-21T06:34:30.946325+0000","last_unstale":"2024-09-21T06:36:05.835670+0000","last_undegraded":"2024-09-21T06:36:05.835670+0000","last_fullsized":"2024-09-21T06:36:05.835670+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:22.639265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.772211+0000","last_change":"2024-09-21T06:34:39.712429+0000","last_active":"2024-09-21T06:36:05.772211+0000","last_peered":"2024-09-21T06:36:05.772211+0000","last_clean":"2024-09-21T06:36:05.772211+0000","last_became_active":"2024-09-21T06:34:39.711823+0000","last_became_peered":"2024-09-21T06:34:39.711823+0000","last_unstale":"2024-09-21T06:36:05.772211+0000","last_undegraded":"2024-09-21T06:36:05.772211+0000","last_fullsized":"2024-09-21T06:36:05.772211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:03:49.513991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.629145+0000","last_change":"2024-09-21T06:35:05.274326+0000","last_active":"2024-09-21T06:36:05.629145+0000","last_peered":"2024-09-21T06:36:05.629145+0000","last_clean":"2024-09-21T06:36:05.629145+0000","last_became_active":"2024-09-21T06:35:05.273884+0000","last_became_peered":"2024-09-21T06:35:05.273884+0000","last_unstale":"2024-09-21T06:36:05.629145+0000","last_undegraded":"2024-09-21T06:36:05.629145+0000","last_fullsized":"2024-09-21T06:36:05.629145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:03.219765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.772344+0000","last_change":"2024-09-21T06:34:30.676794+0000","last_active":"2024-09-21T06:36:05.772344+0000","last_peered":"2024-09-21T06:36:05.772344+0000","last_clean":"2024-09-21T06:36:05.772344+0000","last_became_active":"2024-09-21T06:34:30.676400+0000","last_became_peered":"2024-09-21T06:34:30.676400+0000","last_unstale":"2024-09-21T06:36:05.772344+0000","last_undegraded":"2024-09-21T06:36:05.772344+0000","last_fullsized":"2024-09-21T06:36:05.772344+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:23.681363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.835574+0000","last_change":"2024-09-21T06:34:39.708980+0000","last_active":"2024-09-21T06:36:04.835574+0000","last_peered":"2024-09-21T06:36:04.835574+0000","last_clean":"2024-09-21T06:36:04.835574+0000","last_became_active":"2024-09-21T06:34:39.707748+0000","last_became_peered":"2024-09-21T06:34:39.707748+0000","last_unstale":"2024-09-21T06:36:04.835574+0000","last_undegraded":"2024-09-21T06:36:04.835574+0000","last_fullsized":"2024-09-21T06:36:04.835574+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:16.812226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.835284+0000","last_change":"2024-09-21T06:34:32.638337+0000","last_active":"2024-09-21T06:36:04.835284+0000","last_peered":"2024-09-21T06:36:04.835284+0000","last_clean":"2024-09-21T06:36:04.835284+0000","last_became_active":"2024-09-21T06:34:32.638013+0000","last_became_peered":"2024-09-21T06:34:32.638013+0000","last_unstale":"2024-09-21T06:36:04.835284+0000","last_undegraded":"2024-09-21T06:36:04.835284+0000","last_fullsized":"2024-09-21T06:36:04.835284+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:26.321190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"83'9","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621666+0000","last_change":"2024-09-21T06:34:30.631086+0000","last_active":"2024-09-21T06:36:05.621666+0000","last_peered":"2024-09-21T06:36:05.621666+0000","last_clean":"2024-09-21T06:36:05.621666+0000","last_became_active":"2024-09-21T06:34:30.630706+0000","last_became_peered":"2024-09-21T06:34:30.630706+0000","last_unstale":"2024-09-21T06:36:05.621666+0000","last_undegraded":"2024-09-21T06:36:05.621666+0000","last_fullsized":"2024-09-21T06:36:05.621666+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:30.260359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.833830+0000","last_change":"2024-09-21T06:34:39.709097+0000","last_active":"2024-09-21T06:36:04.833830+0000","last_peered":"2024-09-21T06:36:04.833830+0000","last_clean":"2024-09-21T06:36:04.833830+0000","last_became_active":"2024-09-21T06:34:39.708609+0000","last_became_peered":"2024-09-21T06:34:39.708609+0000","last_unstale":"2024-09-21T06:36:04.833830+0000","last_undegraded":"2024-09-21T06:36:04.833830+0000","last_fullsized":"2024-09-21T06:36:04.833830+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:48.842174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995363+0000","last_change":"2024-09-21T06:34:32.636816+0000","last_active":"2024-09-21T06:35:04.995363+0000","last_peered":"2024-09-21T06:35:04.995363+0000","last_clean":"2024-09-21T06:35:04.995363+0000","last_became_active":"2024-09-21T06:34:32.636429+0000","last_became_peered":"2024-09-21T06:34:32.636429+0000","last_unstale":"2024-09-21T06:35:04.995363+0000","last_undegraded":"2024-09-21T06:35:04.995363+0000","last_fullsized":"2024-09-21T06:35:04.995363+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:49.010491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.836011+0000","last_change":"2024-09-21T06:34:30.947481+0000","last_active":"2024-09-21T06:36:04.836011+0000","last_peered":"2024-09-21T06:36:04.836011+0000","last_clean":"2024-09-21T06:36:04.836011+0000","last_became_active":"2024-09-21T06:34:30.947040+0000","last_became_peered":"2024-09-21T06:34:30.947040+0000","last_unstale":"2024-09-21T06:36:04.836011+0000","last_undegraded":"2024-09-21T06:36:04.836011+0000","last_fullsized":"2024-09-21T06:36:04.836011+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:52.813476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624899+0000","last_change":"2024-09-21T06:34:39.708674+0000","last_active":"2024-09-21T06:36:05.624899+0000","last_peered":"2024-09-21T06:36:05.624899+0000","last_clean":"2024-09-21T06:36:05.624899+0000","last_became_active":"2024-09-21T06:34:39.708176+0000","last_became_peered":"2024-09-21T06:34:39.708176+0000","last_unstale":"2024-09-21T06:36:05.624899+0000","last_undegraded":"2024-09-21T06:36:05.624899+0000","last_fullsized":"2024-09-21T06:36:05.624899+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:57.070792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.833860+0000","last_change":"2024-09-21T06:34:32.636381+0000","last_active":"2024-09-21T06:36:04.833860+0000","last_peered":"2024-09-21T06:36:04.833860+0000","last_clean":"2024-09-21T06:36:04.833860+0000","last_became_active":"2024-09-21T06:34:32.635937+0000","last_became_peered":"2024-09-21T06:34:32.635937+0000","last_unstale":"2024-09-21T06:36:04.833860+0000","last_undegraded":"2024-09-21T06:36:04.833860+0000","last_fullsized":"2024-09-21T06:36:04.833860+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:52.870676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621464+0000","last_change":"2024-09-21T06:34:30.947160+0000","last_active":"2024-09-21T06:36:05.621464+0000","last_peered":"2024-09-21T06:36:05.621464+0000","last_clean":"2024-09-21T06:36:05.621464+0000","last_became_active":"2024-09-21T06:34:30.946736+0000","last_became_peered":"2024-09-21T06:34:30.946736+0000","last_unstale":"2024-09-21T06:36:05.621464+0000","last_undegraded":"2024-09-21T06:36:05.621464+0000","last_fullsized":"2024-09-21T06:36:05.621464+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:47.425481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.833575+0000","last_change":"2024-09-21T06:35:05.006032+0000","last_active":"2024-09-21T06:36:04.833575+0000","last_peered":"2024-09-21T06:36:04.833575+0000","last_clean":"2024-09-21T06:36:04.833575+0000","last_became_active":"2024-09-21T06:35:05.005429+0000","last_became_peered":"2024-09-21T06:35:05.005429+0000","last_unstale":"2024-09-21T06:36:04.833575+0000","last_undegraded":"2024-09-21T06:36:04.833575+0000","last_fullsized":"2024-09-21T06:36:04.833575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:04.138097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.833789+0000","last_change":"2024-09-21T06:34:32.648256+0000","last_active":"2024-09-21T06:36:04.833789+0000","last_peered":"2024-09-21T06:36:04.833789+0000","last_clean":"2024-09-21T06:36:04.833789+0000","last_became_active":"2024-09-21T06:34:32.647881+0000","last_became_peered":"2024-09-21T06:34:32.647881+0000","last_unstale":"2024-09-21T06:36:04.833789+0000","last_undegraded":"2024-09-21T06:36:04.833789+0000","last_fullsized":"2024-09-21T06:36:04.833789+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:14.590834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622843+0000","last_change":"2024-09-21T06:34:30.629918+0000","last_active":"2024-09-21T06:36:05.622843+0000","last_peered":"2024-09-21T06:36:05.622843+0000","last_clean":"2024-09-21T06:36:05.622843+0000","last_became_active":"2024-09-21T06:34:30.628785+0000","last_became_peered":"2024-09-21T06:34:30.628785+0000","last_unstale":"2024-09-21T06:36:05.622843+0000","last_undegraded":"2024-09-21T06:36:05.622843+0000","last_fullsized":"2024-09-21T06:36:05.622843+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:05.333406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995102+0000","last_change":"2024-09-21T06:34:39.710073+0000","last_active":"2024-09-21T06:35:04.995102+0000","last_peered":"2024-09-21T06:35:04.995102+0000","last_clean":"2024-09-21T06:35:04.995102+0000","last_became_active":"2024-09-21T06:34:39.708986+0000","last_became_peered":"2024-09-21T06:34:39.708986+0000","last_unstale":"2024-09-21T06:35:04.995102+0000","last_undegraded":"2024-09-21T06:35:04.995102+0000","last_fullsized":"2024-09-21T06:35:04.995102+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:46.379902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834487+0000","last_change":"2024-09-21T06:34:32.644224+0000","last_active":"2024-09-21T06:36:05.834487+0000","last_peered":"2024-09-21T06:36:05.834487+0000","last_clean":"2024-09-21T06:36:05.834487+0000","last_became_active":"2024-09-21T06:34:32.643850+0000","last_became_peered":"2024-09-21T06:34:32.643850+0000","last_unstale":"2024-09-21T06:36:05.834487+0000","last_undegraded":"2024-09-21T06:36:05.834487+0000","last_fullsized":"2024-09-21T06:36:05.834487+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:24:33.188537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.994875+0000","last_change":"2024-09-21T06:34:30.946334+0000","last_active":"2024-09-21T06:35:04.994875+0000","last_peered":"2024-09-21T06:35:04.994875+0000","last_clean":"2024-09-21T06:35:04.994875+0000","last_became_active":"2024-09-21T06:34:30.945908+0000","last_became_peered":"2024-09-21T06:34:30.945908+0000","last_unstale":"2024-09-21T06:35:04.994875+0000","last_undegraded":"2024-09-21T06:35:04.994875+0000","last_fullsized":"2024-09-21T06:35:04.994875+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:46.746527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995193+0000","last_change":"2024-09-21T06:34:39.707909+0000","last_active":"2024-09-21T06:35:04.995193+0000","last_peered":"2024-09-21T06:35:04.995193+0000","last_clean":"2024-09-21T06:35:04.995193+0000","last_became_active":"2024-09-21T06:34:39.707492+0000","last_became_peered":"2024-09-21T06:34:39.707492+0000","last_unstale":"2024-09-21T06:35:04.995193+0000","last_undegraded":"2024-09-21T06:35:04.995193+0000","last_fullsized":"2024-09-21T06:35:04.995193+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:56.201212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.994774+0000","last_change":"2024-09-21T06:34:32.635870+0000","last_active":"2024-09-21T06:35:04.994774+0000","last_peered":"2024-09-21T06:35:04.994774+0000","last_clean":"2024-09-21T06:35:04.994774+0000","last_became_active":"2024-09-21T06:34:32.635252+0000","last_became_peered":"2024-09-21T06:34:32.635252+0000","last_unstale":"2024-09-21T06:35:04.994774+0000","last_undegraded":"2024-09-21T06:35:04.994774+0000","last_fullsized":"2024-09-21T06:35:04.994774+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:10.613033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622266+0000","last_change":"2024-09-21T06:34:30.634360+0000","last_active":"2024-09-21T06:36:05.622266+0000","last_peered":"2024-09-21T06:36:05.622266+0000","last_clean":"2024-09-21T06:36:05.622266+0000","last_became_active":"2024-09-21T06:34:30.633845+0000","last_became_peered":"2024-09-21T06:34:30.633845+0000","last_unstale":"2024-09-21T06:36:05.622266+0000","last_undegraded":"2024-09-21T06:36:05.622266+0000","last_fullsized":"2024-09-21T06:36:05.622266+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:34:53.388670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834837+0000","last_change":"2024-09-21T06:34:39.719620+0000","last_active":"2024-09-21T06:36:05.834837+0000","last_peered":"2024-09-21T06:36:05.834837+0000","last_clean":"2024-09-21T06:36:05.834837+0000","last_became_active":"2024-09-21T06:34:39.718786+0000","last_became_peered":"2024-09-21T06:34:39.718786+0000","last_unstale":"2024-09-21T06:36:05.834837+0000","last_undegraded":"2024-09-21T06:36:05.834837+0000","last_fullsized":"2024-09-21T06:36:05.834837+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:43.524932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.834429+0000","last_change":"2024-09-21T06:34:32.634970+0000","last_active":"2024-09-21T06:36:04.834429+0000","last_peered":"2024-09-21T06:36:04.834429+0000","last_clean":"2024-09-21T06:36:04.834429+0000","last_became_active":"2024-09-21T06:34:32.634611+0000","last_became_peered":"2024-09-21T06:34:32.634611+0000","last_unstale":"2024-09-21T06:36:04.834429+0000","last_undegraded":"2024-09-21T06:36:04.834429+0000","last_fullsized":"2024-09-21T06:36:04.834429+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:37.855849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.772108+0000","last_change":"2024-09-21T06:34:30.628123+0000","last_active":"2024-09-21T06:36:05.772108+0000","last_peered":"2024-09-21T06:36:05.772108+0000","last_clean":"2024-09-21T06:36:05.772108+0000","last_became_active":"2024-09-21T06:34:30.627498+0000","last_became_peered":"2024-09-21T06:34:30.627498+0000","last_unstale":"2024-09-21T06:36:05.772108+0000","last_undegraded":"2024-09-21T06:36:05.772108+0000","last_fullsized":"2024-09-21T06:36:05.772108+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:42.576441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":115,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622503+0000","last_change":"2024-09-21T06:33:16.856623+0000","last_active":"2024-09-21T06:36:05.622503+0000","last_peered":"2024-09-21T06:36:05.622503+0000","last_clean":"2024-09-21T06:36:05.622503+0000","last_became_active":"2024-09-21T06:33:16.850787+0000","last_became_peered":"2024-09-21T06:33:16.850787+0000","last_unstale":"2024-09-21T06:36:05.622503+0000","last_undegraded":"2024-09-21T06:36:05.622503+0000","last_fullsized":"2024-09-21T06:36:05.622503+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:50.283252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:50.283252+0000","last_clean_scrub_stamp":"2024-09-21T06:29:50.283252+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:29:20.711208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622480+0000","last_change":"2024-09-21T06:34:39.706472+0000","last_active":"2024-09-21T06:36:05.622480+0000","last_peered":"2024-09-21T06:36:05.622480+0000","last_clean":"2024-09-21T06:36:05.622480+0000","last_became_active":"2024-09-21T06:34:39.705724+0000","last_became_peered":"2024-09-21T06:34:39.705724+0000","last_unstale":"2024-09-21T06:36:05.622480+0000","last_undegraded":"2024-09-21T06:36:05.622480+0000","last_fullsized":"2024-09-21T06:36:05.622480+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:43.238704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622792+0000","last_change":"2024-09-21T06:34:32.645944+0000","last_active":"2024-09-21T06:36:05.622792+0000","last_peered":"2024-09-21T06:36:05.622792+0000","last_clean":"2024-09-21T06:36:05.622792+0000","last_became_active":"2024-09-21T06:34:32.645134+0000","last_became_peered":"2024-09-21T06:34:32.645134+0000","last_unstale":"2024-09-21T06:36:05.622792+0000","last_undegraded":"2024-09-21T06:36:05.622792+0000","last_fullsized":"2024-09-21T06:36:05.622792+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:45.254164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.835407+0000","last_change":"2024-09-21T06:34:30.946296+0000","last_active":"2024-09-21T06:36:04.835407+0000","last_peered":"2024-09-21T06:36:04.835407+0000","last_clean":"2024-09-21T06:36:04.835407+0000","last_became_active":"2024-09-21T06:34:30.945777+0000","last_became_peered":"2024-09-21T06:34:30.945777+0000","last_unstale":"2024-09-21T06:36:04.835407+0000","last_undegraded":"2024-09-21T06:36:04.835407+0000","last_fullsized":"2024-09-21T06:36:04.835407+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:32:25.988512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835077+0000","last_change":"2024-09-21T06:34:39.705876+0000","last_active":"2024-09-21T06:36:05.835077+0000","last_peered":"2024-09-21T06:36:05.835077+0000","last_clean":"2024-09-21T06:36:05.835077+0000","last_became_active":"2024-09-21T06:34:39.705132+0000","last_became_peered":"2024-09-21T06:34:39.705132+0000","last_unstale":"2024-09-21T06:36:05.835077+0000","last_undegraded":"2024-09-21T06:36:05.835077+0000","last_fullsized":"2024-09-21T06:36:05.835077+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:18.180244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622740+0000","last_change":"2024-09-21T06:34:32.638242+0000","last_active":"2024-09-21T06:36:05.622740+0000","last_peered":"2024-09-21T06:36:05.622740+0000","last_clean":"2024-09-21T06:36:05.622740+0000","last_became_active":"2024-09-21T06:34:32.637816+0000","last_became_peered":"2024-09-21T06:34:32.637816+0000","last_unstale":"2024-09-21T06:36:05.622740+0000","last_undegraded":"2024-09-21T06:36:05.622740+0000","last_fullsized":"2024-09-21T06:36:05.622740+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:40.926147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834922+0000","last_change":"2024-09-21T06:34:30.632173+0000","last_active":"2024-09-21T06:36:05.834922+0000","last_peered":"2024-09-21T06:36:05.834922+0000","last_clean":"2024-09-21T06:36:05.834922+0000","last_became_active":"2024-09-21T06:34:30.631638+0000","last_became_peered":"2024-09-21T06:34:30.631638+0000","last_unstale":"2024-09-21T06:36:05.834922+0000","last_undegraded":"2024-09-21T06:36:05.834922+0000","last_fullsized":"2024-09-21T06:36:05.834922+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:29.763272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.628935+0000","last_change":"2024-09-21T06:34:39.714620+0000","last_active":"2024-09-21T06:36:05.628935+0000","last_peered":"2024-09-21T06:36:05.628935+0000","last_clean":"2024-09-21T06:36:05.628935+0000","last_became_active":"2024-09-21T06:34:39.714199+0000","last_became_peered":"2024-09-21T06:34:39.714199+0000","last_unstale":"2024-09-21T06:36:05.628935+0000","last_undegraded":"2024-09-21T06:36:05.628935+0000","last_fullsized":"2024-09-21T06:36:05.628935+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:37.041249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622722+0000","last_change":"2024-09-21T06:34:32.645001+0000","last_active":"2024-09-21T06:36:05.622722+0000","last_peered":"2024-09-21T06:36:05.622722+0000","last_clean":"2024-09-21T06:36:05.622722+0000","last_became_active":"2024-09-21T06:34:32.644584+0000","last_became_peered":"2024-09-21T06:34:32.644584+0000","last_unstale":"2024-09-21T06:36:05.622722+0000","last_undegraded":"2024-09-21T06:36:05.622722+0000","last_fullsized":"2024-09-21T06:36:05.622722+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:47.098850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995420+0000","last_change":"2024-09-21T06:34:30.632593+0000","last_active":"2024-09-21T06:35:04.995420+0000","last_peered":"2024-09-21T06:35:04.995420+0000","last_clean":"2024-09-21T06:35:04.995420+0000","last_became_active":"2024-09-21T06:34:30.632170+0000","last_became_peered":"2024-09-21T06:34:30.632170+0000","last_unstale":"2024-09-21T06:35:04.995420+0000","last_undegraded":"2024-09-21T06:35:04.995420+0000","last_fullsized":"2024-09-21T06:35:04.995420+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:56.170655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834120+0000","last_change":"2024-09-21T06:34:39.710172+0000","last_active":"2024-09-21T06:36:05.834120+0000","last_peered":"2024-09-21T06:36:05.834120+0000","last_clean":"2024-09-21T06:36:05.834120+0000","last_became_active":"2024-09-21T06:34:39.708876+0000","last_became_peered":"2024-09-21T06:34:39.708876+0000","last_unstale":"2024-09-21T06:36:05.834120+0000","last_undegraded":"2024-09-21T06:36:05.834120+0000","last_fullsized":"2024-09-21T06:36:05.834120+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:32.371873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624520+0000","last_change":"2024-09-21T06:34:32.651056+0000","last_active":"2024-09-21T06:36:05.624520+0000","last_peered":"2024-09-21T06:36:05.624520+0000","last_clean":"2024-09-21T06:36:05.624520+0000","last_became_active":"2024-09-21T06:34:32.650676+0000","last_became_peered":"2024-09-21T06:34:32.650676+0000","last_unstale":"2024-09-21T06:36:05.624520+0000","last_undegraded":"2024-09-21T06:36:05.624520+0000","last_fullsized":"2024-09-21T06:36:05.624520+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:34.599654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.835778+0000","last_change":"2024-09-21T06:34:30.632307+0000","last_active":"2024-09-21T06:36:04.835778+0000","last_peered":"2024-09-21T06:36:04.835778+0000","last_clean":"2024-09-21T06:36:04.835778+0000","last_became_active":"2024-09-21T06:34:30.631920+0000","last_became_peered":"2024-09-21T06:34:30.631920+0000","last_unstale":"2024-09-21T06:36:04.835778+0000","last_undegraded":"2024-09-21T06:36:04.835778+0000","last_fullsized":"2024-09-21T06:36:04.835778+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:46.940898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834319+0000","last_change":"2024-09-21T06:34:39.715056+0000","last_active":"2024-09-21T06:36:05.834319+0000","last_peered":"2024-09-21T06:36:05.834319+0000","last_clean":"2024-09-21T06:36:05.834319+0000","last_became_active":"2024-09-21T06:34:39.714563+0000","last_became_peered":"2024-09-21T06:34:39.714563+0000","last_unstale":"2024-09-21T06:36:05.834319+0000","last_undegraded":"2024-09-21T06:36:05.834319+0000","last_fullsized":"2024-09-21T06:36:05.834319+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:48.567057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834369+0000","last_change":"2024-09-21T06:34:32.646934+0000","last_active":"2024-09-21T06:36:05.834369+0000","last_peered":"2024-09-21T06:36:05.834369+0000","last_clean":"2024-09-21T06:36:05.834369+0000","last_became_active":"2024-09-21T06:34:32.646559+0000","last_became_peered":"2024-09-21T06:34:32.646559+0000","last_unstale":"2024-09-21T06:36:05.834369+0000","last_undegraded":"2024-09-21T06:36:05.834369+0000","last_fullsized":"2024-09-21T06:36:05.834369+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:37.802590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":44,"reported_epoch":83,"state":"peering","last_fresh":"2024-09-21T06:35:05.004087+0000","last_change":"2024-09-21T06:36:04.834224+0000","last_active":"2024-09-21T06:36:04.834224+0000","last_peered":"2024-09-21T06:35:05.004087+0000","last_clean":"2024-09-21T06:35:05.004087+0000","last_became_active":"2024-09-21T06:34:30.946327+0000","last_became_peered":"2024-09-21T06:34:30.946327+0000","last_unstale":"2024-09-21T06:35:05.004087+0000","last_undegraded":"2024-09-21T06:35:05.004087+0000","last_fullsized":"2024-09-21T06:35:05.004087+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1,2],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995322+0000","last_change":"2024-09-21T06:34:39.713398+0000","last_active":"2024-09-21T06:35:04.995322+0000","last_peered":"2024-09-21T06:35:04.995322+0000","last_clean":"2024-09-21T06:35:04.995322+0000","last_became_active":"2024-09-21T06:34:39.712954+0000","last_became_peered":"2024-09-21T06:34:39.712954+0000","last_unstale":"2024-09-21T06:35:04.995322+0000","last_undegraded":"2024-09-21T06:35:04.995322+0000","last_fullsized":"2024-09-21T06:35:04.995322+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:58.881014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622265+0000","last_change":"2024-09-21T06:34:32.640879+0000","last_active":"2024-09-21T06:36:05.622265+0000","last_peered":"2024-09-21T06:36:05.622265+0000","last_clean":"2024-09-21T06:36:05.622265+0000","last_became_active":"2024-09-21T06:34:32.640526+0000","last_became_peered":"2024-09-21T06:34:32.640526+0000","last_unstale":"2024-09-21T06:36:05.622265+0000","last_undegraded":"2024-09-21T06:36:05.622265+0000","last_fullsized":"2024-09-21T06:36:05.622265+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:12.327195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771895+0000","last_change":"2024-09-21T06:34:30.631449+0000","last_active":"2024-09-21T06:36:05.771895+0000","last_peered":"2024-09-21T06:36:05.771895+0000","last_clean":"2024-09-21T06:36:05.771895+0000","last_became_active":"2024-09-21T06:34:30.631081+0000","last_became_peered":"2024-09-21T06:34:30.631081+0000","last_unstale":"2024-09-21T06:36:05.771895+0000","last_undegraded":"2024-09-21T06:36:05.771895+0000","last_fullsized":"2024-09-21T06:36:05.771895+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:09.202696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621660+0000","last_change":"2024-09-21T06:34:39.707379+0000","last_active":"2024-09-21T06:36:05.621660+0000","last_peered":"2024-09-21T06:36:05.621660+0000","last_clean":"2024-09-21T06:36:05.621660+0000","last_became_active":"2024-09-21T06:34:39.706979+0000","last_became_peered":"2024-09-21T06:34:39.706979+0000","last_unstale":"2024-09-21T06:36:05.621660+0000","last_undegraded":"2024-09-21T06:36:05.621660+0000","last_fullsized":"2024-09-21T06:36:05.621660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:24.394233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622354+0000","last_change":"2024-09-21T06:34:32.649458+0000","last_active":"2024-09-21T06:36:05.622354+0000","last_peered":"2024-09-21T06:36:05.622354+0000","last_clean":"2024-09-21T06:36:05.622354+0000","last_became_active":"2024-09-21T06:34:32.649051+0000","last_became_peered":"2024-09-21T06:34:32.649051+0000","last_unstale":"2024-09-21T06:36:05.622354+0000","last_undegraded":"2024-09-21T06:36:05.622354+0000","last_fullsized":"2024-09-21T06:36:05.622354+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:12.631446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834455+0000","last_change":"2024-09-21T06:34:30.634592+0000","last_active":"2024-09-21T06:36:05.834455+0000","last_peered":"2024-09-21T06:36:05.834455+0000","last_clean":"2024-09-21T06:36:05.834455+0000","last_became_active":"2024-09-21T06:34:30.634251+0000","last_became_peered":"2024-09-21T06:34:30.634251+0000","last_unstale":"2024-09-21T06:36:05.834455+0000","last_undegraded":"2024-09-21T06:36:05.834455+0000","last_fullsized":"2024-09-21T06:36:05.834455+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:32.817467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624233+0000","last_change":"2024-09-21T06:34:39.719416+0000","last_active":"2024-09-21T06:36:05.624233+0000","last_peered":"2024-09-21T06:36:05.624233+0000","last_clean":"2024-09-21T06:36:05.624233+0000","last_became_active":"2024-09-21T06:34:39.718910+0000","last_became_peered":"2024-09-21T06:34:39.718910+0000","last_unstale":"2024-09-21T06:36:05.624233+0000","last_undegraded":"2024-09-21T06:36:05.624233+0000","last_fullsized":"2024-09-21T06:36:05.624233+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:41.413250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.994958+0000","last_change":"2024-09-21T06:34:32.640727+0000","last_active":"2024-09-21T06:35:04.994958+0000","last_peered":"2024-09-21T06:35:04.994958+0000","last_clean":"2024-09-21T06:35:04.994958+0000","last_became_active":"2024-09-21T06:34:32.640296+0000","last_became_peered":"2024-09-21T06:34:32.640296+0000","last_unstale":"2024-09-21T06:35:04.994958+0000","last_undegraded":"2024-09-21T06:35:04.994958+0000","last_fullsized":"2024-09-21T06:35:04.994958+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:35.124059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.834648+0000","last_change":"2024-09-21T06:34:30.633332+0000","last_active":"2024-09-21T06:36:04.834648+0000","last_peered":"2024-09-21T06:36:04.834648+0000","last_clean":"2024-09-21T06:36:04.834648+0000","last_became_active":"2024-09-21T06:34:30.632641+0000","last_became_peered":"2024-09-21T06:34:30.632641+0000","last_unstale":"2024-09-21T06:36:04.834648+0000","last_undegraded":"2024-09-21T06:36:04.834648+0000","last_fullsized":"2024-09-21T06:36:04.834648+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:30:23.930313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834659+0000","last_change":"2024-09-21T06:34:39.712504+0000","last_active":"2024-09-21T06:36:05.834659+0000","last_peered":"2024-09-21T06:36:05.834659+0000","last_clean":"2024-09-21T06:36:05.834659+0000","last_became_active":"2024-09-21T06:34:39.711638+0000","last_became_peered":"2024-09-21T06:34:39.711638+0000","last_unstale":"2024-09-21T06:36:05.834659+0000","last_undegraded":"2024-09-21T06:36:05.834659+0000","last_fullsized":"2024-09-21T06:36:05.834659+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:26:09.003367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.835429+0000","last_change":"2024-09-21T06:34:32.639582+0000","last_active":"2024-09-21T06:36:04.835429+0000","last_peered":"2024-09-21T06:36:04.835429+0000","last_clean":"2024-09-21T06:36:04.835429+0000","last_became_active":"2024-09-21T06:34:32.639023+0000","last_became_peered":"2024-09-21T06:34:32.639023+0000","last_unstale":"2024-09-21T06:36:04.835429+0000","last_undegraded":"2024-09-21T06:36:04.835429+0000","last_fullsized":"2024-09-21T06:36:04.835429+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:26.773389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"83'30","reported_seq":190,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.127841+0000","last_change":"2024-09-21T06:34:30.626280+0000","last_active":"2024-09-21T06:36:05.127841+0000","last_peered":"2024-09-21T06:36:05.127841+0000","last_clean":"2024-09-21T06:36:05.127841+0000","last_became_active":"2024-09-21T06:34:30.625662+0000","last_became_peered":"2024-09-21T06:34:30.625662+0000","last_unstale":"2024-09-21T06:36:05.127841+0000","last_undegraded":"2024-09-21T06:36:05.127841+0000","last_fullsized":"2024-09-21T06:36:05.127841+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:10.541881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68785,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":123,"num_read_kb":129,"num_write":30,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621183+0000","last_change":"2024-09-21T06:34:39.708999+0000","last_active":"2024-09-21T06:36:05.621183+0000","last_peered":"2024-09-21T06:36:05.621183+0000","last_clean":"2024-09-21T06:36:05.621183+0000","last_became_active":"2024-09-21T06:34:39.708580+0000","last_became_peered":"2024-09-21T06:34:39.708580+0000","last_unstale":"2024-09-21T06:36:05.621183+0000","last_undegraded":"2024-09-21T06:36:05.621183+0000","last_fullsized":"2024-09-21T06:36:05.621183+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:07.648358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835047+0000","last_change":"2024-09-21T06:34:32.651157+0000","last_active":"2024-09-21T06:36:05.835047+0000","last_peered":"2024-09-21T06:36:05.835047+0000","last_clean":"2024-09-21T06:36:05.835047+0000","last_became_active":"2024-09-21T06:34:32.650804+0000","last_became_peered":"2024-09-21T06:34:32.650804+0000","last_unstale":"2024-09-21T06:36:05.835047+0000","last_undegraded":"2024-09-21T06:36:05.835047+0000","last_fullsized":"2024-09-21T06:36:05.835047+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:49.758496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834880+0000","last_change":"2024-09-21T06:34:30.947142+0000","last_active":"2024-09-21T06:36:05.834880+0000","last_peered":"2024-09-21T06:36:05.834880+0000","last_clean":"2024-09-21T06:36:05.834880+0000","last_became_active":"2024-09-21T06:34:30.946748+0000","last_became_peered":"2024-09-21T06:34:30.946748+0000","last_unstale":"2024-09-21T06:36:05.834880+0000","last_undegraded":"2024-09-21T06:36:05.834880+0000","last_fullsized":"2024-09-21T06:36:05.834880+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:39.222286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771885+0000","last_change":"2024-09-21T06:34:39.709266+0000","last_active":"2024-09-21T06:36:05.771885+0000","last_peered":"2024-09-21T06:36:05.771885+0000","last_clean":"2024-09-21T06:36:05.771885+0000","last_became_active":"2024-09-21T06:34:39.708753+0000","last_became_peered":"2024-09-21T06:34:39.708753+0000","last_unstale":"2024-09-21T06:36:05.771885+0000","last_undegraded":"2024-09-21T06:36:05.771885+0000","last_fullsized":"2024-09-21T06:36:05.771885+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:48.711681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.629321+0000","last_change":"2024-09-21T06:34:32.637976+0000","last_active":"2024-09-21T06:36:05.629321+0000","last_peered":"2024-09-21T06:36:05.629321+0000","last_clean":"2024-09-21T06:36:05.629321+0000","last_became_active":"2024-09-21T06:34:32.637267+0000","last_became_peered":"2024-09-21T06:34:32.637267+0000","last_unstale":"2024-09-21T06:36:05.629321+0000","last_undegraded":"2024-09-21T06:36:05.629321+0000","last_fullsized":"2024-09-21T06:36:05.629321+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:31.340583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.836482+0000","last_change":"2024-09-21T06:36:05.836382+0000","last_active":"2024-09-21T06:36:05.836482+0000","last_peered":"2024-09-21T06:36:05.836482+0000","last_clean":"2024-09-21T06:36:05.836482+0000","last_became_active":"2024-09-21T06:36:05.835918+0000","last_became_peered":"2024-09-21T06:36:05.835918+0000","last_unstale":"2024-09-21T06:36:05.836482+0000","last_undegraded":"2024-09-21T06:36:05.836482+0000","last_fullsized":"2024-09-21T06:36:05.836482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:47.346400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621023+0000","last_change":"2024-09-21T06:34:39.713831+0000","last_active":"2024-09-21T06:36:05.621023+0000","last_peered":"2024-09-21T06:36:05.621023+0000","last_clean":"2024-09-21T06:36:05.621023+0000","last_became_active":"2024-09-21T06:34:39.713403+0000","last_became_peered":"2024-09-21T06:34:39.713403+0000","last_unstale":"2024-09-21T06:36:05.621023+0000","last_undegraded":"2024-09-21T06:36:05.621023+0000","last_fullsized":"2024-09-21T06:36:05.621023+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:22.201063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834704+0000","last_change":"2024-09-21T06:34:32.646924+0000","last_active":"2024-09-21T06:36:05.834704+0000","last_peered":"2024-09-21T06:36:05.834704+0000","last_clean":"2024-09-21T06:36:05.834704+0000","last_became_active":"2024-09-21T06:34:32.646531+0000","last_became_peered":"2024-09-21T06:34:32.646531+0000","last_unstale":"2024-09-21T06:36:05.834704+0000","last_undegraded":"2024-09-21T06:36:05.834704+0000","last_fullsized":"2024-09-21T06:36:05.834704+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:48.550193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.833967+0000","last_change":"2024-09-21T06:34:30.631901+0000","last_active":"2024-09-21T06:36:04.833967+0000","last_peered":"2024-09-21T06:36:04.833967+0000","last_clean":"2024-09-21T06:36:04.833967+0000","last_became_active":"2024-09-21T06:34:30.631450+0000","last_became_peered":"2024-09-21T06:34:30.631450+0000","last_unstale":"2024-09-21T06:36:04.833967+0000","last_undegraded":"2024-09-21T06:36:04.833967+0000","last_fullsized":"2024-09-21T06:36:04.833967+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:20.709456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.833960+0000","last_change":"2024-09-21T06:34:39.711951+0000","last_active":"2024-09-21T06:36:04.833960+0000","last_peered":"2024-09-21T06:36:04.833960+0000","last_clean":"2024-09-21T06:36:04.833960+0000","last_became_active":"2024-09-21T06:34:39.710783+0000","last_became_peered":"2024-09-21T06:34:39.710783+0000","last_unstale":"2024-09-21T06:36:04.833960+0000","last_undegraded":"2024-09-21T06:36:04.833960+0000","last_fullsized":"2024-09-21T06:36:04.833960+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:27:58.501155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.629019+0000","last_change":"2024-09-21T06:34:39.709639+0000","last_active":"2024-09-21T06:36:05.629019+0000","last_peered":"2024-09-21T06:36:05.629019+0000","last_clean":"2024-09-21T06:36:05.629019+0000","last_became_active":"2024-09-21T06:34:39.709241+0000","last_became_peered":"2024-09-21T06:34:39.709241+0000","last_unstale":"2024-09-21T06:36:05.629019+0000","last_undegraded":"2024-09-21T06:36:05.629019+0000","last_fullsized":"2024-09-21T06:36:05.629019+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.357399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621131+0000","last_change":"2024-09-21T06:34:32.635365+0000","last_active":"2024-09-21T06:36:05.621131+0000","last_peered":"2024-09-21T06:36:05.621131+0000","last_clean":"2024-09-21T06:36:05.621131+0000","last_became_active":"2024-09-21T06:34:32.635012+0000","last_became_peered":"2024-09-21T06:34:32.635012+0000","last_unstale":"2024-09-21T06:36:05.621131+0000","last_undegraded":"2024-09-21T06:36:05.621131+0000","last_fullsized":"2024-09-21T06:36:05.621131+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:54.671074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834581+0000","last_change":"2024-09-21T06:34:30.633347+0000","last_active":"2024-09-21T06:36:05.834581+0000","last_peered":"2024-09-21T06:36:05.834581+0000","last_clean":"2024-09-21T06:36:05.834581+0000","last_became_active":"2024-09-21T06:34:30.632513+0000","last_became_peered":"2024-09-21T06:34:30.632513+0000","last_unstale":"2024-09-21T06:36:05.834581+0000","last_undegraded":"2024-09-21T06:36:05.834581+0000","last_fullsized":"2024-09-21T06:36:05.834581+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:12.439109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624773+0000","last_change":"2024-09-21T06:34:39.710204+0000","last_active":"2024-09-21T06:36:05.624773+0000","last_peered":"2024-09-21T06:36:05.624773+0000","last_clean":"2024-09-21T06:36:05.624773+0000","last_became_active":"2024-09-21T06:34:39.709744+0000","last_became_peered":"2024-09-21T06:34:39.709744+0000","last_unstale":"2024-09-21T06:36:05.624773+0000","last_undegraded":"2024-09-21T06:36:05.624773+0000","last_fullsized":"2024-09-21T06:36:05.624773+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:45.484088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.833992+0000","last_change":"2024-09-21T06:34:32.634170+0000","last_active":"2024-09-21T06:36:04.833992+0000","last_peered":"2024-09-21T06:36:04.833992+0000","last_clean":"2024-09-21T06:36:04.833992+0000","last_became_active":"2024-09-21T06:34:32.633831+0000","last_became_peered":"2024-09-21T06:34:32.633831+0000","last_unstale":"2024-09-21T06:36:04.833992+0000","last_undegraded":"2024-09-21T06:36:04.833992+0000","last_fullsized":"2024-09-21T06:36:04.833992+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:36.738279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.994732+0000","last_change":"2024-09-21T06:34:30.629422+0000","last_active":"2024-09-21T06:35:04.994732+0000","last_peered":"2024-09-21T06:35:04.994732+0000","last_clean":"2024-09-21T06:35:04.994732+0000","last_became_active":"2024-09-21T06:34:30.629025+0000","last_became_peered":"2024-09-21T06:34:30.629025+0000","last_unstale":"2024-09-21T06:35:04.994732+0000","last_undegraded":"2024-09-21T06:35:04.994732+0000","last_fullsized":"2024-09-21T06:35:04.994732+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:05.068816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.834375+0000","last_change":"2024-09-21T06:34:30.634562+0000","last_active":"2024-09-21T06:36:04.834375+0000","last_peered":"2024-09-21T06:36:04.834375+0000","last_clean":"2024-09-21T06:36:04.834375+0000","last_became_active":"2024-09-21T06:34:30.634170+0000","last_became_peered":"2024-09-21T06:34:30.634170+0000","last_unstale":"2024-09-21T06:36:04.834375+0000","last_undegraded":"2024-09-21T06:36:04.834375+0000","last_fullsized":"2024-09-21T06:36:04.834375+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:15.243103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834518+0000","last_change":"2024-09-21T06:34:32.645718+0000","last_active":"2024-09-21T06:36:05.834518+0000","last_peered":"2024-09-21T06:36:05.834518+0000","last_clean":"2024-09-21T06:36:05.834518+0000","last_became_active":"2024-09-21T06:34:32.645072+0000","last_became_peered":"2024-09-21T06:34:32.645072+0000","last_unstale":"2024-09-21T06:36:05.834518+0000","last_undegraded":"2024-09-21T06:36:05.834518+0000","last_fullsized":"2024-09-21T06:36:05.834518+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:54.109246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.834864+0000","last_change":"2024-09-21T06:34:39.716947+0000","last_active":"2024-09-21T06:36:04.834864+0000","last_peered":"2024-09-21T06:36:04.834864+0000","last_clean":"2024-09-21T06:36:04.834864+0000","last_became_active":"2024-09-21T06:34:39.716540+0000","last_became_peered":"2024-09-21T06:34:39.716540+0000","last_unstale":"2024-09-21T06:36:04.834864+0000","last_undegraded":"2024-09-21T06:36:04.834864+0000","last_fullsized":"2024-09-21T06:36:04.834864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:12.748444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834526+0000","last_change":"2024-09-21T06:34:30.636389+0000","last_active":"2024-09-21T06:36:05.834526+0000","last_peered":"2024-09-21T06:36:05.834526+0000","last_clean":"2024-09-21T06:36:05.834526+0000","last_became_active":"2024-09-21T06:34:30.636008+0000","last_became_peered":"2024-09-21T06:34:30.636008+0000","last_unstale":"2024-09-21T06:36:05.834526+0000","last_undegraded":"2024-09-21T06:36:05.834526+0000","last_fullsized":"2024-09-21T06:36:05.834526+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:45.476267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.833807+0000","last_change":"2024-09-21T06:34:32.635644+0000","last_active":"2024-09-21T06:36:04.833807+0000","last_peered":"2024-09-21T06:36:04.833807+0000","last_clean":"2024-09-21T06:36:04.833807+0000","last_became_active":"2024-09-21T06:34:32.635253+0000","last_became_peered":"2024-09-21T06:34:32.635253+0000","last_unstale":"2024-09-21T06:36:04.833807+0000","last_undegraded":"2024-09-21T06:36:04.833807+0000","last_fullsized":"2024-09-21T06:36:04.833807+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:34.713656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.835922+0000","last_change":"2024-09-21T06:34:39.710382+0000","last_active":"2024-09-21T06:36:04.835922+0000","last_peered":"2024-09-21T06:36:04.835922+0000","last_clean":"2024-09-21T06:36:04.835922+0000","last_became_active":"2024-09-21T06:34:39.709884+0000","last_became_peered":"2024-09-21T06:34:39.709884+0000","last_unstale":"2024-09-21T06:36:04.835922+0000","last_undegraded":"2024-09-21T06:36:04.835922+0000","last_fullsized":"2024-09-21T06:36:04.835922+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:01.602366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.623967+0000","last_change":"2024-09-21T06:34:32.640025+0000","last_active":"2024-09-21T06:36:05.623967+0000","last_peered":"2024-09-21T06:36:05.623967+0000","last_clean":"2024-09-21T06:36:05.623967+0000","last_became_active":"2024-09-21T06:34:32.639648+0000","last_became_peered":"2024-09-21T06:34:32.639648+0000","last_unstale":"2024-09-21T06:36:05.623967+0000","last_undegraded":"2024-09-21T06:36:05.623967+0000","last_fullsized":"2024-09-21T06:36:05.623967+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:05.833065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.836918+0000","last_change":"2024-09-21T06:35:05.115344+0000","last_active":"2024-09-21T06:36:04.836918+0000","last_peered":"2024-09-21T06:36:04.836918+0000","last_clean":"2024-09-21T06:36:04.836918+0000","last_became_active":"2024-09-21T06:35:05.114678+0000","last_became_peered":"2024-09-21T06:35:05.114678+0000","last_unstale":"2024-09-21T06:36:04.836918+0000","last_undegraded":"2024-09-21T06:36:04.836918+0000","last_fullsized":"2024-09-21T06:36:04.836918+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:24.593145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.834295+0000","last_change":"2024-09-21T06:35:05.115301+0000","last_active":"2024-09-21T06:36:04.834295+0000","last_peered":"2024-09-21T06:36:04.834295+0000","last_clean":"2024-09-21T06:36:04.834295+0000","last_became_active":"2024-09-21T06:35:05.114706+0000","last_became_peered":"2024-09-21T06:35:05.114706+0000","last_unstale":"2024-09-21T06:36:04.834295+0000","last_undegraded":"2024-09-21T06:36:04.834295+0000","last_fullsized":"2024-09-21T06:36:04.834295+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:09.513594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.628918+0000","last_change":"2024-09-21T06:34:32.634679+0000","last_active":"2024-09-21T06:36:05.628918+0000","last_peered":"2024-09-21T06:36:05.628918+0000","last_clean":"2024-09-21T06:36:05.628918+0000","last_became_active":"2024-09-21T06:34:32.634285+0000","last_became_peered":"2024-09-21T06:34:32.634285+0000","last_unstale":"2024-09-21T06:36:05.628918+0000","last_undegraded":"2024-09-21T06:36:05.628918+0000","last_fullsized":"2024-09-21T06:36:05.628918+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:15.185453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995303+0000","last_change":"2024-09-21T06:34:32.639037+0000","last_active":"2024-09-21T06:35:04.995303+0000","last_peered":"2024-09-21T06:35:04.995303+0000","last_clean":"2024-09-21T06:35:04.995303+0000","last_became_active":"2024-09-21T06:34:32.638689+0000","last_became_peered":"2024-09-21T06:34:32.638689+0000","last_unstale":"2024-09-21T06:35:04.995303+0000","last_undegraded":"2024-09-21T06:35:04.995303+0000","last_fullsized":"2024-09-21T06:35:04.995303+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:13.609191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.625323+0000","last_change":"2024-09-21T06:35:05.115051+0000","last_active":"2024-09-21T06:36:05.625323+0000","last_peered":"2024-09-21T06:36:05.625323+0000","last_clean":"2024-09-21T06:36:05.625323+0000","last_became_active":"2024-09-21T06:35:05.114602+0000","last_became_peered":"2024-09-21T06:35:05.114602+0000","last_unstale":"2024-09-21T06:36:05.625323+0000","last_undegraded":"2024-09-21T06:36:05.625323+0000","last_fullsized":"2024-09-21T06:36:05.625323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:48.778188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.835155+0000","last_change":"2024-09-21T06:34:32.637237+0000","last_active":"2024-09-21T06:36:04.835155+0000","last_peered":"2024-09-21T06:36:04.835155+0000","last_clean":"2024-09-21T06:36:04.835155+0000","last_became_active":"2024-09-21T06:34:32.634105+0000","last_became_peered":"2024-09-21T06:34:32.634105+0000","last_unstale":"2024-09-21T06:36:04.835155+0000","last_undegraded":"2024-09-21T06:36:04.835155+0000","last_fullsized":"2024-09-21T06:36:04.835155+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:52.530461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.835249+0000","last_change":"2024-09-21T06:34:32.624441+0000","last_active":"2024-09-21T06:36:04.835249+0000","last_peered":"2024-09-21T06:36:04.835249+0000","last_clean":"2024-09-21T06:36:04.835249+0000","last_became_active":"2024-09-21T06:34:32.623974+0000","last_became_peered":"2024-09-21T06:34:32.623974+0000","last_unstale":"2024-09-21T06:36:04.835249+0000","last_undegraded":"2024-09-21T06:36:04.835249+0000","last_fullsized":"2024-09-21T06:36:04.835249+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:12.378113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834314+0000","last_change":"2024-09-21T06:34:32.633879+0000","last_active":"2024-09-21T06:36:05.834314+0000","last_peered":"2024-09-21T06:36:05.834314+0000","last_clean":"2024-09-21T06:36:05.834314+0000","last_became_active":"2024-09-21T06:34:32.631791+0000","last_became_peered":"2024-09-21T06:34:32.631791+0000","last_unstale":"2024-09-21T06:36:05.834314+0000","last_undegraded":"2024-09-21T06:36:05.834314+0000","last_fullsized":"2024-09-21T06:36:05.834314+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:46.246892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.834118+0000","last_change":"2024-09-21T06:34:32.639974+0000","last_active":"2024-09-21T06:36:04.834118+0000","last_peered":"2024-09-21T06:36:04.834118+0000","last_clean":"2024-09-21T06:36:04.834118+0000","last_became_active":"2024-09-21T06:34:32.639631+0000","last_became_peered":"2024-09-21T06:34:32.639631+0000","last_unstale":"2024-09-21T06:36:04.834118+0000","last_undegraded":"2024-09-21T06:36:04.834118+0000","last_fullsized":"2024-09-21T06:36:04.834118+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:50.069378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835236+0000","last_change":"2024-09-21T06:34:32.645403+0000","last_active":"2024-09-21T06:36:05.835236+0000","last_peered":"2024-09-21T06:36:05.835236+0000","last_clean":"2024-09-21T06:36:05.835236+0000","last_became_active":"2024-09-21T06:34:32.644986+0000","last_became_peered":"2024-09-21T06:34:32.644986+0000","last_unstale":"2024-09-21T06:36:05.835236+0000","last_undegraded":"2024-09-21T06:36:05.835236+0000","last_fullsized":"2024-09-21T06:36:05.835236+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:43.640107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995183+0000","last_change":"2024-09-21T06:34:32.639933+0000","last_active":"2024-09-21T06:35:04.995183+0000","last_peered":"2024-09-21T06:35:04.995183+0000","last_clean":"2024-09-21T06:35:04.995183+0000","last_became_active":"2024-09-21T06:34:32.639438+0000","last_became_peered":"2024-09-21T06:34:32.639438+0000","last_unstale":"2024-09-21T06:35:04.995183+0000","last_undegraded":"2024-09-21T06:35:04.995183+0000","last_fullsized":"2024-09-21T06:35:04.995183+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:18.860219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624647+0000","last_change":"2024-09-21T06:34:32.648580+0000","last_active":"2024-09-21T06:36:05.624647+0000","last_peered":"2024-09-21T06:36:05.624647+0000","last_clean":"2024-09-21T06:36:05.624647+0000","last_became_active":"2024-09-21T06:34:32.648016+0000","last_became_peered":"2024-09-21T06:34:32.648016+0000","last_unstale":"2024-09-21T06:36:05.624647+0000","last_undegraded":"2024-09-21T06:36:05.624647+0000","last_fullsized":"2024-09-21T06:36:05.624647+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:52.858069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.836427+0000","last_change":"2024-09-21T06:34:32.633376+0000","last_active":"2024-09-21T06:36:04.836427+0000","last_peered":"2024-09-21T06:36:04.836427+0000","last_clean":"2024-09-21T06:36:04.836427+0000","last_became_active":"2024-09-21T06:34:32.631052+0000","last_became_peered":"2024-09-21T06:34:32.631052+0000","last_unstale":"2024-09-21T06:36:04.836427+0000","last_undegraded":"2024-09-21T06:36:04.836427+0000","last_fullsized":"2024-09-21T06:36:04.836427+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:46.229599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621638+0000","last_change":"2024-09-21T06:34:32.636966+0000","last_active":"2024-09-21T06:36:05.621638+0000","last_peered":"2024-09-21T06:36:05.621638+0000","last_clean":"2024-09-21T06:36:05.621638+0000","last_became_active":"2024-09-21T06:34:32.636565+0000","last_became_peered":"2024-09-21T06:34:32.636565+0000","last_unstale":"2024-09-21T06:36:05.621638+0000","last_undegraded":"2024-09-21T06:36:05.621638+0000","last_fullsized":"2024-09-21T06:36:05.621638+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:36.233082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.625201+0000","last_change":"2024-09-21T06:34:32.646895+0000","last_active":"2024-09-21T06:36:05.625201+0000","last_peered":"2024-09-21T06:36:05.625201+0000","last_clean":"2024-09-21T06:36:05.625201+0000","last_became_active":"2024-09-21T06:34:32.646505+0000","last_became_peered":"2024-09-21T06:34:32.646505+0000","last_unstale":"2024-09-21T06:36:05.625201+0000","last_undegraded":"2024-09-21T06:36:05.625201+0000","last_fullsized":"2024-09-21T06:36:05.625201+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:43.441337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835545+0000","last_change":"2024-09-21T06:34:32.645402+0000","last_active":"2024-09-21T06:36:05.835545+0000","last_peered":"2024-09-21T06:36:05.835545+0000","last_clean":"2024-09-21T06:36:05.835545+0000","last_became_active":"2024-09-21T06:34:32.645013+0000","last_became_peered":"2024-09-21T06:34:32.645013+0000","last_unstale":"2024-09-21T06:36:05.835545+0000","last_undegraded":"2024-09-21T06:36:05.835545+0000","last_fullsized":"2024-09-21T06:36:05.835545+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:33.475847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.628586+0000","last_change":"2024-09-21T06:34:32.643234+0000","last_active":"2024-09-21T06:36:05.628586+0000","last_peered":"2024-09-21T06:36:05.628586+0000","last_clean":"2024-09-21T06:36:05.628586+0000","last_became_active":"2024-09-21T06:34:32.642614+0000","last_became_peered":"2024-09-21T06:34:32.642614+0000","last_unstale":"2024-09-21T06:36:05.628586+0000","last_undegraded":"2024-09-21T06:36:05.628586+0000","last_fullsized":"2024-09-21T06:36:05.628586+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:50.903500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.628594+0000","last_change":"2024-09-21T06:34:32.641887+0000","last_active":"2024-09-21T06:36:05.628594+0000","last_peered":"2024-09-21T06:36:05.628594+0000","last_clean":"2024-09-21T06:36:05.628594+0000","last_became_active":"2024-09-21T06:34:32.641490+0000","last_became_peered":"2024-09-21T06:34:32.641490+0000","last_unstale":"2024-09-21T06:36:05.628594+0000","last_undegraded":"2024-09-21T06:36:05.628594+0000","last_fullsized":"2024-09-21T06:36:05.628594+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:34.550970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T06:35:04.995289+0000","last_change":"2024-09-21T06:34:32.641865+0000","last_active":"2024-09-21T06:35:04.995289+0000","last_peered":"2024-09-21T06:35:04.995289+0000","last_clean":"2024-09-21T06:35:04.995289+0000","last_became_active":"2024-09-21T06:34:32.641252+0000","last_became_peered":"2024-09-21T06:34:32.641252+0000","last_unstale":"2024-09-21T06:35:04.995289+0000","last_undegraded":"2024-09-21T06:35:04.995289+0000","last_fullsized":"2024-09-21T06:35:04.995289+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:44.459202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.772096+0000","last_change":"2024-09-21T06:34:32.641259+0000","last_active":"2024-09-21T06:36:05.772096+0000","last_peered":"2024-09-21T06:36:05.772096+0000","last_clean":"2024-09-21T06:36:05.772096+0000","last_became_active":"2024-09-21T06:34:32.640907+0000","last_became_peered":"2024-09-21T06:34:32.640907+0000","last_unstale":"2024-09-21T06:36:05.772096+0000","last_undegraded":"2024-09-21T06:36:05.772096+0000","last_fullsized":"2024-09-21T06:36:05.772096+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:59.966857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622720+0000","last_change":"2024-09-21T06:34:32.646095+0000","last_active":"2024-09-21T06:36:05.622720+0000","last_peered":"2024-09-21T06:36:05.622720+0000","last_clean":"2024-09-21T06:36:05.622720+0000","last_became_active":"2024-09-21T06:34:32.645604+0000","last_became_peered":"2024-09-21T06:34:32.645604+0000","last_unstale":"2024-09-21T06:36:05.622720+0000","last_undegraded":"2024-09-21T06:36:05.622720+0000","last_fullsized":"2024-09-21T06:36:05.622720+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:30.872428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.834243+0000","last_change":"2024-09-21T06:34:32.649303+0000","last_active":"2024-09-21T06:36:04.834243+0000","last_peered":"2024-09-21T06:36:04.834243+0000","last_clean":"2024-09-21T06:36:04.834243+0000","last_became_active":"2024-09-21T06:34:32.648965+0000","last_became_peered":"2024-09-21T06:34:32.648965+0000","last_unstale":"2024-09-21T06:36:04.834243+0000","last_undegraded":"2024-09-21T06:36:04.834243+0000","last_fullsized":"2024-09-21T06:36:04.834243+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:51.335949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834838+0000","last_change":"2024-09-21T06:34:32.637261+0000","last_active":"2024-09-21T06:36:05.834838+0000","last_peered":"2024-09-21T06:36:05.834838+0000","last_clean":"2024-09-21T06:36:05.834838+0000","last_became_active":"2024-09-21T06:34:32.636861+0000","last_became_peered":"2024-09-21T06:34:32.636861+0000","last_unstale":"2024-09-21T06:36:05.834838+0000","last_undegraded":"2024-09-21T06:36:05.834838+0000","last_fullsized":"2024-09-21T06:36:05.834838+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:55.271027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.835379+0000","last_change":"2024-09-21T06:34:32.635156+0000","last_active":"2024-09-21T06:36:04.835379+0000","last_peered":"2024-09-21T06:36:04.835379+0000","last_clean":"2024-09-21T06:36:04.835379+0000","last_became_active":"2024-09-21T06:34:32.634802+0000","last_became_peered":"2024-09-21T06:34:32.634802+0000","last_unstale":"2024-09-21T06:36:04.835379+0000","last_undegraded":"2024-09-21T06:36:04.835379+0000","last_fullsized":"2024-09-21T06:36:04.835379+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:16.139829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622299+0000","last_change":"2024-09-21T06:34:32.637509+0000","last_active":"2024-09-21T06:36:05.622299+0000","last_peered":"2024-09-21T06:36:05.622299+0000","last_clean":"2024-09-21T06:36:05.622299+0000","last_became_active":"2024-09-21T06:34:32.637136+0000","last_became_peered":"2024-09-21T06:34:32.637136+0000","last_unstale":"2024-09-21T06:36:05.622299+0000","last_undegraded":"2024-09-21T06:36:05.622299+0000","last_fullsized":"2024-09-21T06:36:05.622299+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:48.088988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.834258+0000","last_change":"2024-09-21T06:35:05.114819+0000","last_active":"2024-09-21T06:36:04.834258+0000","last_peered":"2024-09-21T06:36:04.834258+0000","last_clean":"2024-09-21T06:36:04.834258+0000","last_became_active":"2024-09-21T06:35:05.114424+0000","last_became_peered":"2024-09-21T06:35:05.114424+0000","last_unstale":"2024-09-21T06:36:04.834258+0000","last_undegraded":"2024-09-21T06:36:04.834258+0000","last_fullsized":"2024-09-21T06:36:04.834258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:59.323643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T06:36:04.834338+0000","last_change":"2024-09-21T06:34:32.636103+0000","last_active":"2024-09-21T06:36:04.834338+0000","last_peered":"2024-09-21T06:36:04.834338+0000","last_clean":"2024-09-21T06:36:04.834338+0000","last_became_active":"2024-09-21T06:34:32.635728+0000","last_became_peered":"2024-09-21T06:34:32.635728+0000","last_unstale":"2024-09-21T06:36:04.834338+0000","last_undegraded":"2024-09-21T06:36:04.834338+0000","last_fullsized":"2024-09-21T06:36:04.834338+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:16.731402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.772051+0000","last_change":"2024-09-21T06:34:32.635654+0000","last_active":"2024-09-21T06:36:05.772051+0000","last_peered":"2024-09-21T06:36:05.772051+0000","last_clean":"2024-09-21T06:36:05.772051+0000","last_became_active":"2024-09-21T06:34:32.635265+0000","last_became_peered":"2024-09-21T06:34:32.635265+0000","last_unstale":"2024-09-21T06:36:05.772051+0000","last_undegraded":"2024-09-21T06:36:05.772051+0000","last_fullsized":"2024-09-21T06:36:05.772051+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:28.169625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.628625+0000","last_change":"2024-09-21T06:34:32.644640+0000","last_active":"2024-09-21T06:36:05.628625+0000","last_peered":"2024-09-21T06:36:05.628625+0000","last_clean":"2024-09-21T06:36:05.628625+0000","last_became_active":"2024-09-21T06:34:32.644334+0000","last_became_peered":"2024-09-21T06:34:32.644334+0000","last_unstale":"2024-09-21T06:36:05.628625+0000","last_undegraded":"2024-09-21T06:36:05.628625+0000","last_fullsized":"2024-09-21T06:36:05.628625+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:25.572178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.629082+0000","last_change":"2024-09-21T06:34:32.643705+0000","last_active":"2024-09-21T06:36:05.629082+0000","last_peered":"2024-09-21T06:36:05.629082+0000","last_clean":"2024-09-21T06:36:05.629082+0000","last_became_active":"2024-09-21T06:34:32.642705+0000","last_became_peered":"2024-09-21T06:34:32.642705+0000","last_unstale":"2024-09-21T06:36:05.629082+0000","last_undegraded":"2024-09-21T06:36:05.629082+0000","last_fullsized":"2024-09-21T06:36:05.629082+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:22.542559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624987+0000","last_change":"2024-09-21T06:34:32.646539+0000","last_active":"2024-09-21T06:36:05.624987+0000","last_peered":"2024-09-21T06:36:05.624987+0000","last_clean":"2024-09-21T06:36:05.624987+0000","last_became_active":"2024-09-21T06:34:32.645992+0000","last_became_peered":"2024-09-21T06:34:32.645992+0000","last_unstale":"2024-09-21T06:36:05.624987+0000","last_undegraded":"2024-09-21T06:36:05.624987+0000","last_fullsized":"2024-09-21T06:36:05.624987+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:38.856001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69583,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":127,"num_read_kb":133,"num_write":85,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":208749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71,"ondisk_log_size":71,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282016,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743457,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":929792,"data_stored":485699,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939685,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1052672,"data_stored":1131518,"data_compressed":62455,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103211,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":480945,"data_compressed":26594,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299440,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":467347,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495669,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":987136,"data_stored":1080745,"data_compressed":65460,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659194,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":901120,"data_stored":462647,"data_compressed":23596,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855424,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":991232,"data_stored":1076249,"data_compressed":65460,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018949,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":476289,"data_compressed":26594,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215177,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":921600,"data_stored":472275,"data_compressed":26594,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":995328,"data_stored":540990,"data_compressed":26594,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":991232,"data_stored":540956,"data_compressed":26594,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":929792,"data_stored":481693,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":68819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:36:09.487 INFO:tasks.cephadm.ceph_manager.ceph:PG 2.10 is not active+clean 2024-09-21T06:36:09.487 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '2.10', 'version': "75'8", 'reported_seq': 44, 'reported_epoch': 83, 'state': 'peering', 'last_fresh': '2024-09-21T06:35:05.004087+0000', 'last_change': '2024-09-21T06:36:04.834224+0000', 'last_active': '2024-09-21T06:36:04.834224+0000', 'last_peered': '2024-09-21T06:35:05.004087+0000', 'last_clean': '2024-09-21T06:35:05.004087+0000', 'last_became_active': '2024-09-21T06:34:30.946327+0000', 'last_became_peered': '2024-09-21T06:34:30.946327+0000', 'last_unstale': '2024-09-21T06:35:05.004087+0000', 'last_undegraded': '2024-09-21T06:35:05.004087+0000', 'last_fullsized': '2024-09-21T06:35:05.004087+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 72, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T06:34:29.598926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T06:34:29.598926+0000', 'last_clean_scrub_stamp': '2024-09-21T06:34:29.598926+0000', 'objects_scrubbed': 0, 'log_size': 8, 'log_dups_size': 0, 'ondisk_log_size': 8, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 4, 'num_object_clones': 0, 'num_object_copies': 12, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 4, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [8, 0, 1], 'acting': [8, 0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0, 1, 2], 'up_primary': 8, 'acting_primary': 8, 'purged_snaps': []} 2024-09-21T06:36:09.488 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph status --format=json 2024-09-21T06:36:09.748 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:36:09.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:09 smithi067 ceph-mon[31232]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:36:09.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:09 smithi203 ceph-mon[33735]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:36:10.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:09 smithi100 ceph-mon[33401]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:36:10.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.213+0000 7efe5af4c640 1 -- 172.21.15.67:0/2466191204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe54102a60 msgr2=0x7efe54102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:10.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.213+0000 7efe5af4c640 1 --2- 172.21.15.67:0/2466191204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe54102a60 0x7efe54102e70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7efe48009900 tx=0x7efe4802a3f0 comp rx=0 tx=0).stop 2024-09-21T06:36:10.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.213+0000 7efe5af4c640 1 -- 172.21.15.67:0/2466191204 shutdown_connections 2024-09-21T06:36:10.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.213+0000 7efe5af4c640 1 --2- 172.21.15.67:0/2466191204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe54108cf0 0x7efe5410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:10.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.213+0000 7efe5af4c640 1 --2- 172.21.15.67:0/2466191204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe541083a0 0x7efe541087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:10.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.213+0000 7efe5af4c640 1 --2- 172.21.15.67:0/2466191204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe54102a60 0x7efe54102e70 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:10.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.213+0000 7efe5af4c640 1 -- 172.21.15.67:0/2466191204 >> 172.21.15.67:0/2466191204 conn(0x7efe54079730 msgr2=0x7efe54079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:10.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.213+0000 7efe5af4c640 1 -- 172.21.15.67:0/2466191204 shutdown_connections 2024-09-21T06:36:10.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.213+0000 7efe5af4c640 1 -- 172.21.15.67:0/2466191204 wait complete. 2024-09-21T06:36:10.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.214+0000 7efe5af4c640 1 Processor -- start 2024-09-21T06:36:10.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.214+0000 7efe5af4c640 1 -- start start 2024-09-21T06:36:10.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.214+0000 7efe5af4c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe54102a60 0x7efe5419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:10.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.214+0000 7efe5af4c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe541083a0 0x7efe5419a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:10.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.214+0000 7efe5af4c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe54108cf0 0x7efe5419fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:10.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.214+0000 7efe5af4c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efe54112810 con 0x7efe541083a0 2024-09-21T06:36:10.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.214+0000 7efe5af4c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efe54112690 con 0x7efe54102a60 2024-09-21T06:36:10.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.214+0000 7efe5af4c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7efe54112990 con 0x7efe54108cf0 2024-09-21T06:36:10.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.214+0000 7efe53fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe541083a0 0x7efe5419a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:10.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.214+0000 7efe58ce8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe54102a60 0x7efe5419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:10.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.214+0000 7efe53fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe541083a0 0x7efe5419a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44630/0 (socket says 172.21.15.67:44630) 2024-09-21T06:36:10.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.214+0000 7efe58ce8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe54102a60 0x7efe5419a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.67:36796/0 (socket says 172.21.15.67:36796) 2024-09-21T06:36:10.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.214+0000 7efe53fff640 1 -- 172.21.15.67:0/4254848818 learned_addr learned my addr 172.21.15.67:0/4254848818 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:10.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.214+0000 7efe594e9640 1 --2- 172.21.15.67:0/4254848818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe54108cf0 0x7efe5419fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:10.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.215+0000 7efe53fff640 1 -- 172.21.15.67:0/4254848818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe54108cf0 msgr2=0x7efe5419fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:10.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.215+0000 7efe53fff640 1 --2- 172.21.15.67:0/4254848818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe54108cf0 0x7efe5419fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:10.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.215+0000 7efe53fff640 1 -- 172.21.15.67:0/4254848818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe54102a60 msgr2=0x7efe5419a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:10.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.215+0000 7efe53fff640 1 --2- 172.21.15.67:0/4254848818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe54102a60 0x7efe5419a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:10.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.215+0000 7efe53fff640 1 -- 172.21.15.67:0/4254848818 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe541a01e0 con 0x7efe541083a0 2024-09-21T06:36:10.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.215+0000 7efe53fff640 1 --2- 172.21.15.67:0/4254848818 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe541083a0 0x7efe5419a8d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7efe4000c970 tx=0x7efe4000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:10.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.215+0000 7efe51ffb640 1 -- 172.21.15.67:0/4254848818 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe40014070 con 0x7efe541083a0 2024-09-21T06:36:10.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.215+0000 7efe5af4c640 1 -- 172.21.15.67:0/4254848818 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe541a04d0 con 0x7efe541083a0 2024-09-21T06:36:10.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.215+0000 7efe51ffb640 1 -- 172.21.15.67:0/4254848818 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe40004440 con 0x7efe541083a0 2024-09-21T06:36:10.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.215+0000 7efe51ffb640 1 -- 172.21.15.67:0/4254848818 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe40003d40 con 0x7efe541083a0 2024-09-21T06:36:10.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.215+0000 7efe5af4c640 1 -- 172.21.15.67:0/4254848818 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe541a6cf0 con 0x7efe541083a0 2024-09-21T06:36:10.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.216+0000 7efe5af4c640 1 -- 172.21.15.67:0/4254848818 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe54102e70 con 0x7efe541083a0 2024-09-21T06:36:10.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.218+0000 7efe51ffb640 1 -- 172.21.15.67:0/4254848818 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7efe40017070 con 0x7efe541083a0 2024-09-21T06:36:10.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.219+0000 7efe51ffb640 1 --2- 172.21.15.67:0/4254848818 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7efe440b65d0 0x7efe440b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:10.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.219+0000 7efe58ce8640 1 --2- 172.21.15.67:0/4254848818 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7efe440b65d0 0x7efe440b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:10.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.219+0000 7efe51ffb640 1 -- 172.21.15.67:0/4254848818 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7efe400e7a80 con 0x7efe541083a0 2024-09-21T06:36:10.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.219+0000 7efe58ce8640 1 --2- 172.21.15.67:0/4254848818 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7efe440b65d0 0x7efe440b8ae0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efe480057c0 tx=0x7efe480058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:10.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.222+0000 7efe51ffb640 1 -- 172.21.15.67:0/4254848818 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efe4001e030 con 0x7efe541083a0 2024-09-21T06:36:10.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.481+0000 7efe5af4c640 1 -- 172.21.15.67:0/4254848818 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7efe5419b460 con 0x7efe541083a0 2024-09-21T06:36:10.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.481+0000 7efe51ffb640 1 -- 172.21.15.67:0/4254848818 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1742 (secure 0 0 0) 0x7efe400adde0 con 0x7efe541083a0 2024-09-21T06:36:10.484 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:36:10.484 INFO:teuthology.orchestra.run.smithi067.stdout:{"fsid":"2a791768-77e2-11ef-baf5-efdc52797490","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 1 pg peering","count":1},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":496,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":85,"num_osds":12,"num_up_osds":12,"osd_up_since":1726900420,"num_in_osds":12,"osd_in_since":1726900397,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":128},{"state_name":"peering","count":1}],"num_pgs":129,"num_pools":4,"num_objects":26,"data_bytes":660185,"bytes_used":373411840,"bytes_avail":1151567015936,"bytes_total":1151940427776,"inactive_pgs_ratio":0.0077519379556179047,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"read_bytes_sec":1023,"read_op_per_sec":2,"write_op_per_sec":0},"fsmap":{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"h","status":"up:standby-replay","gid":24641},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24671}],"up:standby":10},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.67:8443/"}},"servicemap":{"epoch":29,"modified":"2024-09-21T06:35:37.854020+0000","services":{"mds":{"daemons":{"summary":"","h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T06:36:10.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.484+0000 7efe5af4c640 1 -- 172.21.15.67:0/4254848818 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7efe440b65d0 msgr2=0x7efe440b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:10.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.484+0000 7efe5af4c640 1 --2- 172.21.15.67:0/4254848818 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7efe440b65d0 0x7efe440b8ae0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efe480057c0 tx=0x7efe480058d0 comp rx=0 tx=0).stop 2024-09-21T06:36:10.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.484+0000 7efe5af4c640 1 -- 172.21.15.67:0/4254848818 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe541083a0 msgr2=0x7efe5419a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:10.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.484+0000 7efe5af4c640 1 --2- 172.21.15.67:0/4254848818 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe541083a0 0x7efe5419a8d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7efe4000c970 tx=0x7efe4000ce30 comp rx=0 tx=0).stop 2024-09-21T06:36:10.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.484+0000 7efe5af4c640 1 -- 172.21.15.67:0/4254848818 shutdown_connections 2024-09-21T06:36:10.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.484+0000 7efe5af4c640 1 --2- 172.21.15.67:0/4254848818 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7efe440b65d0 0x7efe440b8ae0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:10.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.484+0000 7efe5af4c640 1 --2- 172.21.15.67:0/4254848818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe54108cf0 0x7efe5419fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:10.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.484+0000 7efe5af4c640 1 --2- 172.21.15.67:0/4254848818 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe541083a0 0x7efe5419a8d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:10.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.484+0000 7efe5af4c640 1 --2- 172.21.15.67:0/4254848818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe54102a60 0x7efe5419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:10.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.484+0000 7efe5af4c640 1 -- 172.21.15.67:0/4254848818 >> 172.21.15.67:0/4254848818 conn(0x7efe54079730 msgr2=0x7efe54101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:10.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.484+0000 7efe5af4c640 1 -- 172.21.15.67:0/4254848818 shutdown_connections 2024-09-21T06:36:10.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:10.484+0000 7efe5af4c640 1 -- 172.21.15.67:0/4254848818 wait complete. 2024-09-21T06:36:10.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:10 smithi067 ceph-mon[31232]: pgmap v348: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T06:36:10.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:10 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/4254848818' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-21T06:36:10.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:10 smithi203 ceph-mon[33735]: pgmap v348: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T06:36:10.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:10 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/4254848818' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-21T06:36:11.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:10 smithi100 ceph-mon[33401]: pgmap v348: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T06:36:11.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:10 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/4254848818' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-21T06:36:11.211 INFO:tasks.cephadm.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-21T06:36:11.211 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T06:36:11.468 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:36:11.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.966+0000 7f8d27d68640 1 -- 172.21.15.67:0/4233468291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d20102a80 msgr2=0x7f8d20102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:11.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.966+0000 7f8d27d68640 1 --2- 172.21.15.67:0/4233468291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d20102a80 0x7f8d20102e90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009900 tx=0x7f8d1c02a410 comp rx=0 tx=0).stop 2024-09-21T06:36:11.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.966+0000 7f8d27d68640 1 -- 172.21.15.67:0/4233468291 shutdown_connections 2024-09-21T06:36:11.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.966+0000 7f8d27d68640 1 --2- 172.21.15.67:0/4233468291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d20108d10 0x7f8d2010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:11.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.966+0000 7f8d27d68640 1 --2- 172.21.15.67:0/4233468291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d201083c0 0x7f8d201087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:11.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.966+0000 7f8d27d68640 1 --2- 172.21.15.67:0/4233468291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d20102a80 0x7f8d20102e90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:11.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.966+0000 7f8d27d68640 1 -- 172.21.15.67:0/4233468291 >> 172.21.15.67:0/4233468291 conn(0x7f8d20079740 msgr2=0x7f8d20079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:11.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.966+0000 7f8d27d68640 1 -- 172.21.15.67:0/4233468291 shutdown_connections 2024-09-21T06:36:11.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.966+0000 7f8d27d68640 1 -- 172.21.15.67:0/4233468291 wait complete. 2024-09-21T06:36:11.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.967+0000 7f8d27d68640 1 Processor -- start 2024-09-21T06:36:11.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.967+0000 7f8d27d68640 1 -- start start 2024-09-21T06:36:11.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.967+0000 7f8d27d68640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d20102a80 0x7f8d2010ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:11.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.967+0000 7f8d27d68640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d201083c0 0x7f8d20110510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:11.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.967+0000 7f8d27d68640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d20108d10 0x7f8d20110a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:11.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.967+0000 7f8d27d68640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8d20113950 con 0x7f8d20102a80 2024-09-21T06:36:11.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.967+0000 7f8d27d68640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8d201137d0 con 0x7f8d201083c0 2024-09-21T06:36:11.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.967+0000 7f8d27d68640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8d20113ad0 con 0x7f8d20108d10 2024-09-21T06:36:11.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.967+0000 7f8d25b04640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d20102a80 0x7f8d2010ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:11.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.967+0000 7f8d25b04640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d20102a80 0x7f8d2010ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44646/0 (socket says 172.21.15.67:44646) 2024-09-21T06:36:11.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.967+0000 7f8d25b04640 1 -- 172.21.15.67:0/1528692985 learned_addr learned my addr 172.21.15.67:0/1528692985 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:11.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.967+0000 7f8d25303640 1 --2- 172.21.15.67:0/1528692985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d201083c0 0x7f8d20110510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:11.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.967+0000 7f8d26305640 1 --2- 172.21.15.67:0/1528692985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d20108d10 0x7f8d20110a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:11.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.968+0000 7f8d25b04640 1 -- 172.21.15.67:0/1528692985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d20108d10 msgr2=0x7f8d20110a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:11.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.968+0000 7f8d25b04640 1 --2- 172.21.15.67:0/1528692985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d20108d10 0x7f8d20110a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:11.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.968+0000 7f8d25b04640 1 -- 172.21.15.67:0/1528692985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d201083c0 msgr2=0x7f8d20110510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:11.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.968+0000 7f8d25b04640 1 --2- 172.21.15.67:0/1528692985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d201083c0 0x7f8d20110510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:11.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.968+0000 7f8d25b04640 1 -- 172.21.15.67:0/1528692985 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d201a86a0 con 0x7f8d20102a80 2024-09-21T06:36:11.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.968+0000 7f8d25b04640 1 --2- 172.21.15.67:0/1528692985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d20102a80 0x7f8d2010ffd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009230 tx=0x7f8d1c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:11.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.968+0000 7f8d16ffd640 1 -- 172.21.15.67:0/1528692985 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c03f030 con 0x7f8d20102a80 2024-09-21T06:36:11.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.968+0000 7f8d16ffd640 1 -- 172.21.15.67:0/1528692985 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8d1c003a80 con 0x7f8d20102a80 2024-09-21T06:36:11.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.968+0000 7f8d27d68640 1 -- 172.21.15.67:0/1528692985 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d201a8930 con 0x7f8d20102a80 2024-09-21T06:36:11.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.968+0000 7f8d16ffd640 1 -- 172.21.15.67:0/1528692985 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c035070 con 0x7f8d20102a80 2024-09-21T06:36:11.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.968+0000 7f8d27d68640 1 -- 172.21.15.67:0/1528692985 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d201a8e10 con 0x7f8d20102a80 2024-09-21T06:36:11.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.969+0000 7f8d27d68640 1 -- 172.21.15.67:0/1528692985 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d20103d00 con 0x7f8d20102a80 2024-09-21T06:36:11.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.971+0000 7f8d16ffd640 1 -- 172.21.15.67:0/1528692985 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f8d1c042030 con 0x7f8d20102a80 2024-09-21T06:36:11.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.972+0000 7f8d16ffd640 1 --2- 172.21.15.67:0/1528692985 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f8d0c0b65a0 0x7f8d0c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:11.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.972+0000 7f8d25303640 1 --2- 172.21.15.67:0/1528692985 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f8d0c0b65a0 0x7f8d0c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:11.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.972+0000 7f8d16ffd640 1 -- 172.21.15.67:0/1528692985 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f8d1c108390 con 0x7f8d20102a80 2024-09-21T06:36:11.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.973+0000 7f8d25303640 1 --2- 172.21.15.67:0/1528692985 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f8d0c0b65a0 0x7f8d0c0b8ab0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8d20079e80 tx=0x7f8d1000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:11.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:11.976+0000 7f8d16ffd640 1 -- 172.21.15.67:0/1528692985 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8d1c0d2ad0 con 0x7f8d20102a80 2024-09-21T06:36:12.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:12.163+0000 7f8d27d68640 1 -- 172.21.15.67:0/1528692985 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8d20063800 con 0x7f8d0c0b65a0 2024-09-21T06:36:12.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:11 smithi067 ceph-mon[31232]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-21T06:36:12.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:11 smithi067 ceph-mon[31232]: Cluster is now healthy 2024-09-21T06:36:12.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:12.168+0000 7f8d16ffd640 1 -- 172.21.15.67:0/1528692985 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340557 (secure 0 0 0) 0x7f8d20063800 con 0x7f8d0c0b65a0 2024-09-21T06:36:12.174 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:36:12.175 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-09-21T06:36:12.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:12.172+0000 7f8d27d68640 1 -- 172.21.15.67:0/1528692985 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f8d0c0b65a0 msgr2=0x7f8d0c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:12.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:12.172+0000 7f8d27d68640 1 --2- 172.21.15.67:0/1528692985 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f8d0c0b65a0 0x7f8d0c0b8ab0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8d20079e80 tx=0x7f8d1000b390 comp rx=0 tx=0).stop 2024-09-21T06:36:12.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:12.172+0000 7f8d27d68640 1 -- 172.21.15.67:0/1528692985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d20102a80 msgr2=0x7f8d2010ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:12.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:12.172+0000 7f8d27d68640 1 --2- 172.21.15.67:0/1528692985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d20102a80 0x7f8d2010ffd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009230 tx=0x7f8d1c004270 comp rx=0 tx=0).stop 2024-09-21T06:36:12.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:12.172+0000 7f8d27d68640 1 -- 172.21.15.67:0/1528692985 shutdown_connections 2024-09-21T06:36:12.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:12.172+0000 7f8d27d68640 1 --2- 172.21.15.67:0/1528692985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d201083c0 0x7f8d20110510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:12.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:12.172+0000 7f8d27d68640 1 --2- 172.21.15.67:0/1528692985 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f8d0c0b65a0 0x7f8d0c0b8ab0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:12.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:12.172+0000 7f8d27d68640 1 --2- 172.21.15.67:0/1528692985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d20108d10 0x7f8d20110a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:12.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:12.172+0000 7f8d27d68640 1 --2- 172.21.15.67:0/1528692985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d20102a80 0x7f8d2010ffd0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:12.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:12.172+0000 7f8d27d68640 1 -- 172.21.15.67:0/1528692985 >> 172.21.15.67:0/1528692985 conn(0x7f8d20079740 msgr2=0x7f8d20101670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:12.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:12.172+0000 7f8d27d68640 1 -- 172.21.15.67:0/1528692985 shutdown_connections 2024-09-21T06:36:12.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:12.173+0000 7f8d27d68640 1 -- 172.21.15.67:0/1528692985 wait complete. 2024-09-21T06:36:12.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:11 smithi203 ceph-mon[33735]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-21T06:36:12.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:11 smithi203 ceph-mon[33735]: Cluster is now healthy 2024-09-21T06:36:12.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:11 smithi100 ceph-mon[33401]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-21T06:36:12.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:11 smithi100 ceph-mon[33401]: Cluster is now healthy 2024-09-21T06:36:12.860 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-09-21T06:36:11.869572+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660185,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":213,"num_read_kb":202,"num_write":210,"num_write_kb":2248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":165,"ondisk_log_size":165,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364804,"kb_used_data":11312,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577020,"statfs":{"total":1151940427776,"available":1151566868480,"internally_reserved":0,"allocated":11583488,"data_stored":7791857,"data_compressed":462771,"data_compressed_allocated":626688,"data_compressed_original":2752512,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":8,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002256"},"pg_stats":[{"pgid":"2.1f","version":"75'4","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621781+0000","last_change":"2024-09-21T06:34:30.633366+0000","last_active":"2024-09-21T06:36:05.621781+0000","last_peered":"2024-09-21T06:36:05.621781+0000","last_clean":"2024-09-21T06:36:05.621781+0000","last_became_active":"2024-09-21T06:34:30.632929+0000","last_became_peered":"2024-09-21T06:34:30.632929+0000","last_unstale":"2024-09-21T06:36:05.621781+0000","last_undegraded":"2024-09-21T06:36:05.621781+0000","last_fullsized":"2024-09-21T06:36:05.621781+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:47.722461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834970+0000","last_change":"2024-09-21T06:34:32.632989+0000","last_active":"2024-09-21T06:36:05.834970+0000","last_peered":"2024-09-21T06:36:05.834970+0000","last_clean":"2024-09-21T06:36:05.834970+0000","last_became_active":"2024-09-21T06:34:32.632024+0000","last_became_peered":"2024-09-21T06:34:32.632024+0000","last_unstale":"2024-09-21T06:36:05.834970+0000","last_undegraded":"2024-09-21T06:36:05.834970+0000","last_fullsized":"2024-09-21T06:36:05.834970+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:13.669613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621767+0000","last_change":"2024-09-21T06:35:05.274434+0000","last_active":"2024-09-21T06:36:05.621767+0000","last_peered":"2024-09-21T06:36:05.621767+0000","last_clean":"2024-09-21T06:36:05.621767+0000","last_became_active":"2024-09-21T06:35:05.273833+0000","last_became_peered":"2024-09-21T06:35:05.273833+0000","last_unstale":"2024-09-21T06:36:05.621767+0000","last_undegraded":"2024-09-21T06:36:05.621767+0000","last_fullsized":"2024-09-21T06:36:05.621767+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:48.847861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835182+0000","last_change":"2024-09-21T06:34:30.946309+0000","last_active":"2024-09-21T06:36:05.835182+0000","last_peered":"2024-09-21T06:36:05.835182+0000","last_clean":"2024-09-21T06:36:05.835182+0000","last_became_active":"2024-09-21T06:34:30.945913+0000","last_became_peered":"2024-09-21T06:34:30.945913+0000","last_unstale":"2024-09-21T06:36:05.835182+0000","last_undegraded":"2024-09-21T06:36:05.835182+0000","last_fullsized":"2024-09-21T06:36:05.835182+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:21.676524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.629308+0000","last_change":"2024-09-21T06:34:32.637636+0000","last_active":"2024-09-21T06:36:05.629308+0000","last_peered":"2024-09-21T06:36:05.629308+0000","last_clean":"2024-09-21T06:36:05.629308+0000","last_became_active":"2024-09-21T06:34:32.636534+0000","last_became_peered":"2024-09-21T06:34:32.636534+0000","last_unstale":"2024-09-21T06:36:05.629308+0000","last_undegraded":"2024-09-21T06:36:05.629308+0000","last_fullsized":"2024-09-21T06:36:05.629308+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:05.802118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771498+0000","last_change":"2024-09-21T06:35:05.274419+0000","last_active":"2024-09-21T06:36:05.771498+0000","last_peered":"2024-09-21T06:36:05.771498+0000","last_clean":"2024-09-21T06:36:05.771498+0000","last_became_active":"2024-09-21T06:35:05.273865+0000","last_became_peered":"2024-09-21T06:35:05.273865+0000","last_unstale":"2024-09-21T06:36:05.771498+0000","last_undegraded":"2024-09-21T06:36:05.771498+0000","last_fullsized":"2024-09-21T06:36:05.771498+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:04.490355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.631709+0000","last_change":"2024-09-21T06:36:05.631098+0000","last_active":"2024-09-21T06:36:05.631709+0000","last_peered":"2024-09-21T06:36:05.631709+0000","last_clean":"2024-09-21T06:36:05.631709+0000","last_became_active":"2024-09-21T06:36:05.627521+0000","last_became_peered":"2024-09-21T06:36:05.627521+0000","last_unstale":"2024-09-21T06:36:05.631709+0000","last_undegraded":"2024-09-21T06:36:05.631709+0000","last_fullsized":"2024-09-21T06:36:05.631709+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:02.467758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834863+0000","last_change":"2024-09-21T06:34:32.650303+0000","last_active":"2024-09-21T06:36:05.834863+0000","last_peered":"2024-09-21T06:36:05.834863+0000","last_clean":"2024-09-21T06:36:05.834863+0000","last_became_active":"2024-09-21T06:34:32.647189+0000","last_became_peered":"2024-09-21T06:34:32.647189+0000","last_unstale":"2024-09-21T06:36:05.834863+0000","last_undegraded":"2024-09-21T06:36:05.834863+0000","last_fullsized":"2024-09-21T06:36:05.834863+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:14.889384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771714+0000","last_change":"2024-09-21T06:34:39.713139+0000","last_active":"2024-09-21T06:36:05.771714+0000","last_peered":"2024-09-21T06:36:05.771714+0000","last_clean":"2024-09-21T06:36:05.771714+0000","last_became_active":"2024-09-21T06:34:39.712379+0000","last_became_peered":"2024-09-21T06:34:39.712379+0000","last_unstale":"2024-09-21T06:36:05.771714+0000","last_undegraded":"2024-09-21T06:36:05.771714+0000","last_fullsized":"2024-09-21T06:36:05.771714+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:53.100529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.625272+0000","last_change":"2024-09-21T06:34:30.633850+0000","last_active":"2024-09-21T06:36:05.625272+0000","last_peered":"2024-09-21T06:36:05.625272+0000","last_clean":"2024-09-21T06:36:05.625272+0000","last_became_active":"2024-09-21T06:34:30.633468+0000","last_became_peered":"2024-09-21T06:34:30.633468+0000","last_unstale":"2024-09-21T06:36:05.625272+0000","last_undegraded":"2024-09-21T06:36:05.625272+0000","last_fullsized":"2024-09-21T06:36:05.625272+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:59.095803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834771+0000","last_change":"2024-09-21T06:35:05.005240+0000","last_active":"2024-09-21T06:36:05.834771+0000","last_peered":"2024-09-21T06:36:05.834771+0000","last_clean":"2024-09-21T06:36:05.834771+0000","last_became_active":"2024-09-21T06:35:05.004849+0000","last_became_peered":"2024-09-21T06:35:05.004849+0000","last_unstale":"2024-09-21T06:36:05.834771+0000","last_undegraded":"2024-09-21T06:36:05.834771+0000","last_fullsized":"2024-09-21T06:36:05.834771+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:28.849604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771556+0000","last_change":"2024-09-21T06:35:05.274437+0000","last_active":"2024-09-21T06:36:05.771556+0000","last_peered":"2024-09-21T06:36:05.771556+0000","last_clean":"2024-09-21T06:36:05.771556+0000","last_became_active":"2024-09-21T06:35:05.273865+0000","last_became_peered":"2024-09-21T06:35:05.273865+0000","last_unstale":"2024-09-21T06:36:05.771556+0000","last_undegraded":"2024-09-21T06:36:05.771556+0000","last_fullsized":"2024-09-21T06:36:05.771556+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:43.339504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622649+0000","last_change":"2024-09-21T06:34:32.645630+0000","last_active":"2024-09-21T06:36:05.622649+0000","last_peered":"2024-09-21T06:36:05.622649+0000","last_clean":"2024-09-21T06:36:05.622649+0000","last_became_active":"2024-09-21T06:34:32.645237+0000","last_became_peered":"2024-09-21T06:34:32.645237+0000","last_unstale":"2024-09-21T06:36:05.622649+0000","last_undegraded":"2024-09-21T06:36:05.622649+0000","last_fullsized":"2024-09-21T06:36:05.622649+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:25.064318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.633983+0000","last_change":"2024-09-21T06:36:05.633484+0000","last_active":"2024-09-21T06:36:05.633983+0000","last_peered":"2024-09-21T06:36:05.633983+0000","last_clean":"2024-09-21T06:36:05.633983+0000","last_became_active":"2024-09-21T06:36:05.627637+0000","last_became_peered":"2024-09-21T06:36:05.627637+0000","last_unstale":"2024-09-21T06:36:05.633983+0000","last_undegraded":"2024-09-21T06:36:05.633983+0000","last_fullsized":"2024-09-21T06:36:05.633983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:50:00.626538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771936+0000","last_change":"2024-09-21T06:34:39.714177+0000","last_active":"2024-09-21T06:36:05.771936+0000","last_peered":"2024-09-21T06:36:05.771936+0000","last_clean":"2024-09-21T06:36:05.771936+0000","last_became_active":"2024-09-21T06:34:39.713713+0000","last_became_peered":"2024-09-21T06:34:39.713713+0000","last_unstale":"2024-09-21T06:36:05.771936+0000","last_undegraded":"2024-09-21T06:36:05.771936+0000","last_fullsized":"2024-09-21T06:36:05.771936+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:07.042240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834413+0000","last_change":"2024-09-21T06:34:32.636297+0000","last_active":"2024-09-21T06:36:05.834413+0000","last_peered":"2024-09-21T06:36:05.834413+0000","last_clean":"2024-09-21T06:36:05.834413+0000","last_became_active":"2024-09-21T06:34:32.635930+0000","last_became_peered":"2024-09-21T06:34:32.635930+0000","last_unstale":"2024-09-21T06:36:05.834413+0000","last_undegraded":"2024-09-21T06:36:05.834413+0000","last_fullsized":"2024-09-21T06:36:05.834413+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:43.091682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621887+0000","last_change":"2024-09-21T06:34:30.631405+0000","last_active":"2024-09-21T06:36:05.621887+0000","last_peered":"2024-09-21T06:36:05.621887+0000","last_clean":"2024-09-21T06:36:05.621887+0000","last_became_active":"2024-09-21T06:34:30.630512+0000","last_became_peered":"2024-09-21T06:34:30.630512+0000","last_unstale":"2024-09-21T06:36:05.621887+0000","last_undegraded":"2024-09-21T06:36:05.621887+0000","last_fullsized":"2024-09-21T06:36:05.621887+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:43.412695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"83'1","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771897+0000","last_change":"2024-09-21T06:34:39.710573+0000","last_active":"2024-09-21T06:36:05.771897+0000","last_peered":"2024-09-21T06:36:05.771897+0000","last_clean":"2024-09-21T06:36:05.771897+0000","last_became_active":"2024-09-21T06:34:39.708969+0000","last_became_peered":"2024-09-21T06:34:39.708969+0000","last_unstale":"2024-09-21T06:36:05.771897+0000","last_undegraded":"2024-09-21T06:36:05.771897+0000","last_fullsized":"2024-09-21T06:36:05.771897+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:34.847587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624019+0000","last_change":"2024-09-21T06:34:32.641085+0000","last_active":"2024-09-21T06:36:05.624019+0000","last_peered":"2024-09-21T06:36:05.624019+0000","last_clean":"2024-09-21T06:36:05.624019+0000","last_became_active":"2024-09-21T06:34:32.640723+0000","last_became_peered":"2024-09-21T06:34:32.640723+0000","last_unstale":"2024-09-21T06:36:05.624019+0000","last_undegraded":"2024-09-21T06:36:05.624019+0000","last_fullsized":"2024-09-21T06:36:05.624019+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:21.410262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834897+0000","last_change":"2024-09-21T06:34:30.626719+0000","last_active":"2024-09-21T06:36:05.834897+0000","last_peered":"2024-09-21T06:36:05.834897+0000","last_clean":"2024-09-21T06:36:05.834897+0000","last_became_active":"2024-09-21T06:34:30.626312+0000","last_became_peered":"2024-09-21T06:34:30.626312+0000","last_unstale":"2024-09-21T06:36:05.834897+0000","last_undegraded":"2024-09-21T06:36:05.834897+0000","last_fullsized":"2024-09-21T06:36:05.834897+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:53.938914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621617+0000","last_change":"2024-09-21T06:34:39.711172+0000","last_active":"2024-09-21T06:36:05.621617+0000","last_peered":"2024-09-21T06:36:05.621617+0000","last_clean":"2024-09-21T06:36:05.621617+0000","last_became_active":"2024-09-21T06:34:39.708424+0000","last_became_peered":"2024-09-21T06:34:39.708424+0000","last_unstale":"2024-09-21T06:36:05.621617+0000","last_undegraded":"2024-09-21T06:36:05.621617+0000","last_fullsized":"2024-09-21T06:36:05.621617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:43.918355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624648+0000","last_change":"2024-09-21T06:34:32.641463+0000","last_active":"2024-09-21T06:36:05.624648+0000","last_peered":"2024-09-21T06:36:05.624648+0000","last_clean":"2024-09-21T06:36:05.624648+0000","last_became_active":"2024-09-21T06:34:32.641086+0000","last_became_peered":"2024-09-21T06:34:32.641086+0000","last_unstale":"2024-09-21T06:36:05.624648+0000","last_undegraded":"2024-09-21T06:36:05.624648+0000","last_fullsized":"2024-09-21T06:36:05.624648+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:58.991374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835670+0000","last_change":"2024-09-21T06:34:30.946698+0000","last_active":"2024-09-21T06:36:05.835670+0000","last_peered":"2024-09-21T06:36:05.835670+0000","last_clean":"2024-09-21T06:36:05.835670+0000","last_became_active":"2024-09-21T06:34:30.946325+0000","last_became_peered":"2024-09-21T06:34:30.946325+0000","last_unstale":"2024-09-21T06:36:05.835670+0000","last_undegraded":"2024-09-21T06:36:05.835670+0000","last_fullsized":"2024-09-21T06:36:05.835670+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:22.639265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.772211+0000","last_change":"2024-09-21T06:34:39.712429+0000","last_active":"2024-09-21T06:36:05.772211+0000","last_peered":"2024-09-21T06:36:05.772211+0000","last_clean":"2024-09-21T06:36:05.772211+0000","last_became_active":"2024-09-21T06:34:39.711823+0000","last_became_peered":"2024-09-21T06:34:39.711823+0000","last_unstale":"2024-09-21T06:36:05.772211+0000","last_undegraded":"2024-09-21T06:36:05.772211+0000","last_fullsized":"2024-09-21T06:36:05.772211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:03:49.513991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.629145+0000","last_change":"2024-09-21T06:35:05.274326+0000","last_active":"2024-09-21T06:36:05.629145+0000","last_peered":"2024-09-21T06:36:05.629145+0000","last_clean":"2024-09-21T06:36:05.629145+0000","last_became_active":"2024-09-21T06:35:05.273884+0000","last_became_peered":"2024-09-21T06:35:05.273884+0000","last_unstale":"2024-09-21T06:36:05.629145+0000","last_undegraded":"2024-09-21T06:36:05.629145+0000","last_fullsized":"2024-09-21T06:36:05.629145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:03.219765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.772344+0000","last_change":"2024-09-21T06:34:30.676794+0000","last_active":"2024-09-21T06:36:05.772344+0000","last_peered":"2024-09-21T06:36:05.772344+0000","last_clean":"2024-09-21T06:36:05.772344+0000","last_became_active":"2024-09-21T06:34:30.676400+0000","last_became_peered":"2024-09-21T06:34:30.676400+0000","last_unstale":"2024-09-21T06:36:05.772344+0000","last_undegraded":"2024-09-21T06:36:05.772344+0000","last_fullsized":"2024-09-21T06:36:05.772344+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:23.681363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622082+0000","last_change":"2024-09-21T06:34:39.708980+0000","last_active":"2024-09-21T06:36:05.622082+0000","last_peered":"2024-09-21T06:36:05.622082+0000","last_clean":"2024-09-21T06:36:05.622082+0000","last_became_active":"2024-09-21T06:34:39.707748+0000","last_became_peered":"2024-09-21T06:34:39.707748+0000","last_unstale":"2024-09-21T06:36:05.622082+0000","last_undegraded":"2024-09-21T06:36:05.622082+0000","last_fullsized":"2024-09-21T06:36:05.622082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:16.812226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835129+0000","last_change":"2024-09-21T06:34:32.638337+0000","last_active":"2024-09-21T06:36:05.835129+0000","last_peered":"2024-09-21T06:36:05.835129+0000","last_clean":"2024-09-21T06:36:05.835129+0000","last_became_active":"2024-09-21T06:34:32.638013+0000","last_became_peered":"2024-09-21T06:34:32.638013+0000","last_unstale":"2024-09-21T06:36:05.835129+0000","last_undegraded":"2024-09-21T06:36:05.835129+0000","last_fullsized":"2024-09-21T06:36:05.835129+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:26.321190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"83'9","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621666+0000","last_change":"2024-09-21T06:34:30.631086+0000","last_active":"2024-09-21T06:36:05.621666+0000","last_peered":"2024-09-21T06:36:05.621666+0000","last_clean":"2024-09-21T06:36:05.621666+0000","last_became_active":"2024-09-21T06:34:30.630706+0000","last_became_peered":"2024-09-21T06:34:30.630706+0000","last_unstale":"2024-09-21T06:36:05.621666+0000","last_undegraded":"2024-09-21T06:36:05.621666+0000","last_fullsized":"2024-09-21T06:36:05.621666+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:30.260359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.833935+0000","last_change":"2024-09-21T06:34:39.709097+0000","last_active":"2024-09-21T06:36:05.833935+0000","last_peered":"2024-09-21T06:36:05.833935+0000","last_clean":"2024-09-21T06:36:05.833935+0000","last_became_active":"2024-09-21T06:34:39.708609+0000","last_became_peered":"2024-09-21T06:34:39.708609+0000","last_unstale":"2024-09-21T06:36:05.833935+0000","last_undegraded":"2024-09-21T06:36:05.833935+0000","last_fullsized":"2024-09-21T06:36:05.833935+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:48.842174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834498+0000","last_change":"2024-09-21T06:34:32.636816+0000","last_active":"2024-09-21T06:36:05.834498+0000","last_peered":"2024-09-21T06:36:05.834498+0000","last_clean":"2024-09-21T06:36:05.834498+0000","last_became_active":"2024-09-21T06:34:32.636429+0000","last_became_peered":"2024-09-21T06:34:32.636429+0000","last_unstale":"2024-09-21T06:36:05.834498+0000","last_undegraded":"2024-09-21T06:36:05.834498+0000","last_fullsized":"2024-09-21T06:36:05.834498+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:49.010491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.625566+0000","last_change":"2024-09-21T06:34:30.947481+0000","last_active":"2024-09-21T06:36:05.625566+0000","last_peered":"2024-09-21T06:36:05.625566+0000","last_clean":"2024-09-21T06:36:05.625566+0000","last_became_active":"2024-09-21T06:34:30.947040+0000","last_became_peered":"2024-09-21T06:34:30.947040+0000","last_unstale":"2024-09-21T06:36:05.625566+0000","last_undegraded":"2024-09-21T06:36:05.625566+0000","last_fullsized":"2024-09-21T06:36:05.625566+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:52.813476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624899+0000","last_change":"2024-09-21T06:34:39.708674+0000","last_active":"2024-09-21T06:36:05.624899+0000","last_peered":"2024-09-21T06:36:05.624899+0000","last_clean":"2024-09-21T06:36:05.624899+0000","last_became_active":"2024-09-21T06:34:39.708176+0000","last_became_peered":"2024-09-21T06:34:39.708176+0000","last_unstale":"2024-09-21T06:36:05.624899+0000","last_undegraded":"2024-09-21T06:36:05.624899+0000","last_fullsized":"2024-09-21T06:36:05.624899+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:57.070792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834505+0000","last_change":"2024-09-21T06:34:32.636381+0000","last_active":"2024-09-21T06:36:05.834505+0000","last_peered":"2024-09-21T06:36:05.834505+0000","last_clean":"2024-09-21T06:36:05.834505+0000","last_became_active":"2024-09-21T06:34:32.635937+0000","last_became_peered":"2024-09-21T06:34:32.635937+0000","last_unstale":"2024-09-21T06:36:05.834505+0000","last_undegraded":"2024-09-21T06:36:05.834505+0000","last_fullsized":"2024-09-21T06:36:05.834505+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:52.870676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621464+0000","last_change":"2024-09-21T06:34:30.947160+0000","last_active":"2024-09-21T06:36:05.621464+0000","last_peered":"2024-09-21T06:36:05.621464+0000","last_clean":"2024-09-21T06:36:05.621464+0000","last_became_active":"2024-09-21T06:34:30.946736+0000","last_became_peered":"2024-09-21T06:34:30.946736+0000","last_unstale":"2024-09-21T06:36:05.621464+0000","last_undegraded":"2024-09-21T06:36:05.621464+0000","last_fullsized":"2024-09-21T06:36:05.621464+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:47.425481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624815+0000","last_change":"2024-09-21T06:35:05.006032+0000","last_active":"2024-09-21T06:36:05.624815+0000","last_peered":"2024-09-21T06:36:05.624815+0000","last_clean":"2024-09-21T06:36:05.624815+0000","last_became_active":"2024-09-21T06:35:05.005429+0000","last_became_peered":"2024-09-21T06:35:05.005429+0000","last_unstale":"2024-09-21T06:36:05.624815+0000","last_undegraded":"2024-09-21T06:36:05.624815+0000","last_fullsized":"2024-09-21T06:36:05.624815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:04.138097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834598+0000","last_change":"2024-09-21T06:34:32.648256+0000","last_active":"2024-09-21T06:36:05.834598+0000","last_peered":"2024-09-21T06:36:05.834598+0000","last_clean":"2024-09-21T06:36:05.834598+0000","last_became_active":"2024-09-21T06:34:32.647881+0000","last_became_peered":"2024-09-21T06:34:32.647881+0000","last_unstale":"2024-09-21T06:36:05.834598+0000","last_undegraded":"2024-09-21T06:36:05.834598+0000","last_fullsized":"2024-09-21T06:36:05.834598+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:14.590834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622843+0000","last_change":"2024-09-21T06:34:30.629918+0000","last_active":"2024-09-21T06:36:05.622843+0000","last_peered":"2024-09-21T06:36:05.622843+0000","last_clean":"2024-09-21T06:36:05.622843+0000","last_became_active":"2024-09-21T06:34:30.628785+0000","last_became_peered":"2024-09-21T06:34:30.628785+0000","last_unstale":"2024-09-21T06:36:05.622843+0000","last_undegraded":"2024-09-21T06:36:05.622843+0000","last_fullsized":"2024-09-21T06:36:05.622843+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:05.333406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834369+0000","last_change":"2024-09-21T06:34:39.710073+0000","last_active":"2024-09-21T06:36:05.834369+0000","last_peered":"2024-09-21T06:36:05.834369+0000","last_clean":"2024-09-21T06:36:05.834369+0000","last_became_active":"2024-09-21T06:34:39.708986+0000","last_became_peered":"2024-09-21T06:34:39.708986+0000","last_unstale":"2024-09-21T06:36:05.834369+0000","last_undegraded":"2024-09-21T06:36:05.834369+0000","last_fullsized":"2024-09-21T06:36:05.834369+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:46.379902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834487+0000","last_change":"2024-09-21T06:34:32.644224+0000","last_active":"2024-09-21T06:36:05.834487+0000","last_peered":"2024-09-21T06:36:05.834487+0000","last_clean":"2024-09-21T06:36:05.834487+0000","last_became_active":"2024-09-21T06:34:32.643850+0000","last_became_peered":"2024-09-21T06:34:32.643850+0000","last_unstale":"2024-09-21T06:36:05.834487+0000","last_undegraded":"2024-09-21T06:36:05.834487+0000","last_fullsized":"2024-09-21T06:36:05.834487+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:24:33.188537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.833892+0000","last_change":"2024-09-21T06:34:30.946334+0000","last_active":"2024-09-21T06:36:05.833892+0000","last_peered":"2024-09-21T06:36:05.833892+0000","last_clean":"2024-09-21T06:36:05.833892+0000","last_became_active":"2024-09-21T06:34:30.945908+0000","last_became_peered":"2024-09-21T06:34:30.945908+0000","last_unstale":"2024-09-21T06:36:05.833892+0000","last_undegraded":"2024-09-21T06:36:05.833892+0000","last_fullsized":"2024-09-21T06:36:05.833892+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:46.746527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834586+0000","last_change":"2024-09-21T06:34:39.707909+0000","last_active":"2024-09-21T06:36:05.834586+0000","last_peered":"2024-09-21T06:36:05.834586+0000","last_clean":"2024-09-21T06:36:05.834586+0000","last_became_active":"2024-09-21T06:34:39.707492+0000","last_became_peered":"2024-09-21T06:34:39.707492+0000","last_unstale":"2024-09-21T06:36:05.834586+0000","last_undegraded":"2024-09-21T06:36:05.834586+0000","last_fullsized":"2024-09-21T06:36:05.834586+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:56.201212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.833745+0000","last_change":"2024-09-21T06:34:32.635870+0000","last_active":"2024-09-21T06:36:05.833745+0000","last_peered":"2024-09-21T06:36:05.833745+0000","last_clean":"2024-09-21T06:36:05.833745+0000","last_became_active":"2024-09-21T06:34:32.635252+0000","last_became_peered":"2024-09-21T06:34:32.635252+0000","last_unstale":"2024-09-21T06:36:05.833745+0000","last_undegraded":"2024-09-21T06:36:05.833745+0000","last_fullsized":"2024-09-21T06:36:05.833745+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:10.613033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622266+0000","last_change":"2024-09-21T06:34:30.634360+0000","last_active":"2024-09-21T06:36:05.622266+0000","last_peered":"2024-09-21T06:36:05.622266+0000","last_clean":"2024-09-21T06:36:05.622266+0000","last_became_active":"2024-09-21T06:34:30.633845+0000","last_became_peered":"2024-09-21T06:34:30.633845+0000","last_unstale":"2024-09-21T06:36:05.622266+0000","last_undegraded":"2024-09-21T06:36:05.622266+0000","last_fullsized":"2024-09-21T06:36:05.622266+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:34:53.388670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834837+0000","last_change":"2024-09-21T06:34:39.719620+0000","last_active":"2024-09-21T06:36:05.834837+0000","last_peered":"2024-09-21T06:36:05.834837+0000","last_clean":"2024-09-21T06:36:05.834837+0000","last_became_active":"2024-09-21T06:34:39.718786+0000","last_became_peered":"2024-09-21T06:34:39.718786+0000","last_unstale":"2024-09-21T06:36:05.834837+0000","last_undegraded":"2024-09-21T06:36:05.834837+0000","last_fullsized":"2024-09-21T06:36:05.834837+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:43.524932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834613+0000","last_change":"2024-09-21T06:34:32.634970+0000","last_active":"2024-09-21T06:36:05.834613+0000","last_peered":"2024-09-21T06:36:05.834613+0000","last_clean":"2024-09-21T06:36:05.834613+0000","last_became_active":"2024-09-21T06:34:32.634611+0000","last_became_peered":"2024-09-21T06:34:32.634611+0000","last_unstale":"2024-09-21T06:36:05.834613+0000","last_undegraded":"2024-09-21T06:36:05.834613+0000","last_fullsized":"2024-09-21T06:36:05.834613+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:37.855849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.772108+0000","last_change":"2024-09-21T06:34:30.628123+0000","last_active":"2024-09-21T06:36:05.772108+0000","last_peered":"2024-09-21T06:36:05.772108+0000","last_clean":"2024-09-21T06:36:05.772108+0000","last_became_active":"2024-09-21T06:34:30.627498+0000","last_became_peered":"2024-09-21T06:34:30.627498+0000","last_unstale":"2024-09-21T06:36:05.772108+0000","last_undegraded":"2024-09-21T06:36:05.772108+0000","last_fullsized":"2024-09-21T06:36:05.772108+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:42.576441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":115,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622503+0000","last_change":"2024-09-21T06:33:16.856623+0000","last_active":"2024-09-21T06:36:05.622503+0000","last_peered":"2024-09-21T06:36:05.622503+0000","last_clean":"2024-09-21T06:36:05.622503+0000","last_became_active":"2024-09-21T06:33:16.850787+0000","last_became_peered":"2024-09-21T06:33:16.850787+0000","last_unstale":"2024-09-21T06:36:05.622503+0000","last_undegraded":"2024-09-21T06:36:05.622503+0000","last_fullsized":"2024-09-21T06:36:05.622503+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:50.283252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:50.283252+0000","last_clean_scrub_stamp":"2024-09-21T06:29:50.283252+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:29:20.711208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622480+0000","last_change":"2024-09-21T06:34:39.706472+0000","last_active":"2024-09-21T06:36:05.622480+0000","last_peered":"2024-09-21T06:36:05.622480+0000","last_clean":"2024-09-21T06:36:05.622480+0000","last_became_active":"2024-09-21T06:34:39.705724+0000","last_became_peered":"2024-09-21T06:34:39.705724+0000","last_unstale":"2024-09-21T06:36:05.622480+0000","last_undegraded":"2024-09-21T06:36:05.622480+0000","last_fullsized":"2024-09-21T06:36:05.622480+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:43.238704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622792+0000","last_change":"2024-09-21T06:34:32.645944+0000","last_active":"2024-09-21T06:36:05.622792+0000","last_peered":"2024-09-21T06:36:05.622792+0000","last_clean":"2024-09-21T06:36:05.622792+0000","last_became_active":"2024-09-21T06:34:32.645134+0000","last_became_peered":"2024-09-21T06:34:32.645134+0000","last_unstale":"2024-09-21T06:36:05.622792+0000","last_undegraded":"2024-09-21T06:36:05.622792+0000","last_fullsized":"2024-09-21T06:36:05.622792+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:45.254164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622567+0000","last_change":"2024-09-21T06:34:30.946296+0000","last_active":"2024-09-21T06:36:05.622567+0000","last_peered":"2024-09-21T06:36:05.622567+0000","last_clean":"2024-09-21T06:36:05.622567+0000","last_became_active":"2024-09-21T06:34:30.945777+0000","last_became_peered":"2024-09-21T06:34:30.945777+0000","last_unstale":"2024-09-21T06:36:05.622567+0000","last_undegraded":"2024-09-21T06:36:05.622567+0000","last_fullsized":"2024-09-21T06:36:05.622567+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:32:25.988512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835077+0000","last_change":"2024-09-21T06:34:39.705876+0000","last_active":"2024-09-21T06:36:05.835077+0000","last_peered":"2024-09-21T06:36:05.835077+0000","last_clean":"2024-09-21T06:36:05.835077+0000","last_became_active":"2024-09-21T06:34:39.705132+0000","last_became_peered":"2024-09-21T06:34:39.705132+0000","last_unstale":"2024-09-21T06:36:05.835077+0000","last_undegraded":"2024-09-21T06:36:05.835077+0000","last_fullsized":"2024-09-21T06:36:05.835077+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:18.180244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622740+0000","last_change":"2024-09-21T06:34:32.638242+0000","last_active":"2024-09-21T06:36:05.622740+0000","last_peered":"2024-09-21T06:36:05.622740+0000","last_clean":"2024-09-21T06:36:05.622740+0000","last_became_active":"2024-09-21T06:34:32.637816+0000","last_became_peered":"2024-09-21T06:34:32.637816+0000","last_unstale":"2024-09-21T06:36:05.622740+0000","last_undegraded":"2024-09-21T06:36:05.622740+0000","last_fullsized":"2024-09-21T06:36:05.622740+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:40.926147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834922+0000","last_change":"2024-09-21T06:34:30.632173+0000","last_active":"2024-09-21T06:36:05.834922+0000","last_peered":"2024-09-21T06:36:05.834922+0000","last_clean":"2024-09-21T06:36:05.834922+0000","last_became_active":"2024-09-21T06:34:30.631638+0000","last_became_peered":"2024-09-21T06:34:30.631638+0000","last_unstale":"2024-09-21T06:36:05.834922+0000","last_undegraded":"2024-09-21T06:36:05.834922+0000","last_fullsized":"2024-09-21T06:36:05.834922+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:29.763272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.628935+0000","last_change":"2024-09-21T06:34:39.714620+0000","last_active":"2024-09-21T06:36:05.628935+0000","last_peered":"2024-09-21T06:36:05.628935+0000","last_clean":"2024-09-21T06:36:05.628935+0000","last_became_active":"2024-09-21T06:34:39.714199+0000","last_became_peered":"2024-09-21T06:34:39.714199+0000","last_unstale":"2024-09-21T06:36:05.628935+0000","last_undegraded":"2024-09-21T06:36:05.628935+0000","last_fullsized":"2024-09-21T06:36:05.628935+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:37.041249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622722+0000","last_change":"2024-09-21T06:34:32.645001+0000","last_active":"2024-09-21T06:36:05.622722+0000","last_peered":"2024-09-21T06:36:05.622722+0000","last_clean":"2024-09-21T06:36:05.622722+0000","last_became_active":"2024-09-21T06:34:32.644584+0000","last_became_peered":"2024-09-21T06:34:32.644584+0000","last_unstale":"2024-09-21T06:36:05.622722+0000","last_undegraded":"2024-09-21T06:36:05.622722+0000","last_fullsized":"2024-09-21T06:36:05.622722+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:47.098850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834960+0000","last_change":"2024-09-21T06:34:30.632593+0000","last_active":"2024-09-21T06:36:05.834960+0000","last_peered":"2024-09-21T06:36:05.834960+0000","last_clean":"2024-09-21T06:36:05.834960+0000","last_became_active":"2024-09-21T06:34:30.632170+0000","last_became_peered":"2024-09-21T06:34:30.632170+0000","last_unstale":"2024-09-21T06:36:05.834960+0000","last_undegraded":"2024-09-21T06:36:05.834960+0000","last_fullsized":"2024-09-21T06:36:05.834960+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:56.170655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834120+0000","last_change":"2024-09-21T06:34:39.710172+0000","last_active":"2024-09-21T06:36:05.834120+0000","last_peered":"2024-09-21T06:36:05.834120+0000","last_clean":"2024-09-21T06:36:05.834120+0000","last_became_active":"2024-09-21T06:34:39.708876+0000","last_became_peered":"2024-09-21T06:34:39.708876+0000","last_unstale":"2024-09-21T06:36:05.834120+0000","last_undegraded":"2024-09-21T06:36:05.834120+0000","last_fullsized":"2024-09-21T06:36:05.834120+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:32.371873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624520+0000","last_change":"2024-09-21T06:34:32.651056+0000","last_active":"2024-09-21T06:36:05.624520+0000","last_peered":"2024-09-21T06:36:05.624520+0000","last_clean":"2024-09-21T06:36:05.624520+0000","last_became_active":"2024-09-21T06:34:32.650676+0000","last_became_peered":"2024-09-21T06:34:32.650676+0000","last_unstale":"2024-09-21T06:36:05.624520+0000","last_undegraded":"2024-09-21T06:36:05.624520+0000","last_fullsized":"2024-09-21T06:36:05.624520+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:34.599654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.625649+0000","last_change":"2024-09-21T06:34:30.632307+0000","last_active":"2024-09-21T06:36:05.625649+0000","last_peered":"2024-09-21T06:36:05.625649+0000","last_clean":"2024-09-21T06:36:05.625649+0000","last_became_active":"2024-09-21T06:34:30.631920+0000","last_became_peered":"2024-09-21T06:34:30.631920+0000","last_unstale":"2024-09-21T06:36:05.625649+0000","last_undegraded":"2024-09-21T06:36:05.625649+0000","last_fullsized":"2024-09-21T06:36:05.625649+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:46.940898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834319+0000","last_change":"2024-09-21T06:34:39.715056+0000","last_active":"2024-09-21T06:36:05.834319+0000","last_peered":"2024-09-21T06:36:05.834319+0000","last_clean":"2024-09-21T06:36:05.834319+0000","last_became_active":"2024-09-21T06:34:39.714563+0000","last_became_peered":"2024-09-21T06:34:39.714563+0000","last_unstale":"2024-09-21T06:36:05.834319+0000","last_undegraded":"2024-09-21T06:36:05.834319+0000","last_fullsized":"2024-09-21T06:36:05.834319+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:48.567057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834369+0000","last_change":"2024-09-21T06:34:32.646934+0000","last_active":"2024-09-21T06:36:05.834369+0000","last_peered":"2024-09-21T06:36:05.834369+0000","last_clean":"2024-09-21T06:36:05.834369+0000","last_became_active":"2024-09-21T06:34:32.646559+0000","last_became_peered":"2024-09-21T06:34:32.646559+0000","last_unstale":"2024-09-21T06:36:05.834369+0000","last_undegraded":"2024-09-21T06:36:05.834369+0000","last_fullsized":"2024-09-21T06:36:05.834369+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:37.802590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":64,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.843100+0000","last_change":"2024-09-21T06:36:05.843018+0000","last_active":"2024-09-21T06:36:05.843100+0000","last_peered":"2024-09-21T06:36:05.843100+0000","last_clean":"2024-09-21T06:36:05.843100+0000","last_became_active":"2024-09-21T06:36:05.836284+0000","last_became_peered":"2024-09-21T06:36:05.836284+0000","last_unstale":"2024-09-21T06:36:05.843100+0000","last_undegraded":"2024-09-21T06:36:05.843100+0000","last_fullsized":"2024-09-21T06:36:05.843100+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:51.539804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834952+0000","last_change":"2024-09-21T06:34:39.713398+0000","last_active":"2024-09-21T06:36:05.834952+0000","last_peered":"2024-09-21T06:36:05.834952+0000","last_clean":"2024-09-21T06:36:05.834952+0000","last_became_active":"2024-09-21T06:34:39.712954+0000","last_became_peered":"2024-09-21T06:34:39.712954+0000","last_unstale":"2024-09-21T06:36:05.834952+0000","last_undegraded":"2024-09-21T06:36:05.834952+0000","last_fullsized":"2024-09-21T06:36:05.834952+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:58.881014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622265+0000","last_change":"2024-09-21T06:34:32.640879+0000","last_active":"2024-09-21T06:36:05.622265+0000","last_peered":"2024-09-21T06:36:05.622265+0000","last_clean":"2024-09-21T06:36:05.622265+0000","last_became_active":"2024-09-21T06:34:32.640526+0000","last_became_peered":"2024-09-21T06:34:32.640526+0000","last_unstale":"2024-09-21T06:36:05.622265+0000","last_undegraded":"2024-09-21T06:36:05.622265+0000","last_fullsized":"2024-09-21T06:36:05.622265+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:12.327195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771895+0000","last_change":"2024-09-21T06:34:30.631449+0000","last_active":"2024-09-21T06:36:05.771895+0000","last_peered":"2024-09-21T06:36:05.771895+0000","last_clean":"2024-09-21T06:36:05.771895+0000","last_became_active":"2024-09-21T06:34:30.631081+0000","last_became_peered":"2024-09-21T06:34:30.631081+0000","last_unstale":"2024-09-21T06:36:05.771895+0000","last_undegraded":"2024-09-21T06:36:05.771895+0000","last_fullsized":"2024-09-21T06:36:05.771895+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:09.202696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621660+0000","last_change":"2024-09-21T06:34:39.707379+0000","last_active":"2024-09-21T06:36:05.621660+0000","last_peered":"2024-09-21T06:36:05.621660+0000","last_clean":"2024-09-21T06:36:05.621660+0000","last_became_active":"2024-09-21T06:34:39.706979+0000","last_became_peered":"2024-09-21T06:34:39.706979+0000","last_unstale":"2024-09-21T06:36:05.621660+0000","last_undegraded":"2024-09-21T06:36:05.621660+0000","last_fullsized":"2024-09-21T06:36:05.621660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:24.394233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622354+0000","last_change":"2024-09-21T06:34:32.649458+0000","last_active":"2024-09-21T06:36:05.622354+0000","last_peered":"2024-09-21T06:36:05.622354+0000","last_clean":"2024-09-21T06:36:05.622354+0000","last_became_active":"2024-09-21T06:34:32.649051+0000","last_became_peered":"2024-09-21T06:34:32.649051+0000","last_unstale":"2024-09-21T06:36:05.622354+0000","last_undegraded":"2024-09-21T06:36:05.622354+0000","last_fullsized":"2024-09-21T06:36:05.622354+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:12.631446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834455+0000","last_change":"2024-09-21T06:34:30.634592+0000","last_active":"2024-09-21T06:36:05.834455+0000","last_peered":"2024-09-21T06:36:05.834455+0000","last_clean":"2024-09-21T06:36:05.834455+0000","last_became_active":"2024-09-21T06:34:30.634251+0000","last_became_peered":"2024-09-21T06:34:30.634251+0000","last_unstale":"2024-09-21T06:36:05.834455+0000","last_undegraded":"2024-09-21T06:36:05.834455+0000","last_fullsized":"2024-09-21T06:36:05.834455+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:32.817467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624233+0000","last_change":"2024-09-21T06:34:39.719416+0000","last_active":"2024-09-21T06:36:05.624233+0000","last_peered":"2024-09-21T06:36:05.624233+0000","last_clean":"2024-09-21T06:36:05.624233+0000","last_became_active":"2024-09-21T06:34:39.718910+0000","last_became_peered":"2024-09-21T06:34:39.718910+0000","last_unstale":"2024-09-21T06:36:05.624233+0000","last_undegraded":"2024-09-21T06:36:05.624233+0000","last_fullsized":"2024-09-21T06:36:05.624233+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:41.413250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.833982+0000","last_change":"2024-09-21T06:34:32.640727+0000","last_active":"2024-09-21T06:36:05.833982+0000","last_peered":"2024-09-21T06:36:05.833982+0000","last_clean":"2024-09-21T06:36:05.833982+0000","last_became_active":"2024-09-21T06:34:32.640296+0000","last_became_peered":"2024-09-21T06:34:32.640296+0000","last_unstale":"2024-09-21T06:36:05.833982+0000","last_undegraded":"2024-09-21T06:36:05.833982+0000","last_fullsized":"2024-09-21T06:36:05.833982+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:35.124059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622118+0000","last_change":"2024-09-21T06:34:30.633332+0000","last_active":"2024-09-21T06:36:05.622118+0000","last_peered":"2024-09-21T06:36:05.622118+0000","last_clean":"2024-09-21T06:36:05.622118+0000","last_became_active":"2024-09-21T06:34:30.632641+0000","last_became_peered":"2024-09-21T06:34:30.632641+0000","last_unstale":"2024-09-21T06:36:05.622118+0000","last_undegraded":"2024-09-21T06:36:05.622118+0000","last_fullsized":"2024-09-21T06:36:05.622118+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:30:23.930313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834659+0000","last_change":"2024-09-21T06:34:39.712504+0000","last_active":"2024-09-21T06:36:05.834659+0000","last_peered":"2024-09-21T06:36:05.834659+0000","last_clean":"2024-09-21T06:36:05.834659+0000","last_became_active":"2024-09-21T06:34:39.711638+0000","last_became_peered":"2024-09-21T06:34:39.711638+0000","last_unstale":"2024-09-21T06:36:05.834659+0000","last_undegraded":"2024-09-21T06:36:05.834659+0000","last_fullsized":"2024-09-21T06:36:05.834659+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:26:09.003367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.625165+0000","last_change":"2024-09-21T06:34:32.639582+0000","last_active":"2024-09-21T06:36:05.625165+0000","last_peered":"2024-09-21T06:36:05.625165+0000","last_clean":"2024-09-21T06:36:05.625165+0000","last_became_active":"2024-09-21T06:34:32.639023+0000","last_became_peered":"2024-09-21T06:34:32.639023+0000","last_unstale":"2024-09-21T06:36:05.625165+0000","last_undegraded":"2024-09-21T06:36:05.625165+0000","last_fullsized":"2024-09-21T06:36:05.625165+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:26.773389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"83'30","reported_seq":202,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:10.136335+0000","last_change":"2024-09-21T06:34:30.626280+0000","last_active":"2024-09-21T06:36:10.136335+0000","last_peered":"2024-09-21T06:36:10.136335+0000","last_clean":"2024-09-21T06:36:10.136335+0000","last_became_active":"2024-09-21T06:34:30.625662+0000","last_became_peered":"2024-09-21T06:34:30.625662+0000","last_unstale":"2024-09-21T06:36:10.136335+0000","last_undegraded":"2024-09-21T06:36:10.136335+0000","last_fullsized":"2024-09-21T06:36:10.136335+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:10.541881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68785,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":133,"num_read_kb":134,"num_write":30,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621183+0000","last_change":"2024-09-21T06:34:39.708999+0000","last_active":"2024-09-21T06:36:05.621183+0000","last_peered":"2024-09-21T06:36:05.621183+0000","last_clean":"2024-09-21T06:36:05.621183+0000","last_became_active":"2024-09-21T06:34:39.708580+0000","last_became_peered":"2024-09-21T06:34:39.708580+0000","last_unstale":"2024-09-21T06:36:05.621183+0000","last_undegraded":"2024-09-21T06:36:05.621183+0000","last_fullsized":"2024-09-21T06:36:05.621183+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:07.648358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835047+0000","last_change":"2024-09-21T06:34:32.651157+0000","last_active":"2024-09-21T06:36:05.835047+0000","last_peered":"2024-09-21T06:36:05.835047+0000","last_clean":"2024-09-21T06:36:05.835047+0000","last_became_active":"2024-09-21T06:34:32.650804+0000","last_became_peered":"2024-09-21T06:34:32.650804+0000","last_unstale":"2024-09-21T06:36:05.835047+0000","last_undegraded":"2024-09-21T06:36:05.835047+0000","last_fullsized":"2024-09-21T06:36:05.835047+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:49.758496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834880+0000","last_change":"2024-09-21T06:34:30.947142+0000","last_active":"2024-09-21T06:36:05.834880+0000","last_peered":"2024-09-21T06:36:05.834880+0000","last_clean":"2024-09-21T06:36:05.834880+0000","last_became_active":"2024-09-21T06:34:30.946748+0000","last_became_peered":"2024-09-21T06:34:30.946748+0000","last_unstale":"2024-09-21T06:36:05.834880+0000","last_undegraded":"2024-09-21T06:36:05.834880+0000","last_fullsized":"2024-09-21T06:36:05.834880+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:39.222286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771885+0000","last_change":"2024-09-21T06:34:39.709266+0000","last_active":"2024-09-21T06:36:05.771885+0000","last_peered":"2024-09-21T06:36:05.771885+0000","last_clean":"2024-09-21T06:36:05.771885+0000","last_became_active":"2024-09-21T06:34:39.708753+0000","last_became_peered":"2024-09-21T06:34:39.708753+0000","last_unstale":"2024-09-21T06:36:05.771885+0000","last_undegraded":"2024-09-21T06:36:05.771885+0000","last_fullsized":"2024-09-21T06:36:05.771885+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:48.711681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.629321+0000","last_change":"2024-09-21T06:34:32.637976+0000","last_active":"2024-09-21T06:36:05.629321+0000","last_peered":"2024-09-21T06:36:05.629321+0000","last_clean":"2024-09-21T06:36:05.629321+0000","last_became_active":"2024-09-21T06:34:32.637267+0000","last_became_peered":"2024-09-21T06:34:32.637267+0000","last_unstale":"2024-09-21T06:36:05.629321+0000","last_undegraded":"2024-09-21T06:36:05.629321+0000","last_fullsized":"2024-09-21T06:36:05.629321+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:31.340583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.836482+0000","last_change":"2024-09-21T06:36:05.836382+0000","last_active":"2024-09-21T06:36:05.836482+0000","last_peered":"2024-09-21T06:36:05.836482+0000","last_clean":"2024-09-21T06:36:05.836482+0000","last_became_active":"2024-09-21T06:36:05.835918+0000","last_became_peered":"2024-09-21T06:36:05.835918+0000","last_unstale":"2024-09-21T06:36:05.836482+0000","last_undegraded":"2024-09-21T06:36:05.836482+0000","last_fullsized":"2024-09-21T06:36:05.836482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:47.346400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621023+0000","last_change":"2024-09-21T06:34:39.713831+0000","last_active":"2024-09-21T06:36:05.621023+0000","last_peered":"2024-09-21T06:36:05.621023+0000","last_clean":"2024-09-21T06:36:05.621023+0000","last_became_active":"2024-09-21T06:34:39.713403+0000","last_became_peered":"2024-09-21T06:34:39.713403+0000","last_unstale":"2024-09-21T06:36:05.621023+0000","last_undegraded":"2024-09-21T06:36:05.621023+0000","last_fullsized":"2024-09-21T06:36:05.621023+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:22.201063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834704+0000","last_change":"2024-09-21T06:34:32.646924+0000","last_active":"2024-09-21T06:36:05.834704+0000","last_peered":"2024-09-21T06:36:05.834704+0000","last_clean":"2024-09-21T06:36:05.834704+0000","last_became_active":"2024-09-21T06:34:32.646531+0000","last_became_peered":"2024-09-21T06:34:32.646531+0000","last_unstale":"2024-09-21T06:36:05.834704+0000","last_undegraded":"2024-09-21T06:36:05.834704+0000","last_fullsized":"2024-09-21T06:36:05.834704+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:48.550193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835325+0000","last_change":"2024-09-21T06:34:30.631901+0000","last_active":"2024-09-21T06:36:05.835325+0000","last_peered":"2024-09-21T06:36:05.835325+0000","last_clean":"2024-09-21T06:36:05.835325+0000","last_became_active":"2024-09-21T06:34:30.631450+0000","last_became_peered":"2024-09-21T06:34:30.631450+0000","last_unstale":"2024-09-21T06:36:05.835325+0000","last_undegraded":"2024-09-21T06:36:05.835325+0000","last_fullsized":"2024-09-21T06:36:05.835325+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:20.709456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834441+0000","last_change":"2024-09-21T06:34:39.711951+0000","last_active":"2024-09-21T06:36:05.834441+0000","last_peered":"2024-09-21T06:36:05.834441+0000","last_clean":"2024-09-21T06:36:05.834441+0000","last_became_active":"2024-09-21T06:34:39.710783+0000","last_became_peered":"2024-09-21T06:34:39.710783+0000","last_unstale":"2024-09-21T06:36:05.834441+0000","last_undegraded":"2024-09-21T06:36:05.834441+0000","last_fullsized":"2024-09-21T06:36:05.834441+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:27:58.501155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.629019+0000","last_change":"2024-09-21T06:34:39.709639+0000","last_active":"2024-09-21T06:36:05.629019+0000","last_peered":"2024-09-21T06:36:05.629019+0000","last_clean":"2024-09-21T06:36:05.629019+0000","last_became_active":"2024-09-21T06:34:39.709241+0000","last_became_peered":"2024-09-21T06:34:39.709241+0000","last_unstale":"2024-09-21T06:36:05.629019+0000","last_undegraded":"2024-09-21T06:36:05.629019+0000","last_fullsized":"2024-09-21T06:36:05.629019+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.357399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621131+0000","last_change":"2024-09-21T06:34:32.635365+0000","last_active":"2024-09-21T06:36:05.621131+0000","last_peered":"2024-09-21T06:36:05.621131+0000","last_clean":"2024-09-21T06:36:05.621131+0000","last_became_active":"2024-09-21T06:34:32.635012+0000","last_became_peered":"2024-09-21T06:34:32.635012+0000","last_unstale":"2024-09-21T06:36:05.621131+0000","last_undegraded":"2024-09-21T06:36:05.621131+0000","last_fullsized":"2024-09-21T06:36:05.621131+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:54.671074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834581+0000","last_change":"2024-09-21T06:34:30.633347+0000","last_active":"2024-09-21T06:36:05.834581+0000","last_peered":"2024-09-21T06:36:05.834581+0000","last_clean":"2024-09-21T06:36:05.834581+0000","last_became_active":"2024-09-21T06:34:30.632513+0000","last_became_peered":"2024-09-21T06:34:30.632513+0000","last_unstale":"2024-09-21T06:36:05.834581+0000","last_undegraded":"2024-09-21T06:36:05.834581+0000","last_fullsized":"2024-09-21T06:36:05.834581+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:12.439109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624773+0000","last_change":"2024-09-21T06:34:39.710204+0000","last_active":"2024-09-21T06:36:05.624773+0000","last_peered":"2024-09-21T06:36:05.624773+0000","last_clean":"2024-09-21T06:36:05.624773+0000","last_became_active":"2024-09-21T06:34:39.709744+0000","last_became_peered":"2024-09-21T06:34:39.709744+0000","last_unstale":"2024-09-21T06:36:05.624773+0000","last_undegraded":"2024-09-21T06:36:05.624773+0000","last_fullsized":"2024-09-21T06:36:05.624773+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:45.484088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834193+0000","last_change":"2024-09-21T06:34:32.634170+0000","last_active":"2024-09-21T06:36:05.834193+0000","last_peered":"2024-09-21T06:36:05.834193+0000","last_clean":"2024-09-21T06:36:05.834193+0000","last_became_active":"2024-09-21T06:34:32.633831+0000","last_became_peered":"2024-09-21T06:34:32.633831+0000","last_unstale":"2024-09-21T06:36:05.834193+0000","last_undegraded":"2024-09-21T06:36:05.834193+0000","last_fullsized":"2024-09-21T06:36:05.834193+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:36.738279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.833746+0000","last_change":"2024-09-21T06:34:30.629422+0000","last_active":"2024-09-21T06:36:05.833746+0000","last_peered":"2024-09-21T06:36:05.833746+0000","last_clean":"2024-09-21T06:36:05.833746+0000","last_became_active":"2024-09-21T06:34:30.629025+0000","last_became_peered":"2024-09-21T06:34:30.629025+0000","last_unstale":"2024-09-21T06:36:05.833746+0000","last_undegraded":"2024-09-21T06:36:05.833746+0000","last_fullsized":"2024-09-21T06:36:05.833746+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:05.068816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834561+0000","last_change":"2024-09-21T06:34:30.634562+0000","last_active":"2024-09-21T06:36:05.834561+0000","last_peered":"2024-09-21T06:36:05.834561+0000","last_clean":"2024-09-21T06:36:05.834561+0000","last_became_active":"2024-09-21T06:34:30.634170+0000","last_became_peered":"2024-09-21T06:34:30.634170+0000","last_unstale":"2024-09-21T06:36:05.834561+0000","last_undegraded":"2024-09-21T06:36:05.834561+0000","last_fullsized":"2024-09-21T06:36:05.834561+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:15.243103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834518+0000","last_change":"2024-09-21T06:34:32.645718+0000","last_active":"2024-09-21T06:36:05.834518+0000","last_peered":"2024-09-21T06:36:05.834518+0000","last_clean":"2024-09-21T06:36:05.834518+0000","last_became_active":"2024-09-21T06:34:32.645072+0000","last_became_peered":"2024-09-21T06:34:32.645072+0000","last_unstale":"2024-09-21T06:36:05.834518+0000","last_undegraded":"2024-09-21T06:36:05.834518+0000","last_fullsized":"2024-09-21T06:36:05.834518+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:54.109246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622307+0000","last_change":"2024-09-21T06:34:39.716947+0000","last_active":"2024-09-21T06:36:05.622307+0000","last_peered":"2024-09-21T06:36:05.622307+0000","last_clean":"2024-09-21T06:36:05.622307+0000","last_became_active":"2024-09-21T06:34:39.716540+0000","last_became_peered":"2024-09-21T06:34:39.716540+0000","last_unstale":"2024-09-21T06:36:05.622307+0000","last_undegraded":"2024-09-21T06:36:05.622307+0000","last_fullsized":"2024-09-21T06:36:05.622307+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:12.748444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834526+0000","last_change":"2024-09-21T06:34:30.636389+0000","last_active":"2024-09-21T06:36:05.834526+0000","last_peered":"2024-09-21T06:36:05.834526+0000","last_clean":"2024-09-21T06:36:05.834526+0000","last_became_active":"2024-09-21T06:34:30.636008+0000","last_became_peered":"2024-09-21T06:34:30.636008+0000","last_unstale":"2024-09-21T06:36:05.834526+0000","last_undegraded":"2024-09-21T06:36:05.834526+0000","last_fullsized":"2024-09-21T06:36:05.834526+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:45.476267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.625201+0000","last_change":"2024-09-21T06:34:32.635644+0000","last_active":"2024-09-21T06:36:05.625201+0000","last_peered":"2024-09-21T06:36:05.625201+0000","last_clean":"2024-09-21T06:36:05.625201+0000","last_became_active":"2024-09-21T06:34:32.635253+0000","last_became_peered":"2024-09-21T06:34:32.635253+0000","last_unstale":"2024-09-21T06:36:05.625201+0000","last_undegraded":"2024-09-21T06:36:05.625201+0000","last_fullsized":"2024-09-21T06:36:05.625201+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:34.713656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622327+0000","last_change":"2024-09-21T06:34:39.710382+0000","last_active":"2024-09-21T06:36:05.622327+0000","last_peered":"2024-09-21T06:36:05.622327+0000","last_clean":"2024-09-21T06:36:05.622327+0000","last_became_active":"2024-09-21T06:34:39.709884+0000","last_became_peered":"2024-09-21T06:34:39.709884+0000","last_unstale":"2024-09-21T06:36:05.622327+0000","last_undegraded":"2024-09-21T06:36:05.622327+0000","last_fullsized":"2024-09-21T06:36:05.622327+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:01.602366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.623967+0000","last_change":"2024-09-21T06:34:32.640025+0000","last_active":"2024-09-21T06:36:05.623967+0000","last_peered":"2024-09-21T06:36:05.623967+0000","last_clean":"2024-09-21T06:36:05.623967+0000","last_became_active":"2024-09-21T06:34:32.639648+0000","last_became_peered":"2024-09-21T06:34:32.639648+0000","last_unstale":"2024-09-21T06:36:05.623967+0000","last_undegraded":"2024-09-21T06:36:05.623967+0000","last_fullsized":"2024-09-21T06:36:05.623967+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:05.833065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622241+0000","last_change":"2024-09-21T06:35:05.115344+0000","last_active":"2024-09-21T06:36:05.622241+0000","last_peered":"2024-09-21T06:36:05.622241+0000","last_clean":"2024-09-21T06:36:05.622241+0000","last_became_active":"2024-09-21T06:35:05.114678+0000","last_became_peered":"2024-09-21T06:35:05.114678+0000","last_unstale":"2024-09-21T06:36:05.622241+0000","last_undegraded":"2024-09-21T06:36:05.622241+0000","last_fullsized":"2024-09-21T06:36:05.622241+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:24.593145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621860+0000","last_change":"2024-09-21T06:35:05.115301+0000","last_active":"2024-09-21T06:36:05.621860+0000","last_peered":"2024-09-21T06:36:05.621860+0000","last_clean":"2024-09-21T06:36:05.621860+0000","last_became_active":"2024-09-21T06:35:05.114706+0000","last_became_peered":"2024-09-21T06:35:05.114706+0000","last_unstale":"2024-09-21T06:36:05.621860+0000","last_undegraded":"2024-09-21T06:36:05.621860+0000","last_fullsized":"2024-09-21T06:36:05.621860+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:09.513594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.628918+0000","last_change":"2024-09-21T06:34:32.634679+0000","last_active":"2024-09-21T06:36:05.628918+0000","last_peered":"2024-09-21T06:36:05.628918+0000","last_clean":"2024-09-21T06:36:05.628918+0000","last_became_active":"2024-09-21T06:34:32.634285+0000","last_became_peered":"2024-09-21T06:34:32.634285+0000","last_unstale":"2024-09-21T06:36:05.628918+0000","last_undegraded":"2024-09-21T06:36:05.628918+0000","last_fullsized":"2024-09-21T06:36:05.628918+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:15.185453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834373+0000","last_change":"2024-09-21T06:34:32.639037+0000","last_active":"2024-09-21T06:36:05.834373+0000","last_peered":"2024-09-21T06:36:05.834373+0000","last_clean":"2024-09-21T06:36:05.834373+0000","last_became_active":"2024-09-21T06:34:32.638689+0000","last_became_peered":"2024-09-21T06:34:32.638689+0000","last_unstale":"2024-09-21T06:36:05.834373+0000","last_undegraded":"2024-09-21T06:36:05.834373+0000","last_fullsized":"2024-09-21T06:36:05.834373+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:13.609191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.625323+0000","last_change":"2024-09-21T06:35:05.115051+0000","last_active":"2024-09-21T06:36:05.625323+0000","last_peered":"2024-09-21T06:36:05.625323+0000","last_clean":"2024-09-21T06:36:05.625323+0000","last_became_active":"2024-09-21T06:35:05.114602+0000","last_became_peered":"2024-09-21T06:35:05.114602+0000","last_unstale":"2024-09-21T06:36:05.625323+0000","last_undegraded":"2024-09-21T06:36:05.625323+0000","last_fullsized":"2024-09-21T06:36:05.625323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:48.778188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622758+0000","last_change":"2024-09-21T06:34:32.637237+0000","last_active":"2024-09-21T06:36:05.622758+0000","last_peered":"2024-09-21T06:36:05.622758+0000","last_clean":"2024-09-21T06:36:05.622758+0000","last_became_active":"2024-09-21T06:34:32.634105+0000","last_became_peered":"2024-09-21T06:34:32.634105+0000","last_unstale":"2024-09-21T06:36:05.622758+0000","last_undegraded":"2024-09-21T06:36:05.622758+0000","last_fullsized":"2024-09-21T06:36:05.622758+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:52.530461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835100+0000","last_change":"2024-09-21T06:34:32.624441+0000","last_active":"2024-09-21T06:36:05.835100+0000","last_peered":"2024-09-21T06:36:05.835100+0000","last_clean":"2024-09-21T06:36:05.835100+0000","last_became_active":"2024-09-21T06:34:32.623974+0000","last_became_peered":"2024-09-21T06:34:32.623974+0000","last_unstale":"2024-09-21T06:36:05.835100+0000","last_undegraded":"2024-09-21T06:36:05.835100+0000","last_fullsized":"2024-09-21T06:36:05.835100+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:12.378113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834314+0000","last_change":"2024-09-21T06:34:32.633879+0000","last_active":"2024-09-21T06:36:05.834314+0000","last_peered":"2024-09-21T06:36:05.834314+0000","last_clean":"2024-09-21T06:36:05.834314+0000","last_became_active":"2024-09-21T06:34:32.631791+0000","last_became_peered":"2024-09-21T06:34:32.631791+0000","last_unstale":"2024-09-21T06:36:05.834314+0000","last_undegraded":"2024-09-21T06:36:05.834314+0000","last_fullsized":"2024-09-21T06:36:05.834314+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:46.246892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834708+0000","last_change":"2024-09-21T06:34:32.639974+0000","last_active":"2024-09-21T06:36:05.834708+0000","last_peered":"2024-09-21T06:36:05.834708+0000","last_clean":"2024-09-21T06:36:05.834708+0000","last_became_active":"2024-09-21T06:34:32.639631+0000","last_became_peered":"2024-09-21T06:34:32.639631+0000","last_unstale":"2024-09-21T06:36:05.834708+0000","last_undegraded":"2024-09-21T06:36:05.834708+0000","last_fullsized":"2024-09-21T06:36:05.834708+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:50.069378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835236+0000","last_change":"2024-09-21T06:34:32.645403+0000","last_active":"2024-09-21T06:36:05.835236+0000","last_peered":"2024-09-21T06:36:05.835236+0000","last_clean":"2024-09-21T06:36:05.835236+0000","last_became_active":"2024-09-21T06:34:32.644986+0000","last_became_peered":"2024-09-21T06:34:32.644986+0000","last_unstale":"2024-09-21T06:36:05.835236+0000","last_undegraded":"2024-09-21T06:36:05.835236+0000","last_fullsized":"2024-09-21T06:36:05.835236+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:43.640107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834233+0000","last_change":"2024-09-21T06:34:32.639933+0000","last_active":"2024-09-21T06:36:05.834233+0000","last_peered":"2024-09-21T06:36:05.834233+0000","last_clean":"2024-09-21T06:36:05.834233+0000","last_became_active":"2024-09-21T06:34:32.639438+0000","last_became_peered":"2024-09-21T06:34:32.639438+0000","last_unstale":"2024-09-21T06:36:05.834233+0000","last_undegraded":"2024-09-21T06:36:05.834233+0000","last_fullsized":"2024-09-21T06:36:05.834233+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:18.860219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624647+0000","last_change":"2024-09-21T06:34:32.648580+0000","last_active":"2024-09-21T06:36:05.624647+0000","last_peered":"2024-09-21T06:36:05.624647+0000","last_clean":"2024-09-21T06:36:05.624647+0000","last_became_active":"2024-09-21T06:34:32.648016+0000","last_became_peered":"2024-09-21T06:34:32.648016+0000","last_unstale":"2024-09-21T06:36:05.624647+0000","last_undegraded":"2024-09-21T06:36:05.624647+0000","last_fullsized":"2024-09-21T06:36:05.624647+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:52.858069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622477+0000","last_change":"2024-09-21T06:34:32.633376+0000","last_active":"2024-09-21T06:36:05.622477+0000","last_peered":"2024-09-21T06:36:05.622477+0000","last_clean":"2024-09-21T06:36:05.622477+0000","last_became_active":"2024-09-21T06:34:32.631052+0000","last_became_peered":"2024-09-21T06:34:32.631052+0000","last_unstale":"2024-09-21T06:36:05.622477+0000","last_undegraded":"2024-09-21T06:36:05.622477+0000","last_fullsized":"2024-09-21T06:36:05.622477+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:46.229599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621638+0000","last_change":"2024-09-21T06:34:32.636966+0000","last_active":"2024-09-21T06:36:05.621638+0000","last_peered":"2024-09-21T06:36:05.621638+0000","last_clean":"2024-09-21T06:36:05.621638+0000","last_became_active":"2024-09-21T06:34:32.636565+0000","last_became_peered":"2024-09-21T06:34:32.636565+0000","last_unstale":"2024-09-21T06:36:05.621638+0000","last_undegraded":"2024-09-21T06:36:05.621638+0000","last_fullsized":"2024-09-21T06:36:05.621638+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:36.233082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.625201+0000","last_change":"2024-09-21T06:34:32.646895+0000","last_active":"2024-09-21T06:36:05.625201+0000","last_peered":"2024-09-21T06:36:05.625201+0000","last_clean":"2024-09-21T06:36:05.625201+0000","last_became_active":"2024-09-21T06:34:32.646505+0000","last_became_peered":"2024-09-21T06:34:32.646505+0000","last_unstale":"2024-09-21T06:36:05.625201+0000","last_undegraded":"2024-09-21T06:36:05.625201+0000","last_fullsized":"2024-09-21T06:36:05.625201+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:43.441337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835545+0000","last_change":"2024-09-21T06:34:32.645402+0000","last_active":"2024-09-21T06:36:05.835545+0000","last_peered":"2024-09-21T06:36:05.835545+0000","last_clean":"2024-09-21T06:36:05.835545+0000","last_became_active":"2024-09-21T06:34:32.645013+0000","last_became_peered":"2024-09-21T06:34:32.645013+0000","last_unstale":"2024-09-21T06:36:05.835545+0000","last_undegraded":"2024-09-21T06:36:05.835545+0000","last_fullsized":"2024-09-21T06:36:05.835545+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:33.475847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.628586+0000","last_change":"2024-09-21T06:34:32.643234+0000","last_active":"2024-09-21T06:36:05.628586+0000","last_peered":"2024-09-21T06:36:05.628586+0000","last_clean":"2024-09-21T06:36:05.628586+0000","last_became_active":"2024-09-21T06:34:32.642614+0000","last_became_peered":"2024-09-21T06:34:32.642614+0000","last_unstale":"2024-09-21T06:36:05.628586+0000","last_undegraded":"2024-09-21T06:36:05.628586+0000","last_fullsized":"2024-09-21T06:36:05.628586+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:50.903500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.628594+0000","last_change":"2024-09-21T06:34:32.641887+0000","last_active":"2024-09-21T06:36:05.628594+0000","last_peered":"2024-09-21T06:36:05.628594+0000","last_clean":"2024-09-21T06:36:05.628594+0000","last_became_active":"2024-09-21T06:34:32.641490+0000","last_became_peered":"2024-09-21T06:34:32.641490+0000","last_unstale":"2024-09-21T06:36:05.628594+0000","last_undegraded":"2024-09-21T06:36:05.628594+0000","last_fullsized":"2024-09-21T06:36:05.628594+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:34.550970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834244+0000","last_change":"2024-09-21T06:34:32.641865+0000","last_active":"2024-09-21T06:36:05.834244+0000","last_peered":"2024-09-21T06:36:05.834244+0000","last_clean":"2024-09-21T06:36:05.834244+0000","last_became_active":"2024-09-21T06:34:32.641252+0000","last_became_peered":"2024-09-21T06:34:32.641252+0000","last_unstale":"2024-09-21T06:36:05.834244+0000","last_undegraded":"2024-09-21T06:36:05.834244+0000","last_fullsized":"2024-09-21T06:36:05.834244+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:44.459202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.772096+0000","last_change":"2024-09-21T06:34:32.641259+0000","last_active":"2024-09-21T06:36:05.772096+0000","last_peered":"2024-09-21T06:36:05.772096+0000","last_clean":"2024-09-21T06:36:05.772096+0000","last_became_active":"2024-09-21T06:34:32.640907+0000","last_became_peered":"2024-09-21T06:34:32.640907+0000","last_unstale":"2024-09-21T06:36:05.772096+0000","last_undegraded":"2024-09-21T06:36:05.772096+0000","last_fullsized":"2024-09-21T06:36:05.772096+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:59.966857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622720+0000","last_change":"2024-09-21T06:34:32.646095+0000","last_active":"2024-09-21T06:36:05.622720+0000","last_peered":"2024-09-21T06:36:05.622720+0000","last_clean":"2024-09-21T06:36:05.622720+0000","last_became_active":"2024-09-21T06:34:32.645604+0000","last_became_peered":"2024-09-21T06:34:32.645604+0000","last_unstale":"2024-09-21T06:36:05.622720+0000","last_undegraded":"2024-09-21T06:36:05.622720+0000","last_fullsized":"2024-09-21T06:36:05.622720+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:30.872428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835310+0000","last_change":"2024-09-21T06:34:32.649303+0000","last_active":"2024-09-21T06:36:05.835310+0000","last_peered":"2024-09-21T06:36:05.835310+0000","last_clean":"2024-09-21T06:36:05.835310+0000","last_became_active":"2024-09-21T06:34:32.648965+0000","last_became_peered":"2024-09-21T06:34:32.648965+0000","last_unstale":"2024-09-21T06:36:05.835310+0000","last_undegraded":"2024-09-21T06:36:05.835310+0000","last_fullsized":"2024-09-21T06:36:05.835310+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:51.335949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834838+0000","last_change":"2024-09-21T06:34:32.637261+0000","last_active":"2024-09-21T06:36:05.834838+0000","last_peered":"2024-09-21T06:36:05.834838+0000","last_clean":"2024-09-21T06:36:05.834838+0000","last_became_active":"2024-09-21T06:34:32.636861+0000","last_became_peered":"2024-09-21T06:34:32.636861+0000","last_unstale":"2024-09-21T06:36:05.834838+0000","last_undegraded":"2024-09-21T06:36:05.834838+0000","last_fullsized":"2024-09-21T06:36:05.834838+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:55.271027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834070+0000","last_change":"2024-09-21T06:34:32.635156+0000","last_active":"2024-09-21T06:36:05.834070+0000","last_peered":"2024-09-21T06:36:05.834070+0000","last_clean":"2024-09-21T06:36:05.834070+0000","last_became_active":"2024-09-21T06:34:32.634802+0000","last_became_peered":"2024-09-21T06:34:32.634802+0000","last_unstale":"2024-09-21T06:36:05.834070+0000","last_undegraded":"2024-09-21T06:36:05.834070+0000","last_fullsized":"2024-09-21T06:36:05.834070+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:16.139829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622299+0000","last_change":"2024-09-21T06:34:32.637509+0000","last_active":"2024-09-21T06:36:05.622299+0000","last_peered":"2024-09-21T06:36:05.622299+0000","last_clean":"2024-09-21T06:36:05.622299+0000","last_became_active":"2024-09-21T06:34:32.637136+0000","last_became_peered":"2024-09-21T06:34:32.637136+0000","last_unstale":"2024-09-21T06:36:05.622299+0000","last_undegraded":"2024-09-21T06:36:05.622299+0000","last_fullsized":"2024-09-21T06:36:05.622299+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:48.088988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834524+0000","last_change":"2024-09-21T06:35:05.114819+0000","last_active":"2024-09-21T06:36:05.834524+0000","last_peered":"2024-09-21T06:36:05.834524+0000","last_clean":"2024-09-21T06:36:05.834524+0000","last_became_active":"2024-09-21T06:35:05.114424+0000","last_became_peered":"2024-09-21T06:35:05.114424+0000","last_unstale":"2024-09-21T06:36:05.834524+0000","last_undegraded":"2024-09-21T06:36:05.834524+0000","last_fullsized":"2024-09-21T06:36:05.834524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:59.323643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624873+0000","last_change":"2024-09-21T06:34:32.636103+0000","last_active":"2024-09-21T06:36:05.624873+0000","last_peered":"2024-09-21T06:36:05.624873+0000","last_clean":"2024-09-21T06:36:05.624873+0000","last_became_active":"2024-09-21T06:34:32.635728+0000","last_became_peered":"2024-09-21T06:34:32.635728+0000","last_unstale":"2024-09-21T06:36:05.624873+0000","last_undegraded":"2024-09-21T06:36:05.624873+0000","last_fullsized":"2024-09-21T06:36:05.624873+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:16.731402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.772051+0000","last_change":"2024-09-21T06:34:32.635654+0000","last_active":"2024-09-21T06:36:05.772051+0000","last_peered":"2024-09-21T06:36:05.772051+0000","last_clean":"2024-09-21T06:36:05.772051+0000","last_became_active":"2024-09-21T06:34:32.635265+0000","last_became_peered":"2024-09-21T06:34:32.635265+0000","last_unstale":"2024-09-21T06:36:05.772051+0000","last_undegraded":"2024-09-21T06:36:05.772051+0000","last_fullsized":"2024-09-21T06:36:05.772051+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:28.169625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.628625+0000","last_change":"2024-09-21T06:34:32.644640+0000","last_active":"2024-09-21T06:36:05.628625+0000","last_peered":"2024-09-21T06:36:05.628625+0000","last_clean":"2024-09-21T06:36:05.628625+0000","last_became_active":"2024-09-21T06:34:32.644334+0000","last_became_peered":"2024-09-21T06:34:32.644334+0000","last_unstale":"2024-09-21T06:36:05.628625+0000","last_undegraded":"2024-09-21T06:36:05.628625+0000","last_fullsized":"2024-09-21T06:36:05.628625+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:25.572178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.629082+0000","last_change":"2024-09-21T06:34:32.643705+0000","last_active":"2024-09-21T06:36:05.629082+0000","last_peered":"2024-09-21T06:36:05.629082+0000","last_clean":"2024-09-21T06:36:05.629082+0000","last_became_active":"2024-09-21T06:34:32.642705+0000","last_became_peered":"2024-09-21T06:34:32.642705+0000","last_unstale":"2024-09-21T06:36:05.629082+0000","last_undegraded":"2024-09-21T06:36:05.629082+0000","last_fullsized":"2024-09-21T06:36:05.629082+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:22.542559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624987+0000","last_change":"2024-09-21T06:34:32.646539+0000","last_active":"2024-09-21T06:36:05.624987+0000","last_peered":"2024-09-21T06:36:05.624987+0000","last_clean":"2024-09-21T06:36:05.624987+0000","last_became_active":"2024-09-21T06:34:32.645992+0000","last_became_peered":"2024-09-21T06:34:32.645992+0000","last_unstale":"2024-09-21T06:36:05.624987+0000","last_undegraded":"2024-09-21T06:36:05.624987+0000","last_fullsized":"2024-09-21T06:36:05.624987+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:38.856001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69583,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":137,"num_read_kb":138,"num_write":85,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":208749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71,"ondisk_log_size":71,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282016,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":929792,"data_stored":485699,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939686,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1064960,"data_stored":1140950,"data_compressed":65460,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103212,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":937984,"data_stored":490377,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299441,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":937984,"data_stored":486303,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495669,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":987136,"data_stored":1080745,"data_compressed":65460,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659195,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":925696,"data_stored":481603,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855424,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":991232,"data_stored":1076249,"data_compressed":65460,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018950,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":933888,"data_stored":485721,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215178,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":933888,"data_stored":481707,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1007616,"data_stored":550422,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1003520,"data_stored":550388,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":929792,"data_stored":481693,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":68819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:36:13.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:12 smithi203 ceph-mon[33735]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T06:36:13.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:12 smithi203 ceph-mon[33735]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:36:13.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:12 smithi067 ceph-mon[31232]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T06:36:13.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:12 smithi067 ceph-mon[31232]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:36:13.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:12 smithi100 ceph-mon[33401]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T06:36:13.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:12 smithi100 ceph-mon[33401]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:36:15.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:14 smithi067 ceph-mon[31232]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 885 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T06:36:15.306 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:14 smithi100 ceph-mon[33401]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 885 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T06:36:15.486 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:14 smithi203 ceph-mon[33735]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 885 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T06:36:15.865 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T06:36:16.145 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:36:16.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.669+0000 7fb81b43c640 1 -- 172.21.15.67:0/2605983430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb814105300 msgr2=0x7fb81410d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:16.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.669+0000 7fb81b43c640 1 --2- 172.21.15.67:0/2605983430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb814105300 0x7fb81410d850 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb804009900 tx=0x7fb80402a410 comp rx=0 tx=0).stop 2024-09-21T06:36:16.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.669+0000 7fb81b43c640 1 -- 172.21.15.67:0/2605983430 shutdown_connections 2024-09-21T06:36:16.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.669+0000 7fb81b43c640 1 --2- 172.21.15.67:0/2605983430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb81410de10 0x7fb814110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:16.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.669+0000 7fb81b43c640 1 --2- 172.21.15.67:0/2605983430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb814105300 0x7fb81410d850 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:16.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.669+0000 7fb81b43c640 1 --2- 172.21.15.67:0/2605983430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8141049b0 0x7fb814104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:16.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.669+0000 7fb81b43c640 1 -- 172.21.15.67:0/2605983430 >> 172.21.15.67:0/2605983430 conn(0x7fb814079730 msgr2=0x7fb814079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:16.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.670+0000 7fb81b43c640 1 -- 172.21.15.67:0/2605983430 shutdown_connections 2024-09-21T06:36:16.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.670+0000 7fb81b43c640 1 -- 172.21.15.67:0/2605983430 wait complete. 2024-09-21T06:36:16.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.670+0000 7fb81b43c640 1 Processor -- start 2024-09-21T06:36:16.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.670+0000 7fb81b43c640 1 -- start start 2024-09-21T06:36:16.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.671+0000 7fb81b43c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8141049b0 0x7fb81419a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:16.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.671+0000 7fb81b43c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb814105300 0x7fb81419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:16.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.671+0000 7fb81b43c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb81410de10 0x7fb81419fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:16.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.671+0000 7fb81b43c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb814113920 con 0x7fb814105300 2024-09-21T06:36:16.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.671+0000 7fb81b43c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb8141137a0 con 0x7fb8141049b0 2024-09-21T06:36:16.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.671+0000 7fb81b43c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb814113aa0 con 0x7fb81410de10 2024-09-21T06:36:16.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.671+0000 7fb8189d7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb814105300 0x7fb81419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:16.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.671+0000 7fb8189d7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb814105300 0x7fb81419a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44676/0 (socket says 172.21.15.67:44676) 2024-09-21T06:36:16.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.671+0000 7fb8189d7640 1 -- 172.21.15.67:0/3659343487 learned_addr learned my addr 172.21.15.67:0/3659343487 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:16.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.671+0000 7fb8199d9640 1 --2- 172.21.15.67:0/3659343487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb81410de10 0x7fb81419fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:16.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.671+0000 7fb8191d8640 1 --2- 172.21.15.67:0/3659343487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8141049b0 0x7fb81419a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:16.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.671+0000 7fb8189d7640 1 -- 172.21.15.67:0/3659343487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb81410de10 msgr2=0x7fb81419fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:16.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.671+0000 7fb8189d7640 1 --2- 172.21.15.67:0/3659343487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb81410de10 0x7fb81419fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:16.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.671+0000 7fb8189d7640 1 -- 172.21.15.67:0/3659343487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8141049b0 msgr2=0x7fb81419a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:16.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.671+0000 7fb8189d7640 1 --2- 172.21.15.67:0/3659343487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8141049b0 0x7fb81419a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:16.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.671+0000 7fb8189d7640 1 -- 172.21.15.67:0/3659343487 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8141a0340 con 0x7fb814105300 2024-09-21T06:36:16.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.671+0000 7fb8199d9640 1 --2- 172.21.15.67:0/3659343487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb81410de10 0x7fb81419fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:36:16.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.671+0000 7fb8191d8640 1 --2- 172.21.15.67:0/3659343487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8141049b0 0x7fb81419a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:36:16.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.671+0000 7fb8189d7640 1 --2- 172.21.15.67:0/3659343487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb814105300 0x7fb81419a970 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb804004770 tx=0x7fb804004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:16.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.671+0000 7fb80a7fc640 1 -- 172.21.15.67:0/3659343487 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80403f030 con 0x7fb814105300 2024-09-21T06:36:16.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.671+0000 7fb81b43c640 1 -- 172.21.15.67:0/3659343487 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8141a05d0 con 0x7fb814105300 2024-09-21T06:36:16.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.672+0000 7fb80a7fc640 1 -- 172.21.15.67:0/3659343487 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb804004440 con 0x7fb814105300 2024-09-21T06:36:16.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.672+0000 7fb80a7fc640 1 -- 172.21.15.67:0/3659343487 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb804035070 con 0x7fb814105300 2024-09-21T06:36:16.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.672+0000 7fb81b43c640 1 -- 172.21.15.67:0/3659343487 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8141a6d90 con 0x7fb814105300 2024-09-21T06:36:16.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.673+0000 7fb81b43c640 1 -- 172.21.15.67:0/3659343487 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb814104dc0 con 0x7fb814105300 2024-09-21T06:36:16.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.675+0000 7fb80a7fc640 1 -- 172.21.15.67:0/3659343487 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fb804042030 con 0x7fb814105300 2024-09-21T06:36:16.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.676+0000 7fb80a7fc640 1 --2- 172.21.15.67:0/3659343487 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb8000b65a0 0x7fb8000b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:16.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.676+0000 7fb8191d8640 1 --2- 172.21.15.67:0/3659343487 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb8000b65a0 0x7fb8000b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:16.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.676+0000 7fb80a7fc640 1 -- 172.21.15.67:0/3659343487 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fb804108100 con 0x7fb814105300 2024-09-21T06:36:16.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.677+0000 7fb8191d8640 1 --2- 172.21.15.67:0/3659343487 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb8000b65a0 0x7fb8000b8ab0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb8100058c0 tx=0x7fb810005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:16.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.679+0000 7fb80a7fc640 1 -- 172.21.15.67:0/3659343487 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb8040ce460 con 0x7fb814105300 2024-09-21T06:36:16.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.867+0000 7fb81b43c640 1 -- 172.21.15.67:0/3659343487 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb814108f40 con 0x7fb8000b65a0 2024-09-21T06:36:16.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.872+0000 7fb80a7fc640 1 -- 172.21.15.67:0/3659343487 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340534 (secure 0 0 0) 0x7fb814108f40 con 0x7fb8000b65a0 2024-09-21T06:36:16.875 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:36:16.880 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-09-21T06:36:16.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.876+0000 7fb81b43c640 1 -- 172.21.15.67:0/3659343487 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb8000b65a0 msgr2=0x7fb8000b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:16.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.876+0000 7fb81b43c640 1 --2- 172.21.15.67:0/3659343487 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb8000b65a0 0x7fb8000b8ab0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb8100058c0 tx=0x7fb810005850 comp rx=0 tx=0).stop 2024-09-21T06:36:16.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.876+0000 7fb81b43c640 1 -- 172.21.15.67:0/3659343487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb814105300 msgr2=0x7fb81419a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:16.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.876+0000 7fb81b43c640 1 --2- 172.21.15.67:0/3659343487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb814105300 0x7fb81419a970 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb804004770 tx=0x7fb804004040 comp rx=0 tx=0).stop 2024-09-21T06:36:16.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.876+0000 7fb81b43c640 1 -- 172.21.15.67:0/3659343487 shutdown_connections 2024-09-21T06:36:16.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.876+0000 7fb81b43c640 1 --2- 172.21.15.67:0/3659343487 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb8000b65a0 0x7fb8000b8ab0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:16.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.876+0000 7fb81b43c640 1 --2- 172.21.15.67:0/3659343487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb81410de10 0x7fb81419fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:16.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.876+0000 7fb81b43c640 1 --2- 172.21.15.67:0/3659343487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb814105300 0x7fb81419a970 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:16.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.876+0000 7fb81b43c640 1 --2- 172.21.15.67:0/3659343487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8141049b0 0x7fb81419a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:16.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.876+0000 7fb81b43c640 1 -- 172.21.15.67:0/3659343487 >> 172.21.15.67:0/3659343487 conn(0x7fb814079730 msgr2=0x7fb814100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:16.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.876+0000 7fb81b43c640 1 -- 172.21.15.67:0/3659343487 shutdown_connections 2024-09-21T06:36:16.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:16.876+0000 7fb81b43c640 1 -- 172.21.15.67:0/3659343487 wait complete. 2024-09-21T06:36:17.240 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:16 smithi067 ceph-mon[31232]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 799 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T06:36:17.284 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:16 smithi203 ceph-mon[33735]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 799 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T06:36:17.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:16 smithi100 ceph-mon[33401]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 799 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T06:36:17.731 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":351,"stamp":"2024-09-21T06:36:15.871082+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660185,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":223,"num_read_kb":207,"num_write":210,"num_write_kb":2248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":165,"ondisk_log_size":165,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364804,"kb_used_data":11312,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577020,"statfs":{"total":1151940427776,"available":1151566868480,"internally_reserved":0,"allocated":11583488,"data_stored":7791857,"data_compressed":462771,"data_compressed_allocated":626688,"data_compressed_original":2752512,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":8,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.251473"},"pg_stats":[{"pgid":"2.1f","version":"75'4","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621781+0000","last_change":"2024-09-21T06:34:30.633366+0000","last_active":"2024-09-21T06:36:05.621781+0000","last_peered":"2024-09-21T06:36:05.621781+0000","last_clean":"2024-09-21T06:36:05.621781+0000","last_became_active":"2024-09-21T06:34:30.632929+0000","last_became_peered":"2024-09-21T06:34:30.632929+0000","last_unstale":"2024-09-21T06:36:05.621781+0000","last_undegraded":"2024-09-21T06:36:05.621781+0000","last_fullsized":"2024-09-21T06:36:05.621781+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:47.722461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834970+0000","last_change":"2024-09-21T06:34:32.632989+0000","last_active":"2024-09-21T06:36:05.834970+0000","last_peered":"2024-09-21T06:36:05.834970+0000","last_clean":"2024-09-21T06:36:05.834970+0000","last_became_active":"2024-09-21T06:34:32.632024+0000","last_became_peered":"2024-09-21T06:34:32.632024+0000","last_unstale":"2024-09-21T06:36:05.834970+0000","last_undegraded":"2024-09-21T06:36:05.834970+0000","last_fullsized":"2024-09-21T06:36:05.834970+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:13.669613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621767+0000","last_change":"2024-09-21T06:35:05.274434+0000","last_active":"2024-09-21T06:36:05.621767+0000","last_peered":"2024-09-21T06:36:05.621767+0000","last_clean":"2024-09-21T06:36:05.621767+0000","last_became_active":"2024-09-21T06:35:05.273833+0000","last_became_peered":"2024-09-21T06:35:05.273833+0000","last_unstale":"2024-09-21T06:36:05.621767+0000","last_undegraded":"2024-09-21T06:36:05.621767+0000","last_fullsized":"2024-09-21T06:36:05.621767+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:48.847861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835182+0000","last_change":"2024-09-21T06:34:30.946309+0000","last_active":"2024-09-21T06:36:05.835182+0000","last_peered":"2024-09-21T06:36:05.835182+0000","last_clean":"2024-09-21T06:36:05.835182+0000","last_became_active":"2024-09-21T06:34:30.945913+0000","last_became_peered":"2024-09-21T06:34:30.945913+0000","last_unstale":"2024-09-21T06:36:05.835182+0000","last_undegraded":"2024-09-21T06:36:05.835182+0000","last_fullsized":"2024-09-21T06:36:05.835182+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:21.676524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.629308+0000","last_change":"2024-09-21T06:34:32.637636+0000","last_active":"2024-09-21T06:36:05.629308+0000","last_peered":"2024-09-21T06:36:05.629308+0000","last_clean":"2024-09-21T06:36:05.629308+0000","last_became_active":"2024-09-21T06:34:32.636534+0000","last_became_peered":"2024-09-21T06:34:32.636534+0000","last_unstale":"2024-09-21T06:36:05.629308+0000","last_undegraded":"2024-09-21T06:36:05.629308+0000","last_fullsized":"2024-09-21T06:36:05.629308+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:05.802118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771498+0000","last_change":"2024-09-21T06:35:05.274419+0000","last_active":"2024-09-21T06:36:05.771498+0000","last_peered":"2024-09-21T06:36:05.771498+0000","last_clean":"2024-09-21T06:36:05.771498+0000","last_became_active":"2024-09-21T06:35:05.273865+0000","last_became_peered":"2024-09-21T06:35:05.273865+0000","last_unstale":"2024-09-21T06:36:05.771498+0000","last_undegraded":"2024-09-21T06:36:05.771498+0000","last_fullsized":"2024-09-21T06:36:05.771498+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:04.490355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.631709+0000","last_change":"2024-09-21T06:36:05.631098+0000","last_active":"2024-09-21T06:36:05.631709+0000","last_peered":"2024-09-21T06:36:05.631709+0000","last_clean":"2024-09-21T06:36:05.631709+0000","last_became_active":"2024-09-21T06:36:05.627521+0000","last_became_peered":"2024-09-21T06:36:05.627521+0000","last_unstale":"2024-09-21T06:36:05.631709+0000","last_undegraded":"2024-09-21T06:36:05.631709+0000","last_fullsized":"2024-09-21T06:36:05.631709+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:02.467758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834863+0000","last_change":"2024-09-21T06:34:32.650303+0000","last_active":"2024-09-21T06:36:05.834863+0000","last_peered":"2024-09-21T06:36:05.834863+0000","last_clean":"2024-09-21T06:36:05.834863+0000","last_became_active":"2024-09-21T06:34:32.647189+0000","last_became_peered":"2024-09-21T06:34:32.647189+0000","last_unstale":"2024-09-21T06:36:05.834863+0000","last_undegraded":"2024-09-21T06:36:05.834863+0000","last_fullsized":"2024-09-21T06:36:05.834863+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:14.889384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771714+0000","last_change":"2024-09-21T06:34:39.713139+0000","last_active":"2024-09-21T06:36:05.771714+0000","last_peered":"2024-09-21T06:36:05.771714+0000","last_clean":"2024-09-21T06:36:05.771714+0000","last_became_active":"2024-09-21T06:34:39.712379+0000","last_became_peered":"2024-09-21T06:34:39.712379+0000","last_unstale":"2024-09-21T06:36:05.771714+0000","last_undegraded":"2024-09-21T06:36:05.771714+0000","last_fullsized":"2024-09-21T06:36:05.771714+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:53.100529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.625272+0000","last_change":"2024-09-21T06:34:30.633850+0000","last_active":"2024-09-21T06:36:05.625272+0000","last_peered":"2024-09-21T06:36:05.625272+0000","last_clean":"2024-09-21T06:36:05.625272+0000","last_became_active":"2024-09-21T06:34:30.633468+0000","last_became_peered":"2024-09-21T06:34:30.633468+0000","last_unstale":"2024-09-21T06:36:05.625272+0000","last_undegraded":"2024-09-21T06:36:05.625272+0000","last_fullsized":"2024-09-21T06:36:05.625272+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:59.095803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834771+0000","last_change":"2024-09-21T06:35:05.005240+0000","last_active":"2024-09-21T06:36:05.834771+0000","last_peered":"2024-09-21T06:36:05.834771+0000","last_clean":"2024-09-21T06:36:05.834771+0000","last_became_active":"2024-09-21T06:35:05.004849+0000","last_became_peered":"2024-09-21T06:35:05.004849+0000","last_unstale":"2024-09-21T06:36:05.834771+0000","last_undegraded":"2024-09-21T06:36:05.834771+0000","last_fullsized":"2024-09-21T06:36:05.834771+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:28.849604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771556+0000","last_change":"2024-09-21T06:35:05.274437+0000","last_active":"2024-09-21T06:36:05.771556+0000","last_peered":"2024-09-21T06:36:05.771556+0000","last_clean":"2024-09-21T06:36:05.771556+0000","last_became_active":"2024-09-21T06:35:05.273865+0000","last_became_peered":"2024-09-21T06:35:05.273865+0000","last_unstale":"2024-09-21T06:36:05.771556+0000","last_undegraded":"2024-09-21T06:36:05.771556+0000","last_fullsized":"2024-09-21T06:36:05.771556+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:43.339504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622649+0000","last_change":"2024-09-21T06:34:32.645630+0000","last_active":"2024-09-21T06:36:05.622649+0000","last_peered":"2024-09-21T06:36:05.622649+0000","last_clean":"2024-09-21T06:36:05.622649+0000","last_became_active":"2024-09-21T06:34:32.645237+0000","last_became_peered":"2024-09-21T06:34:32.645237+0000","last_unstale":"2024-09-21T06:36:05.622649+0000","last_undegraded":"2024-09-21T06:36:05.622649+0000","last_fullsized":"2024-09-21T06:36:05.622649+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:25.064318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.633983+0000","last_change":"2024-09-21T06:36:05.633484+0000","last_active":"2024-09-21T06:36:05.633983+0000","last_peered":"2024-09-21T06:36:05.633983+0000","last_clean":"2024-09-21T06:36:05.633983+0000","last_became_active":"2024-09-21T06:36:05.627637+0000","last_became_peered":"2024-09-21T06:36:05.627637+0000","last_unstale":"2024-09-21T06:36:05.633983+0000","last_undegraded":"2024-09-21T06:36:05.633983+0000","last_fullsized":"2024-09-21T06:36:05.633983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:50:00.626538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771936+0000","last_change":"2024-09-21T06:34:39.714177+0000","last_active":"2024-09-21T06:36:05.771936+0000","last_peered":"2024-09-21T06:36:05.771936+0000","last_clean":"2024-09-21T06:36:05.771936+0000","last_became_active":"2024-09-21T06:34:39.713713+0000","last_became_peered":"2024-09-21T06:34:39.713713+0000","last_unstale":"2024-09-21T06:36:05.771936+0000","last_undegraded":"2024-09-21T06:36:05.771936+0000","last_fullsized":"2024-09-21T06:36:05.771936+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:07.042240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834413+0000","last_change":"2024-09-21T06:34:32.636297+0000","last_active":"2024-09-21T06:36:05.834413+0000","last_peered":"2024-09-21T06:36:05.834413+0000","last_clean":"2024-09-21T06:36:05.834413+0000","last_became_active":"2024-09-21T06:34:32.635930+0000","last_became_peered":"2024-09-21T06:34:32.635930+0000","last_unstale":"2024-09-21T06:36:05.834413+0000","last_undegraded":"2024-09-21T06:36:05.834413+0000","last_fullsized":"2024-09-21T06:36:05.834413+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:43.091682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621887+0000","last_change":"2024-09-21T06:34:30.631405+0000","last_active":"2024-09-21T06:36:05.621887+0000","last_peered":"2024-09-21T06:36:05.621887+0000","last_clean":"2024-09-21T06:36:05.621887+0000","last_became_active":"2024-09-21T06:34:30.630512+0000","last_became_peered":"2024-09-21T06:34:30.630512+0000","last_unstale":"2024-09-21T06:36:05.621887+0000","last_undegraded":"2024-09-21T06:36:05.621887+0000","last_fullsized":"2024-09-21T06:36:05.621887+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:43.412695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"83'1","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771897+0000","last_change":"2024-09-21T06:34:39.710573+0000","last_active":"2024-09-21T06:36:05.771897+0000","last_peered":"2024-09-21T06:36:05.771897+0000","last_clean":"2024-09-21T06:36:05.771897+0000","last_became_active":"2024-09-21T06:34:39.708969+0000","last_became_peered":"2024-09-21T06:34:39.708969+0000","last_unstale":"2024-09-21T06:36:05.771897+0000","last_undegraded":"2024-09-21T06:36:05.771897+0000","last_fullsized":"2024-09-21T06:36:05.771897+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:34.847587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624019+0000","last_change":"2024-09-21T06:34:32.641085+0000","last_active":"2024-09-21T06:36:05.624019+0000","last_peered":"2024-09-21T06:36:05.624019+0000","last_clean":"2024-09-21T06:36:05.624019+0000","last_became_active":"2024-09-21T06:34:32.640723+0000","last_became_peered":"2024-09-21T06:34:32.640723+0000","last_unstale":"2024-09-21T06:36:05.624019+0000","last_undegraded":"2024-09-21T06:36:05.624019+0000","last_fullsized":"2024-09-21T06:36:05.624019+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:21.410262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834897+0000","last_change":"2024-09-21T06:34:30.626719+0000","last_active":"2024-09-21T06:36:05.834897+0000","last_peered":"2024-09-21T06:36:05.834897+0000","last_clean":"2024-09-21T06:36:05.834897+0000","last_became_active":"2024-09-21T06:34:30.626312+0000","last_became_peered":"2024-09-21T06:34:30.626312+0000","last_unstale":"2024-09-21T06:36:05.834897+0000","last_undegraded":"2024-09-21T06:36:05.834897+0000","last_fullsized":"2024-09-21T06:36:05.834897+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:53.938914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621617+0000","last_change":"2024-09-21T06:34:39.711172+0000","last_active":"2024-09-21T06:36:05.621617+0000","last_peered":"2024-09-21T06:36:05.621617+0000","last_clean":"2024-09-21T06:36:05.621617+0000","last_became_active":"2024-09-21T06:34:39.708424+0000","last_became_peered":"2024-09-21T06:34:39.708424+0000","last_unstale":"2024-09-21T06:36:05.621617+0000","last_undegraded":"2024-09-21T06:36:05.621617+0000","last_fullsized":"2024-09-21T06:36:05.621617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:43.918355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624648+0000","last_change":"2024-09-21T06:34:32.641463+0000","last_active":"2024-09-21T06:36:05.624648+0000","last_peered":"2024-09-21T06:36:05.624648+0000","last_clean":"2024-09-21T06:36:05.624648+0000","last_became_active":"2024-09-21T06:34:32.641086+0000","last_became_peered":"2024-09-21T06:34:32.641086+0000","last_unstale":"2024-09-21T06:36:05.624648+0000","last_undegraded":"2024-09-21T06:36:05.624648+0000","last_fullsized":"2024-09-21T06:36:05.624648+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:58.991374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835670+0000","last_change":"2024-09-21T06:34:30.946698+0000","last_active":"2024-09-21T06:36:05.835670+0000","last_peered":"2024-09-21T06:36:05.835670+0000","last_clean":"2024-09-21T06:36:05.835670+0000","last_became_active":"2024-09-21T06:34:30.946325+0000","last_became_peered":"2024-09-21T06:34:30.946325+0000","last_unstale":"2024-09-21T06:36:05.835670+0000","last_undegraded":"2024-09-21T06:36:05.835670+0000","last_fullsized":"2024-09-21T06:36:05.835670+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:22.639265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.772211+0000","last_change":"2024-09-21T06:34:39.712429+0000","last_active":"2024-09-21T06:36:05.772211+0000","last_peered":"2024-09-21T06:36:05.772211+0000","last_clean":"2024-09-21T06:36:05.772211+0000","last_became_active":"2024-09-21T06:34:39.711823+0000","last_became_peered":"2024-09-21T06:34:39.711823+0000","last_unstale":"2024-09-21T06:36:05.772211+0000","last_undegraded":"2024-09-21T06:36:05.772211+0000","last_fullsized":"2024-09-21T06:36:05.772211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:03:49.513991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.629145+0000","last_change":"2024-09-21T06:35:05.274326+0000","last_active":"2024-09-21T06:36:05.629145+0000","last_peered":"2024-09-21T06:36:05.629145+0000","last_clean":"2024-09-21T06:36:05.629145+0000","last_became_active":"2024-09-21T06:35:05.273884+0000","last_became_peered":"2024-09-21T06:35:05.273884+0000","last_unstale":"2024-09-21T06:36:05.629145+0000","last_undegraded":"2024-09-21T06:36:05.629145+0000","last_fullsized":"2024-09-21T06:36:05.629145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:03.219765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.772344+0000","last_change":"2024-09-21T06:34:30.676794+0000","last_active":"2024-09-21T06:36:05.772344+0000","last_peered":"2024-09-21T06:36:05.772344+0000","last_clean":"2024-09-21T06:36:05.772344+0000","last_became_active":"2024-09-21T06:34:30.676400+0000","last_became_peered":"2024-09-21T06:34:30.676400+0000","last_unstale":"2024-09-21T06:36:05.772344+0000","last_undegraded":"2024-09-21T06:36:05.772344+0000","last_fullsized":"2024-09-21T06:36:05.772344+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:23.681363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622082+0000","last_change":"2024-09-21T06:34:39.708980+0000","last_active":"2024-09-21T06:36:05.622082+0000","last_peered":"2024-09-21T06:36:05.622082+0000","last_clean":"2024-09-21T06:36:05.622082+0000","last_became_active":"2024-09-21T06:34:39.707748+0000","last_became_peered":"2024-09-21T06:34:39.707748+0000","last_unstale":"2024-09-21T06:36:05.622082+0000","last_undegraded":"2024-09-21T06:36:05.622082+0000","last_fullsized":"2024-09-21T06:36:05.622082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:16.812226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835129+0000","last_change":"2024-09-21T06:34:32.638337+0000","last_active":"2024-09-21T06:36:05.835129+0000","last_peered":"2024-09-21T06:36:05.835129+0000","last_clean":"2024-09-21T06:36:05.835129+0000","last_became_active":"2024-09-21T06:34:32.638013+0000","last_became_peered":"2024-09-21T06:34:32.638013+0000","last_unstale":"2024-09-21T06:36:05.835129+0000","last_undegraded":"2024-09-21T06:36:05.835129+0000","last_fullsized":"2024-09-21T06:36:05.835129+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:26.321190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"83'9","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621666+0000","last_change":"2024-09-21T06:34:30.631086+0000","last_active":"2024-09-21T06:36:05.621666+0000","last_peered":"2024-09-21T06:36:05.621666+0000","last_clean":"2024-09-21T06:36:05.621666+0000","last_became_active":"2024-09-21T06:34:30.630706+0000","last_became_peered":"2024-09-21T06:34:30.630706+0000","last_unstale":"2024-09-21T06:36:05.621666+0000","last_undegraded":"2024-09-21T06:36:05.621666+0000","last_fullsized":"2024-09-21T06:36:05.621666+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:30.260359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.833935+0000","last_change":"2024-09-21T06:34:39.709097+0000","last_active":"2024-09-21T06:36:05.833935+0000","last_peered":"2024-09-21T06:36:05.833935+0000","last_clean":"2024-09-21T06:36:05.833935+0000","last_became_active":"2024-09-21T06:34:39.708609+0000","last_became_peered":"2024-09-21T06:34:39.708609+0000","last_unstale":"2024-09-21T06:36:05.833935+0000","last_undegraded":"2024-09-21T06:36:05.833935+0000","last_fullsized":"2024-09-21T06:36:05.833935+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:48.842174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834498+0000","last_change":"2024-09-21T06:34:32.636816+0000","last_active":"2024-09-21T06:36:05.834498+0000","last_peered":"2024-09-21T06:36:05.834498+0000","last_clean":"2024-09-21T06:36:05.834498+0000","last_became_active":"2024-09-21T06:34:32.636429+0000","last_became_peered":"2024-09-21T06:34:32.636429+0000","last_unstale":"2024-09-21T06:36:05.834498+0000","last_undegraded":"2024-09-21T06:36:05.834498+0000","last_fullsized":"2024-09-21T06:36:05.834498+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:49.010491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.625566+0000","last_change":"2024-09-21T06:34:30.947481+0000","last_active":"2024-09-21T06:36:05.625566+0000","last_peered":"2024-09-21T06:36:05.625566+0000","last_clean":"2024-09-21T06:36:05.625566+0000","last_became_active":"2024-09-21T06:34:30.947040+0000","last_became_peered":"2024-09-21T06:34:30.947040+0000","last_unstale":"2024-09-21T06:36:05.625566+0000","last_undegraded":"2024-09-21T06:36:05.625566+0000","last_fullsized":"2024-09-21T06:36:05.625566+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:52.813476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624899+0000","last_change":"2024-09-21T06:34:39.708674+0000","last_active":"2024-09-21T06:36:05.624899+0000","last_peered":"2024-09-21T06:36:05.624899+0000","last_clean":"2024-09-21T06:36:05.624899+0000","last_became_active":"2024-09-21T06:34:39.708176+0000","last_became_peered":"2024-09-21T06:34:39.708176+0000","last_unstale":"2024-09-21T06:36:05.624899+0000","last_undegraded":"2024-09-21T06:36:05.624899+0000","last_fullsized":"2024-09-21T06:36:05.624899+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:57.070792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834505+0000","last_change":"2024-09-21T06:34:32.636381+0000","last_active":"2024-09-21T06:36:05.834505+0000","last_peered":"2024-09-21T06:36:05.834505+0000","last_clean":"2024-09-21T06:36:05.834505+0000","last_became_active":"2024-09-21T06:34:32.635937+0000","last_became_peered":"2024-09-21T06:34:32.635937+0000","last_unstale":"2024-09-21T06:36:05.834505+0000","last_undegraded":"2024-09-21T06:36:05.834505+0000","last_fullsized":"2024-09-21T06:36:05.834505+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:52.870676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621464+0000","last_change":"2024-09-21T06:34:30.947160+0000","last_active":"2024-09-21T06:36:05.621464+0000","last_peered":"2024-09-21T06:36:05.621464+0000","last_clean":"2024-09-21T06:36:05.621464+0000","last_became_active":"2024-09-21T06:34:30.946736+0000","last_became_peered":"2024-09-21T06:34:30.946736+0000","last_unstale":"2024-09-21T06:36:05.621464+0000","last_undegraded":"2024-09-21T06:36:05.621464+0000","last_fullsized":"2024-09-21T06:36:05.621464+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:47.425481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624815+0000","last_change":"2024-09-21T06:35:05.006032+0000","last_active":"2024-09-21T06:36:05.624815+0000","last_peered":"2024-09-21T06:36:05.624815+0000","last_clean":"2024-09-21T06:36:05.624815+0000","last_became_active":"2024-09-21T06:35:05.005429+0000","last_became_peered":"2024-09-21T06:35:05.005429+0000","last_unstale":"2024-09-21T06:36:05.624815+0000","last_undegraded":"2024-09-21T06:36:05.624815+0000","last_fullsized":"2024-09-21T06:36:05.624815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:04.138097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834598+0000","last_change":"2024-09-21T06:34:32.648256+0000","last_active":"2024-09-21T06:36:05.834598+0000","last_peered":"2024-09-21T06:36:05.834598+0000","last_clean":"2024-09-21T06:36:05.834598+0000","last_became_active":"2024-09-21T06:34:32.647881+0000","last_became_peered":"2024-09-21T06:34:32.647881+0000","last_unstale":"2024-09-21T06:36:05.834598+0000","last_undegraded":"2024-09-21T06:36:05.834598+0000","last_fullsized":"2024-09-21T06:36:05.834598+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:14.590834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622843+0000","last_change":"2024-09-21T06:34:30.629918+0000","last_active":"2024-09-21T06:36:05.622843+0000","last_peered":"2024-09-21T06:36:05.622843+0000","last_clean":"2024-09-21T06:36:05.622843+0000","last_became_active":"2024-09-21T06:34:30.628785+0000","last_became_peered":"2024-09-21T06:34:30.628785+0000","last_unstale":"2024-09-21T06:36:05.622843+0000","last_undegraded":"2024-09-21T06:36:05.622843+0000","last_fullsized":"2024-09-21T06:36:05.622843+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:05.333406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834369+0000","last_change":"2024-09-21T06:34:39.710073+0000","last_active":"2024-09-21T06:36:05.834369+0000","last_peered":"2024-09-21T06:36:05.834369+0000","last_clean":"2024-09-21T06:36:05.834369+0000","last_became_active":"2024-09-21T06:34:39.708986+0000","last_became_peered":"2024-09-21T06:34:39.708986+0000","last_unstale":"2024-09-21T06:36:05.834369+0000","last_undegraded":"2024-09-21T06:36:05.834369+0000","last_fullsized":"2024-09-21T06:36:05.834369+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:46.379902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834487+0000","last_change":"2024-09-21T06:34:32.644224+0000","last_active":"2024-09-21T06:36:05.834487+0000","last_peered":"2024-09-21T06:36:05.834487+0000","last_clean":"2024-09-21T06:36:05.834487+0000","last_became_active":"2024-09-21T06:34:32.643850+0000","last_became_peered":"2024-09-21T06:34:32.643850+0000","last_unstale":"2024-09-21T06:36:05.834487+0000","last_undegraded":"2024-09-21T06:36:05.834487+0000","last_fullsized":"2024-09-21T06:36:05.834487+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:24:33.188537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.833892+0000","last_change":"2024-09-21T06:34:30.946334+0000","last_active":"2024-09-21T06:36:05.833892+0000","last_peered":"2024-09-21T06:36:05.833892+0000","last_clean":"2024-09-21T06:36:05.833892+0000","last_became_active":"2024-09-21T06:34:30.945908+0000","last_became_peered":"2024-09-21T06:34:30.945908+0000","last_unstale":"2024-09-21T06:36:05.833892+0000","last_undegraded":"2024-09-21T06:36:05.833892+0000","last_fullsized":"2024-09-21T06:36:05.833892+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:46.746527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834586+0000","last_change":"2024-09-21T06:34:39.707909+0000","last_active":"2024-09-21T06:36:05.834586+0000","last_peered":"2024-09-21T06:36:05.834586+0000","last_clean":"2024-09-21T06:36:05.834586+0000","last_became_active":"2024-09-21T06:34:39.707492+0000","last_became_peered":"2024-09-21T06:34:39.707492+0000","last_unstale":"2024-09-21T06:36:05.834586+0000","last_undegraded":"2024-09-21T06:36:05.834586+0000","last_fullsized":"2024-09-21T06:36:05.834586+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:56.201212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.833745+0000","last_change":"2024-09-21T06:34:32.635870+0000","last_active":"2024-09-21T06:36:05.833745+0000","last_peered":"2024-09-21T06:36:05.833745+0000","last_clean":"2024-09-21T06:36:05.833745+0000","last_became_active":"2024-09-21T06:34:32.635252+0000","last_became_peered":"2024-09-21T06:34:32.635252+0000","last_unstale":"2024-09-21T06:36:05.833745+0000","last_undegraded":"2024-09-21T06:36:05.833745+0000","last_fullsized":"2024-09-21T06:36:05.833745+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:10.613033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622266+0000","last_change":"2024-09-21T06:34:30.634360+0000","last_active":"2024-09-21T06:36:05.622266+0000","last_peered":"2024-09-21T06:36:05.622266+0000","last_clean":"2024-09-21T06:36:05.622266+0000","last_became_active":"2024-09-21T06:34:30.633845+0000","last_became_peered":"2024-09-21T06:34:30.633845+0000","last_unstale":"2024-09-21T06:36:05.622266+0000","last_undegraded":"2024-09-21T06:36:05.622266+0000","last_fullsized":"2024-09-21T06:36:05.622266+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:34:53.388670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834837+0000","last_change":"2024-09-21T06:34:39.719620+0000","last_active":"2024-09-21T06:36:05.834837+0000","last_peered":"2024-09-21T06:36:05.834837+0000","last_clean":"2024-09-21T06:36:05.834837+0000","last_became_active":"2024-09-21T06:34:39.718786+0000","last_became_peered":"2024-09-21T06:34:39.718786+0000","last_unstale":"2024-09-21T06:36:05.834837+0000","last_undegraded":"2024-09-21T06:36:05.834837+0000","last_fullsized":"2024-09-21T06:36:05.834837+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:43.524932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834613+0000","last_change":"2024-09-21T06:34:32.634970+0000","last_active":"2024-09-21T06:36:05.834613+0000","last_peered":"2024-09-21T06:36:05.834613+0000","last_clean":"2024-09-21T06:36:05.834613+0000","last_became_active":"2024-09-21T06:34:32.634611+0000","last_became_peered":"2024-09-21T06:34:32.634611+0000","last_unstale":"2024-09-21T06:36:05.834613+0000","last_undegraded":"2024-09-21T06:36:05.834613+0000","last_fullsized":"2024-09-21T06:36:05.834613+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:37.855849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.772108+0000","last_change":"2024-09-21T06:34:30.628123+0000","last_active":"2024-09-21T06:36:05.772108+0000","last_peered":"2024-09-21T06:36:05.772108+0000","last_clean":"2024-09-21T06:36:05.772108+0000","last_became_active":"2024-09-21T06:34:30.627498+0000","last_became_peered":"2024-09-21T06:34:30.627498+0000","last_unstale":"2024-09-21T06:36:05.772108+0000","last_undegraded":"2024-09-21T06:36:05.772108+0000","last_fullsized":"2024-09-21T06:36:05.772108+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:42.576441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":115,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622503+0000","last_change":"2024-09-21T06:33:16.856623+0000","last_active":"2024-09-21T06:36:05.622503+0000","last_peered":"2024-09-21T06:36:05.622503+0000","last_clean":"2024-09-21T06:36:05.622503+0000","last_became_active":"2024-09-21T06:33:16.850787+0000","last_became_peered":"2024-09-21T06:33:16.850787+0000","last_unstale":"2024-09-21T06:36:05.622503+0000","last_undegraded":"2024-09-21T06:36:05.622503+0000","last_fullsized":"2024-09-21T06:36:05.622503+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:50.283252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:50.283252+0000","last_clean_scrub_stamp":"2024-09-21T06:29:50.283252+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:29:20.711208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622480+0000","last_change":"2024-09-21T06:34:39.706472+0000","last_active":"2024-09-21T06:36:05.622480+0000","last_peered":"2024-09-21T06:36:05.622480+0000","last_clean":"2024-09-21T06:36:05.622480+0000","last_became_active":"2024-09-21T06:34:39.705724+0000","last_became_peered":"2024-09-21T06:34:39.705724+0000","last_unstale":"2024-09-21T06:36:05.622480+0000","last_undegraded":"2024-09-21T06:36:05.622480+0000","last_fullsized":"2024-09-21T06:36:05.622480+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:43.238704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622792+0000","last_change":"2024-09-21T06:34:32.645944+0000","last_active":"2024-09-21T06:36:05.622792+0000","last_peered":"2024-09-21T06:36:05.622792+0000","last_clean":"2024-09-21T06:36:05.622792+0000","last_became_active":"2024-09-21T06:34:32.645134+0000","last_became_peered":"2024-09-21T06:34:32.645134+0000","last_unstale":"2024-09-21T06:36:05.622792+0000","last_undegraded":"2024-09-21T06:36:05.622792+0000","last_fullsized":"2024-09-21T06:36:05.622792+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:45.254164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622567+0000","last_change":"2024-09-21T06:34:30.946296+0000","last_active":"2024-09-21T06:36:05.622567+0000","last_peered":"2024-09-21T06:36:05.622567+0000","last_clean":"2024-09-21T06:36:05.622567+0000","last_became_active":"2024-09-21T06:34:30.945777+0000","last_became_peered":"2024-09-21T06:34:30.945777+0000","last_unstale":"2024-09-21T06:36:05.622567+0000","last_undegraded":"2024-09-21T06:36:05.622567+0000","last_fullsized":"2024-09-21T06:36:05.622567+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:32:25.988512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835077+0000","last_change":"2024-09-21T06:34:39.705876+0000","last_active":"2024-09-21T06:36:05.835077+0000","last_peered":"2024-09-21T06:36:05.835077+0000","last_clean":"2024-09-21T06:36:05.835077+0000","last_became_active":"2024-09-21T06:34:39.705132+0000","last_became_peered":"2024-09-21T06:34:39.705132+0000","last_unstale":"2024-09-21T06:36:05.835077+0000","last_undegraded":"2024-09-21T06:36:05.835077+0000","last_fullsized":"2024-09-21T06:36:05.835077+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:18.180244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622740+0000","last_change":"2024-09-21T06:34:32.638242+0000","last_active":"2024-09-21T06:36:05.622740+0000","last_peered":"2024-09-21T06:36:05.622740+0000","last_clean":"2024-09-21T06:36:05.622740+0000","last_became_active":"2024-09-21T06:34:32.637816+0000","last_became_peered":"2024-09-21T06:34:32.637816+0000","last_unstale":"2024-09-21T06:36:05.622740+0000","last_undegraded":"2024-09-21T06:36:05.622740+0000","last_fullsized":"2024-09-21T06:36:05.622740+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:40.926147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834922+0000","last_change":"2024-09-21T06:34:30.632173+0000","last_active":"2024-09-21T06:36:05.834922+0000","last_peered":"2024-09-21T06:36:05.834922+0000","last_clean":"2024-09-21T06:36:05.834922+0000","last_became_active":"2024-09-21T06:34:30.631638+0000","last_became_peered":"2024-09-21T06:34:30.631638+0000","last_unstale":"2024-09-21T06:36:05.834922+0000","last_undegraded":"2024-09-21T06:36:05.834922+0000","last_fullsized":"2024-09-21T06:36:05.834922+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:29.763272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.628935+0000","last_change":"2024-09-21T06:34:39.714620+0000","last_active":"2024-09-21T06:36:05.628935+0000","last_peered":"2024-09-21T06:36:05.628935+0000","last_clean":"2024-09-21T06:36:05.628935+0000","last_became_active":"2024-09-21T06:34:39.714199+0000","last_became_peered":"2024-09-21T06:34:39.714199+0000","last_unstale":"2024-09-21T06:36:05.628935+0000","last_undegraded":"2024-09-21T06:36:05.628935+0000","last_fullsized":"2024-09-21T06:36:05.628935+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:37.041249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622722+0000","last_change":"2024-09-21T06:34:32.645001+0000","last_active":"2024-09-21T06:36:05.622722+0000","last_peered":"2024-09-21T06:36:05.622722+0000","last_clean":"2024-09-21T06:36:05.622722+0000","last_became_active":"2024-09-21T06:34:32.644584+0000","last_became_peered":"2024-09-21T06:34:32.644584+0000","last_unstale":"2024-09-21T06:36:05.622722+0000","last_undegraded":"2024-09-21T06:36:05.622722+0000","last_fullsized":"2024-09-21T06:36:05.622722+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:47.098850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834960+0000","last_change":"2024-09-21T06:34:30.632593+0000","last_active":"2024-09-21T06:36:05.834960+0000","last_peered":"2024-09-21T06:36:05.834960+0000","last_clean":"2024-09-21T06:36:05.834960+0000","last_became_active":"2024-09-21T06:34:30.632170+0000","last_became_peered":"2024-09-21T06:34:30.632170+0000","last_unstale":"2024-09-21T06:36:05.834960+0000","last_undegraded":"2024-09-21T06:36:05.834960+0000","last_fullsized":"2024-09-21T06:36:05.834960+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:56.170655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834120+0000","last_change":"2024-09-21T06:34:39.710172+0000","last_active":"2024-09-21T06:36:05.834120+0000","last_peered":"2024-09-21T06:36:05.834120+0000","last_clean":"2024-09-21T06:36:05.834120+0000","last_became_active":"2024-09-21T06:34:39.708876+0000","last_became_peered":"2024-09-21T06:34:39.708876+0000","last_unstale":"2024-09-21T06:36:05.834120+0000","last_undegraded":"2024-09-21T06:36:05.834120+0000","last_fullsized":"2024-09-21T06:36:05.834120+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:32.371873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624520+0000","last_change":"2024-09-21T06:34:32.651056+0000","last_active":"2024-09-21T06:36:05.624520+0000","last_peered":"2024-09-21T06:36:05.624520+0000","last_clean":"2024-09-21T06:36:05.624520+0000","last_became_active":"2024-09-21T06:34:32.650676+0000","last_became_peered":"2024-09-21T06:34:32.650676+0000","last_unstale":"2024-09-21T06:36:05.624520+0000","last_undegraded":"2024-09-21T06:36:05.624520+0000","last_fullsized":"2024-09-21T06:36:05.624520+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:34.599654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.625649+0000","last_change":"2024-09-21T06:34:30.632307+0000","last_active":"2024-09-21T06:36:05.625649+0000","last_peered":"2024-09-21T06:36:05.625649+0000","last_clean":"2024-09-21T06:36:05.625649+0000","last_became_active":"2024-09-21T06:34:30.631920+0000","last_became_peered":"2024-09-21T06:34:30.631920+0000","last_unstale":"2024-09-21T06:36:05.625649+0000","last_undegraded":"2024-09-21T06:36:05.625649+0000","last_fullsized":"2024-09-21T06:36:05.625649+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:46.940898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834319+0000","last_change":"2024-09-21T06:34:39.715056+0000","last_active":"2024-09-21T06:36:05.834319+0000","last_peered":"2024-09-21T06:36:05.834319+0000","last_clean":"2024-09-21T06:36:05.834319+0000","last_became_active":"2024-09-21T06:34:39.714563+0000","last_became_peered":"2024-09-21T06:34:39.714563+0000","last_unstale":"2024-09-21T06:36:05.834319+0000","last_undegraded":"2024-09-21T06:36:05.834319+0000","last_fullsized":"2024-09-21T06:36:05.834319+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:48.567057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834369+0000","last_change":"2024-09-21T06:34:32.646934+0000","last_active":"2024-09-21T06:36:05.834369+0000","last_peered":"2024-09-21T06:36:05.834369+0000","last_clean":"2024-09-21T06:36:05.834369+0000","last_became_active":"2024-09-21T06:34:32.646559+0000","last_became_peered":"2024-09-21T06:34:32.646559+0000","last_unstale":"2024-09-21T06:36:05.834369+0000","last_undegraded":"2024-09-21T06:36:05.834369+0000","last_fullsized":"2024-09-21T06:36:05.834369+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:37.802590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":64,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.843100+0000","last_change":"2024-09-21T06:36:05.843018+0000","last_active":"2024-09-21T06:36:05.843100+0000","last_peered":"2024-09-21T06:36:05.843100+0000","last_clean":"2024-09-21T06:36:05.843100+0000","last_became_active":"2024-09-21T06:36:05.836284+0000","last_became_peered":"2024-09-21T06:36:05.836284+0000","last_unstale":"2024-09-21T06:36:05.843100+0000","last_undegraded":"2024-09-21T06:36:05.843100+0000","last_fullsized":"2024-09-21T06:36:05.843100+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:51.539804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834952+0000","last_change":"2024-09-21T06:34:39.713398+0000","last_active":"2024-09-21T06:36:05.834952+0000","last_peered":"2024-09-21T06:36:05.834952+0000","last_clean":"2024-09-21T06:36:05.834952+0000","last_became_active":"2024-09-21T06:34:39.712954+0000","last_became_peered":"2024-09-21T06:34:39.712954+0000","last_unstale":"2024-09-21T06:36:05.834952+0000","last_undegraded":"2024-09-21T06:36:05.834952+0000","last_fullsized":"2024-09-21T06:36:05.834952+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:58.881014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622265+0000","last_change":"2024-09-21T06:34:32.640879+0000","last_active":"2024-09-21T06:36:05.622265+0000","last_peered":"2024-09-21T06:36:05.622265+0000","last_clean":"2024-09-21T06:36:05.622265+0000","last_became_active":"2024-09-21T06:34:32.640526+0000","last_became_peered":"2024-09-21T06:34:32.640526+0000","last_unstale":"2024-09-21T06:36:05.622265+0000","last_undegraded":"2024-09-21T06:36:05.622265+0000","last_fullsized":"2024-09-21T06:36:05.622265+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:12.327195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771895+0000","last_change":"2024-09-21T06:34:30.631449+0000","last_active":"2024-09-21T06:36:05.771895+0000","last_peered":"2024-09-21T06:36:05.771895+0000","last_clean":"2024-09-21T06:36:05.771895+0000","last_became_active":"2024-09-21T06:34:30.631081+0000","last_became_peered":"2024-09-21T06:34:30.631081+0000","last_unstale":"2024-09-21T06:36:05.771895+0000","last_undegraded":"2024-09-21T06:36:05.771895+0000","last_fullsized":"2024-09-21T06:36:05.771895+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:09.202696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621660+0000","last_change":"2024-09-21T06:34:39.707379+0000","last_active":"2024-09-21T06:36:05.621660+0000","last_peered":"2024-09-21T06:36:05.621660+0000","last_clean":"2024-09-21T06:36:05.621660+0000","last_became_active":"2024-09-21T06:34:39.706979+0000","last_became_peered":"2024-09-21T06:34:39.706979+0000","last_unstale":"2024-09-21T06:36:05.621660+0000","last_undegraded":"2024-09-21T06:36:05.621660+0000","last_fullsized":"2024-09-21T06:36:05.621660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:24.394233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622354+0000","last_change":"2024-09-21T06:34:32.649458+0000","last_active":"2024-09-21T06:36:05.622354+0000","last_peered":"2024-09-21T06:36:05.622354+0000","last_clean":"2024-09-21T06:36:05.622354+0000","last_became_active":"2024-09-21T06:34:32.649051+0000","last_became_peered":"2024-09-21T06:34:32.649051+0000","last_unstale":"2024-09-21T06:36:05.622354+0000","last_undegraded":"2024-09-21T06:36:05.622354+0000","last_fullsized":"2024-09-21T06:36:05.622354+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:12.631446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834455+0000","last_change":"2024-09-21T06:34:30.634592+0000","last_active":"2024-09-21T06:36:05.834455+0000","last_peered":"2024-09-21T06:36:05.834455+0000","last_clean":"2024-09-21T06:36:05.834455+0000","last_became_active":"2024-09-21T06:34:30.634251+0000","last_became_peered":"2024-09-21T06:34:30.634251+0000","last_unstale":"2024-09-21T06:36:05.834455+0000","last_undegraded":"2024-09-21T06:36:05.834455+0000","last_fullsized":"2024-09-21T06:36:05.834455+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:32.817467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624233+0000","last_change":"2024-09-21T06:34:39.719416+0000","last_active":"2024-09-21T06:36:05.624233+0000","last_peered":"2024-09-21T06:36:05.624233+0000","last_clean":"2024-09-21T06:36:05.624233+0000","last_became_active":"2024-09-21T06:34:39.718910+0000","last_became_peered":"2024-09-21T06:34:39.718910+0000","last_unstale":"2024-09-21T06:36:05.624233+0000","last_undegraded":"2024-09-21T06:36:05.624233+0000","last_fullsized":"2024-09-21T06:36:05.624233+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:41.413250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.833982+0000","last_change":"2024-09-21T06:34:32.640727+0000","last_active":"2024-09-21T06:36:05.833982+0000","last_peered":"2024-09-21T06:36:05.833982+0000","last_clean":"2024-09-21T06:36:05.833982+0000","last_became_active":"2024-09-21T06:34:32.640296+0000","last_became_peered":"2024-09-21T06:34:32.640296+0000","last_unstale":"2024-09-21T06:36:05.833982+0000","last_undegraded":"2024-09-21T06:36:05.833982+0000","last_fullsized":"2024-09-21T06:36:05.833982+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:35.124059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622118+0000","last_change":"2024-09-21T06:34:30.633332+0000","last_active":"2024-09-21T06:36:05.622118+0000","last_peered":"2024-09-21T06:36:05.622118+0000","last_clean":"2024-09-21T06:36:05.622118+0000","last_became_active":"2024-09-21T06:34:30.632641+0000","last_became_peered":"2024-09-21T06:34:30.632641+0000","last_unstale":"2024-09-21T06:36:05.622118+0000","last_undegraded":"2024-09-21T06:36:05.622118+0000","last_fullsized":"2024-09-21T06:36:05.622118+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:30:23.930313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834659+0000","last_change":"2024-09-21T06:34:39.712504+0000","last_active":"2024-09-21T06:36:05.834659+0000","last_peered":"2024-09-21T06:36:05.834659+0000","last_clean":"2024-09-21T06:36:05.834659+0000","last_became_active":"2024-09-21T06:34:39.711638+0000","last_became_peered":"2024-09-21T06:34:39.711638+0000","last_unstale":"2024-09-21T06:36:05.834659+0000","last_undegraded":"2024-09-21T06:36:05.834659+0000","last_fullsized":"2024-09-21T06:36:05.834659+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:26:09.003367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.625165+0000","last_change":"2024-09-21T06:34:32.639582+0000","last_active":"2024-09-21T06:36:05.625165+0000","last_peered":"2024-09-21T06:36:05.625165+0000","last_clean":"2024-09-21T06:36:05.625165+0000","last_became_active":"2024-09-21T06:34:32.639023+0000","last_became_peered":"2024-09-21T06:34:32.639023+0000","last_unstale":"2024-09-21T06:36:05.625165+0000","last_undegraded":"2024-09-21T06:36:05.625165+0000","last_fullsized":"2024-09-21T06:36:05.625165+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:26.773389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"83'30","reported_seq":212,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:15.144903+0000","last_change":"2024-09-21T06:34:30.626280+0000","last_active":"2024-09-21T06:36:15.144903+0000","last_peered":"2024-09-21T06:36:15.144903+0000","last_clean":"2024-09-21T06:36:15.144903+0000","last_became_active":"2024-09-21T06:34:30.625662+0000","last_became_peered":"2024-09-21T06:34:30.625662+0000","last_unstale":"2024-09-21T06:36:15.144903+0000","last_undegraded":"2024-09-21T06:36:15.144903+0000","last_fullsized":"2024-09-21T06:36:15.144903+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:10.541881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68785,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":143,"num_read_kb":139,"num_write":30,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621183+0000","last_change":"2024-09-21T06:34:39.708999+0000","last_active":"2024-09-21T06:36:05.621183+0000","last_peered":"2024-09-21T06:36:05.621183+0000","last_clean":"2024-09-21T06:36:05.621183+0000","last_became_active":"2024-09-21T06:34:39.708580+0000","last_became_peered":"2024-09-21T06:34:39.708580+0000","last_unstale":"2024-09-21T06:36:05.621183+0000","last_undegraded":"2024-09-21T06:36:05.621183+0000","last_fullsized":"2024-09-21T06:36:05.621183+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:07.648358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835047+0000","last_change":"2024-09-21T06:34:32.651157+0000","last_active":"2024-09-21T06:36:05.835047+0000","last_peered":"2024-09-21T06:36:05.835047+0000","last_clean":"2024-09-21T06:36:05.835047+0000","last_became_active":"2024-09-21T06:34:32.650804+0000","last_became_peered":"2024-09-21T06:34:32.650804+0000","last_unstale":"2024-09-21T06:36:05.835047+0000","last_undegraded":"2024-09-21T06:36:05.835047+0000","last_fullsized":"2024-09-21T06:36:05.835047+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:49.758496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834880+0000","last_change":"2024-09-21T06:34:30.947142+0000","last_active":"2024-09-21T06:36:05.834880+0000","last_peered":"2024-09-21T06:36:05.834880+0000","last_clean":"2024-09-21T06:36:05.834880+0000","last_became_active":"2024-09-21T06:34:30.946748+0000","last_became_peered":"2024-09-21T06:34:30.946748+0000","last_unstale":"2024-09-21T06:36:05.834880+0000","last_undegraded":"2024-09-21T06:36:05.834880+0000","last_fullsized":"2024-09-21T06:36:05.834880+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:39.222286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771885+0000","last_change":"2024-09-21T06:34:39.709266+0000","last_active":"2024-09-21T06:36:05.771885+0000","last_peered":"2024-09-21T06:36:05.771885+0000","last_clean":"2024-09-21T06:36:05.771885+0000","last_became_active":"2024-09-21T06:34:39.708753+0000","last_became_peered":"2024-09-21T06:34:39.708753+0000","last_unstale":"2024-09-21T06:36:05.771885+0000","last_undegraded":"2024-09-21T06:36:05.771885+0000","last_fullsized":"2024-09-21T06:36:05.771885+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:48.711681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.629321+0000","last_change":"2024-09-21T06:34:32.637976+0000","last_active":"2024-09-21T06:36:05.629321+0000","last_peered":"2024-09-21T06:36:05.629321+0000","last_clean":"2024-09-21T06:36:05.629321+0000","last_became_active":"2024-09-21T06:34:32.637267+0000","last_became_peered":"2024-09-21T06:34:32.637267+0000","last_unstale":"2024-09-21T06:36:05.629321+0000","last_undegraded":"2024-09-21T06:36:05.629321+0000","last_fullsized":"2024-09-21T06:36:05.629321+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:31.340583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.836482+0000","last_change":"2024-09-21T06:36:05.836382+0000","last_active":"2024-09-21T06:36:05.836482+0000","last_peered":"2024-09-21T06:36:05.836482+0000","last_clean":"2024-09-21T06:36:05.836482+0000","last_became_active":"2024-09-21T06:36:05.835918+0000","last_became_peered":"2024-09-21T06:36:05.835918+0000","last_unstale":"2024-09-21T06:36:05.836482+0000","last_undegraded":"2024-09-21T06:36:05.836482+0000","last_fullsized":"2024-09-21T06:36:05.836482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:47.346400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621023+0000","last_change":"2024-09-21T06:34:39.713831+0000","last_active":"2024-09-21T06:36:05.621023+0000","last_peered":"2024-09-21T06:36:05.621023+0000","last_clean":"2024-09-21T06:36:05.621023+0000","last_became_active":"2024-09-21T06:34:39.713403+0000","last_became_peered":"2024-09-21T06:34:39.713403+0000","last_unstale":"2024-09-21T06:36:05.621023+0000","last_undegraded":"2024-09-21T06:36:05.621023+0000","last_fullsized":"2024-09-21T06:36:05.621023+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:22.201063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834704+0000","last_change":"2024-09-21T06:34:32.646924+0000","last_active":"2024-09-21T06:36:05.834704+0000","last_peered":"2024-09-21T06:36:05.834704+0000","last_clean":"2024-09-21T06:36:05.834704+0000","last_became_active":"2024-09-21T06:34:32.646531+0000","last_became_peered":"2024-09-21T06:34:32.646531+0000","last_unstale":"2024-09-21T06:36:05.834704+0000","last_undegraded":"2024-09-21T06:36:05.834704+0000","last_fullsized":"2024-09-21T06:36:05.834704+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:48.550193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835325+0000","last_change":"2024-09-21T06:34:30.631901+0000","last_active":"2024-09-21T06:36:05.835325+0000","last_peered":"2024-09-21T06:36:05.835325+0000","last_clean":"2024-09-21T06:36:05.835325+0000","last_became_active":"2024-09-21T06:34:30.631450+0000","last_became_peered":"2024-09-21T06:34:30.631450+0000","last_unstale":"2024-09-21T06:36:05.835325+0000","last_undegraded":"2024-09-21T06:36:05.835325+0000","last_fullsized":"2024-09-21T06:36:05.835325+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:20.709456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834441+0000","last_change":"2024-09-21T06:34:39.711951+0000","last_active":"2024-09-21T06:36:05.834441+0000","last_peered":"2024-09-21T06:36:05.834441+0000","last_clean":"2024-09-21T06:36:05.834441+0000","last_became_active":"2024-09-21T06:34:39.710783+0000","last_became_peered":"2024-09-21T06:34:39.710783+0000","last_unstale":"2024-09-21T06:36:05.834441+0000","last_undegraded":"2024-09-21T06:36:05.834441+0000","last_fullsized":"2024-09-21T06:36:05.834441+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:27:58.501155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.629019+0000","last_change":"2024-09-21T06:34:39.709639+0000","last_active":"2024-09-21T06:36:05.629019+0000","last_peered":"2024-09-21T06:36:05.629019+0000","last_clean":"2024-09-21T06:36:05.629019+0000","last_became_active":"2024-09-21T06:34:39.709241+0000","last_became_peered":"2024-09-21T06:34:39.709241+0000","last_unstale":"2024-09-21T06:36:05.629019+0000","last_undegraded":"2024-09-21T06:36:05.629019+0000","last_fullsized":"2024-09-21T06:36:05.629019+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.357399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621131+0000","last_change":"2024-09-21T06:34:32.635365+0000","last_active":"2024-09-21T06:36:05.621131+0000","last_peered":"2024-09-21T06:36:05.621131+0000","last_clean":"2024-09-21T06:36:05.621131+0000","last_became_active":"2024-09-21T06:34:32.635012+0000","last_became_peered":"2024-09-21T06:34:32.635012+0000","last_unstale":"2024-09-21T06:36:05.621131+0000","last_undegraded":"2024-09-21T06:36:05.621131+0000","last_fullsized":"2024-09-21T06:36:05.621131+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:54.671074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834581+0000","last_change":"2024-09-21T06:34:30.633347+0000","last_active":"2024-09-21T06:36:05.834581+0000","last_peered":"2024-09-21T06:36:05.834581+0000","last_clean":"2024-09-21T06:36:05.834581+0000","last_became_active":"2024-09-21T06:34:30.632513+0000","last_became_peered":"2024-09-21T06:34:30.632513+0000","last_unstale":"2024-09-21T06:36:05.834581+0000","last_undegraded":"2024-09-21T06:36:05.834581+0000","last_fullsized":"2024-09-21T06:36:05.834581+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:12.439109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624773+0000","last_change":"2024-09-21T06:34:39.710204+0000","last_active":"2024-09-21T06:36:05.624773+0000","last_peered":"2024-09-21T06:36:05.624773+0000","last_clean":"2024-09-21T06:36:05.624773+0000","last_became_active":"2024-09-21T06:34:39.709744+0000","last_became_peered":"2024-09-21T06:34:39.709744+0000","last_unstale":"2024-09-21T06:36:05.624773+0000","last_undegraded":"2024-09-21T06:36:05.624773+0000","last_fullsized":"2024-09-21T06:36:05.624773+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:45.484088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834193+0000","last_change":"2024-09-21T06:34:32.634170+0000","last_active":"2024-09-21T06:36:05.834193+0000","last_peered":"2024-09-21T06:36:05.834193+0000","last_clean":"2024-09-21T06:36:05.834193+0000","last_became_active":"2024-09-21T06:34:32.633831+0000","last_became_peered":"2024-09-21T06:34:32.633831+0000","last_unstale":"2024-09-21T06:36:05.834193+0000","last_undegraded":"2024-09-21T06:36:05.834193+0000","last_fullsized":"2024-09-21T06:36:05.834193+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:36.738279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.833746+0000","last_change":"2024-09-21T06:34:30.629422+0000","last_active":"2024-09-21T06:36:05.833746+0000","last_peered":"2024-09-21T06:36:05.833746+0000","last_clean":"2024-09-21T06:36:05.833746+0000","last_became_active":"2024-09-21T06:34:30.629025+0000","last_became_peered":"2024-09-21T06:34:30.629025+0000","last_unstale":"2024-09-21T06:36:05.833746+0000","last_undegraded":"2024-09-21T06:36:05.833746+0000","last_fullsized":"2024-09-21T06:36:05.833746+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:05.068816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834561+0000","last_change":"2024-09-21T06:34:30.634562+0000","last_active":"2024-09-21T06:36:05.834561+0000","last_peered":"2024-09-21T06:36:05.834561+0000","last_clean":"2024-09-21T06:36:05.834561+0000","last_became_active":"2024-09-21T06:34:30.634170+0000","last_became_peered":"2024-09-21T06:34:30.634170+0000","last_unstale":"2024-09-21T06:36:05.834561+0000","last_undegraded":"2024-09-21T06:36:05.834561+0000","last_fullsized":"2024-09-21T06:36:05.834561+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:15.243103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834518+0000","last_change":"2024-09-21T06:34:32.645718+0000","last_active":"2024-09-21T06:36:05.834518+0000","last_peered":"2024-09-21T06:36:05.834518+0000","last_clean":"2024-09-21T06:36:05.834518+0000","last_became_active":"2024-09-21T06:34:32.645072+0000","last_became_peered":"2024-09-21T06:34:32.645072+0000","last_unstale":"2024-09-21T06:36:05.834518+0000","last_undegraded":"2024-09-21T06:36:05.834518+0000","last_fullsized":"2024-09-21T06:36:05.834518+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:54.109246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622307+0000","last_change":"2024-09-21T06:34:39.716947+0000","last_active":"2024-09-21T06:36:05.622307+0000","last_peered":"2024-09-21T06:36:05.622307+0000","last_clean":"2024-09-21T06:36:05.622307+0000","last_became_active":"2024-09-21T06:34:39.716540+0000","last_became_peered":"2024-09-21T06:34:39.716540+0000","last_unstale":"2024-09-21T06:36:05.622307+0000","last_undegraded":"2024-09-21T06:36:05.622307+0000","last_fullsized":"2024-09-21T06:36:05.622307+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:12.748444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834526+0000","last_change":"2024-09-21T06:34:30.636389+0000","last_active":"2024-09-21T06:36:05.834526+0000","last_peered":"2024-09-21T06:36:05.834526+0000","last_clean":"2024-09-21T06:36:05.834526+0000","last_became_active":"2024-09-21T06:34:30.636008+0000","last_became_peered":"2024-09-21T06:34:30.636008+0000","last_unstale":"2024-09-21T06:36:05.834526+0000","last_undegraded":"2024-09-21T06:36:05.834526+0000","last_fullsized":"2024-09-21T06:36:05.834526+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:45.476267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.625201+0000","last_change":"2024-09-21T06:34:32.635644+0000","last_active":"2024-09-21T06:36:05.625201+0000","last_peered":"2024-09-21T06:36:05.625201+0000","last_clean":"2024-09-21T06:36:05.625201+0000","last_became_active":"2024-09-21T06:34:32.635253+0000","last_became_peered":"2024-09-21T06:34:32.635253+0000","last_unstale":"2024-09-21T06:36:05.625201+0000","last_undegraded":"2024-09-21T06:36:05.625201+0000","last_fullsized":"2024-09-21T06:36:05.625201+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:34.713656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622327+0000","last_change":"2024-09-21T06:34:39.710382+0000","last_active":"2024-09-21T06:36:05.622327+0000","last_peered":"2024-09-21T06:36:05.622327+0000","last_clean":"2024-09-21T06:36:05.622327+0000","last_became_active":"2024-09-21T06:34:39.709884+0000","last_became_peered":"2024-09-21T06:34:39.709884+0000","last_unstale":"2024-09-21T06:36:05.622327+0000","last_undegraded":"2024-09-21T06:36:05.622327+0000","last_fullsized":"2024-09-21T06:36:05.622327+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:01.602366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.623967+0000","last_change":"2024-09-21T06:34:32.640025+0000","last_active":"2024-09-21T06:36:05.623967+0000","last_peered":"2024-09-21T06:36:05.623967+0000","last_clean":"2024-09-21T06:36:05.623967+0000","last_became_active":"2024-09-21T06:34:32.639648+0000","last_became_peered":"2024-09-21T06:34:32.639648+0000","last_unstale":"2024-09-21T06:36:05.623967+0000","last_undegraded":"2024-09-21T06:36:05.623967+0000","last_fullsized":"2024-09-21T06:36:05.623967+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:05.833065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622241+0000","last_change":"2024-09-21T06:35:05.115344+0000","last_active":"2024-09-21T06:36:05.622241+0000","last_peered":"2024-09-21T06:36:05.622241+0000","last_clean":"2024-09-21T06:36:05.622241+0000","last_became_active":"2024-09-21T06:35:05.114678+0000","last_became_peered":"2024-09-21T06:35:05.114678+0000","last_unstale":"2024-09-21T06:36:05.622241+0000","last_undegraded":"2024-09-21T06:36:05.622241+0000","last_fullsized":"2024-09-21T06:36:05.622241+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:24.593145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621860+0000","last_change":"2024-09-21T06:35:05.115301+0000","last_active":"2024-09-21T06:36:05.621860+0000","last_peered":"2024-09-21T06:36:05.621860+0000","last_clean":"2024-09-21T06:36:05.621860+0000","last_became_active":"2024-09-21T06:35:05.114706+0000","last_became_peered":"2024-09-21T06:35:05.114706+0000","last_unstale":"2024-09-21T06:36:05.621860+0000","last_undegraded":"2024-09-21T06:36:05.621860+0000","last_fullsized":"2024-09-21T06:36:05.621860+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:09.513594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.628918+0000","last_change":"2024-09-21T06:34:32.634679+0000","last_active":"2024-09-21T06:36:05.628918+0000","last_peered":"2024-09-21T06:36:05.628918+0000","last_clean":"2024-09-21T06:36:05.628918+0000","last_became_active":"2024-09-21T06:34:32.634285+0000","last_became_peered":"2024-09-21T06:34:32.634285+0000","last_unstale":"2024-09-21T06:36:05.628918+0000","last_undegraded":"2024-09-21T06:36:05.628918+0000","last_fullsized":"2024-09-21T06:36:05.628918+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:15.185453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834373+0000","last_change":"2024-09-21T06:34:32.639037+0000","last_active":"2024-09-21T06:36:05.834373+0000","last_peered":"2024-09-21T06:36:05.834373+0000","last_clean":"2024-09-21T06:36:05.834373+0000","last_became_active":"2024-09-21T06:34:32.638689+0000","last_became_peered":"2024-09-21T06:34:32.638689+0000","last_unstale":"2024-09-21T06:36:05.834373+0000","last_undegraded":"2024-09-21T06:36:05.834373+0000","last_fullsized":"2024-09-21T06:36:05.834373+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:13.609191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.625323+0000","last_change":"2024-09-21T06:35:05.115051+0000","last_active":"2024-09-21T06:36:05.625323+0000","last_peered":"2024-09-21T06:36:05.625323+0000","last_clean":"2024-09-21T06:36:05.625323+0000","last_became_active":"2024-09-21T06:35:05.114602+0000","last_became_peered":"2024-09-21T06:35:05.114602+0000","last_unstale":"2024-09-21T06:36:05.625323+0000","last_undegraded":"2024-09-21T06:36:05.625323+0000","last_fullsized":"2024-09-21T06:36:05.625323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:48.778188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622758+0000","last_change":"2024-09-21T06:34:32.637237+0000","last_active":"2024-09-21T06:36:05.622758+0000","last_peered":"2024-09-21T06:36:05.622758+0000","last_clean":"2024-09-21T06:36:05.622758+0000","last_became_active":"2024-09-21T06:34:32.634105+0000","last_became_peered":"2024-09-21T06:34:32.634105+0000","last_unstale":"2024-09-21T06:36:05.622758+0000","last_undegraded":"2024-09-21T06:36:05.622758+0000","last_fullsized":"2024-09-21T06:36:05.622758+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:52.530461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835100+0000","last_change":"2024-09-21T06:34:32.624441+0000","last_active":"2024-09-21T06:36:05.835100+0000","last_peered":"2024-09-21T06:36:05.835100+0000","last_clean":"2024-09-21T06:36:05.835100+0000","last_became_active":"2024-09-21T06:34:32.623974+0000","last_became_peered":"2024-09-21T06:34:32.623974+0000","last_unstale":"2024-09-21T06:36:05.835100+0000","last_undegraded":"2024-09-21T06:36:05.835100+0000","last_fullsized":"2024-09-21T06:36:05.835100+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:12.378113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834314+0000","last_change":"2024-09-21T06:34:32.633879+0000","last_active":"2024-09-21T06:36:05.834314+0000","last_peered":"2024-09-21T06:36:05.834314+0000","last_clean":"2024-09-21T06:36:05.834314+0000","last_became_active":"2024-09-21T06:34:32.631791+0000","last_became_peered":"2024-09-21T06:34:32.631791+0000","last_unstale":"2024-09-21T06:36:05.834314+0000","last_undegraded":"2024-09-21T06:36:05.834314+0000","last_fullsized":"2024-09-21T06:36:05.834314+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:46.246892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834708+0000","last_change":"2024-09-21T06:34:32.639974+0000","last_active":"2024-09-21T06:36:05.834708+0000","last_peered":"2024-09-21T06:36:05.834708+0000","last_clean":"2024-09-21T06:36:05.834708+0000","last_became_active":"2024-09-21T06:34:32.639631+0000","last_became_peered":"2024-09-21T06:34:32.639631+0000","last_unstale":"2024-09-21T06:36:05.834708+0000","last_undegraded":"2024-09-21T06:36:05.834708+0000","last_fullsized":"2024-09-21T06:36:05.834708+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:50.069378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835236+0000","last_change":"2024-09-21T06:34:32.645403+0000","last_active":"2024-09-21T06:36:05.835236+0000","last_peered":"2024-09-21T06:36:05.835236+0000","last_clean":"2024-09-21T06:36:05.835236+0000","last_became_active":"2024-09-21T06:34:32.644986+0000","last_became_peered":"2024-09-21T06:34:32.644986+0000","last_unstale":"2024-09-21T06:36:05.835236+0000","last_undegraded":"2024-09-21T06:36:05.835236+0000","last_fullsized":"2024-09-21T06:36:05.835236+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:43.640107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834233+0000","last_change":"2024-09-21T06:34:32.639933+0000","last_active":"2024-09-21T06:36:05.834233+0000","last_peered":"2024-09-21T06:36:05.834233+0000","last_clean":"2024-09-21T06:36:05.834233+0000","last_became_active":"2024-09-21T06:34:32.639438+0000","last_became_peered":"2024-09-21T06:34:32.639438+0000","last_unstale":"2024-09-21T06:36:05.834233+0000","last_undegraded":"2024-09-21T06:36:05.834233+0000","last_fullsized":"2024-09-21T06:36:05.834233+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:18.860219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624647+0000","last_change":"2024-09-21T06:34:32.648580+0000","last_active":"2024-09-21T06:36:05.624647+0000","last_peered":"2024-09-21T06:36:05.624647+0000","last_clean":"2024-09-21T06:36:05.624647+0000","last_became_active":"2024-09-21T06:34:32.648016+0000","last_became_peered":"2024-09-21T06:34:32.648016+0000","last_unstale":"2024-09-21T06:36:05.624647+0000","last_undegraded":"2024-09-21T06:36:05.624647+0000","last_fullsized":"2024-09-21T06:36:05.624647+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:52.858069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622477+0000","last_change":"2024-09-21T06:34:32.633376+0000","last_active":"2024-09-21T06:36:05.622477+0000","last_peered":"2024-09-21T06:36:05.622477+0000","last_clean":"2024-09-21T06:36:05.622477+0000","last_became_active":"2024-09-21T06:34:32.631052+0000","last_became_peered":"2024-09-21T06:34:32.631052+0000","last_unstale":"2024-09-21T06:36:05.622477+0000","last_undegraded":"2024-09-21T06:36:05.622477+0000","last_fullsized":"2024-09-21T06:36:05.622477+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:46.229599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621638+0000","last_change":"2024-09-21T06:34:32.636966+0000","last_active":"2024-09-21T06:36:05.621638+0000","last_peered":"2024-09-21T06:36:05.621638+0000","last_clean":"2024-09-21T06:36:05.621638+0000","last_became_active":"2024-09-21T06:34:32.636565+0000","last_became_peered":"2024-09-21T06:34:32.636565+0000","last_unstale":"2024-09-21T06:36:05.621638+0000","last_undegraded":"2024-09-21T06:36:05.621638+0000","last_fullsized":"2024-09-21T06:36:05.621638+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:36.233082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.625201+0000","last_change":"2024-09-21T06:34:32.646895+0000","last_active":"2024-09-21T06:36:05.625201+0000","last_peered":"2024-09-21T06:36:05.625201+0000","last_clean":"2024-09-21T06:36:05.625201+0000","last_became_active":"2024-09-21T06:34:32.646505+0000","last_became_peered":"2024-09-21T06:34:32.646505+0000","last_unstale":"2024-09-21T06:36:05.625201+0000","last_undegraded":"2024-09-21T06:36:05.625201+0000","last_fullsized":"2024-09-21T06:36:05.625201+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:43.441337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835545+0000","last_change":"2024-09-21T06:34:32.645402+0000","last_active":"2024-09-21T06:36:05.835545+0000","last_peered":"2024-09-21T06:36:05.835545+0000","last_clean":"2024-09-21T06:36:05.835545+0000","last_became_active":"2024-09-21T06:34:32.645013+0000","last_became_peered":"2024-09-21T06:34:32.645013+0000","last_unstale":"2024-09-21T06:36:05.835545+0000","last_undegraded":"2024-09-21T06:36:05.835545+0000","last_fullsized":"2024-09-21T06:36:05.835545+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:33.475847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.628586+0000","last_change":"2024-09-21T06:34:32.643234+0000","last_active":"2024-09-21T06:36:05.628586+0000","last_peered":"2024-09-21T06:36:05.628586+0000","last_clean":"2024-09-21T06:36:05.628586+0000","last_became_active":"2024-09-21T06:34:32.642614+0000","last_became_peered":"2024-09-21T06:34:32.642614+0000","last_unstale":"2024-09-21T06:36:05.628586+0000","last_undegraded":"2024-09-21T06:36:05.628586+0000","last_fullsized":"2024-09-21T06:36:05.628586+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:50.903500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.628594+0000","last_change":"2024-09-21T06:34:32.641887+0000","last_active":"2024-09-21T06:36:05.628594+0000","last_peered":"2024-09-21T06:36:05.628594+0000","last_clean":"2024-09-21T06:36:05.628594+0000","last_became_active":"2024-09-21T06:34:32.641490+0000","last_became_peered":"2024-09-21T06:34:32.641490+0000","last_unstale":"2024-09-21T06:36:05.628594+0000","last_undegraded":"2024-09-21T06:36:05.628594+0000","last_fullsized":"2024-09-21T06:36:05.628594+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:34.550970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834244+0000","last_change":"2024-09-21T06:34:32.641865+0000","last_active":"2024-09-21T06:36:05.834244+0000","last_peered":"2024-09-21T06:36:05.834244+0000","last_clean":"2024-09-21T06:36:05.834244+0000","last_became_active":"2024-09-21T06:34:32.641252+0000","last_became_peered":"2024-09-21T06:34:32.641252+0000","last_unstale":"2024-09-21T06:36:05.834244+0000","last_undegraded":"2024-09-21T06:36:05.834244+0000","last_fullsized":"2024-09-21T06:36:05.834244+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:44.459202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.772096+0000","last_change":"2024-09-21T06:34:32.641259+0000","last_active":"2024-09-21T06:36:05.772096+0000","last_peered":"2024-09-21T06:36:05.772096+0000","last_clean":"2024-09-21T06:36:05.772096+0000","last_became_active":"2024-09-21T06:34:32.640907+0000","last_became_peered":"2024-09-21T06:34:32.640907+0000","last_unstale":"2024-09-21T06:36:05.772096+0000","last_undegraded":"2024-09-21T06:36:05.772096+0000","last_fullsized":"2024-09-21T06:36:05.772096+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:59.966857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622720+0000","last_change":"2024-09-21T06:34:32.646095+0000","last_active":"2024-09-21T06:36:05.622720+0000","last_peered":"2024-09-21T06:36:05.622720+0000","last_clean":"2024-09-21T06:36:05.622720+0000","last_became_active":"2024-09-21T06:34:32.645604+0000","last_became_peered":"2024-09-21T06:34:32.645604+0000","last_unstale":"2024-09-21T06:36:05.622720+0000","last_undegraded":"2024-09-21T06:36:05.622720+0000","last_fullsized":"2024-09-21T06:36:05.622720+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:30.872428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835310+0000","last_change":"2024-09-21T06:34:32.649303+0000","last_active":"2024-09-21T06:36:05.835310+0000","last_peered":"2024-09-21T06:36:05.835310+0000","last_clean":"2024-09-21T06:36:05.835310+0000","last_became_active":"2024-09-21T06:34:32.648965+0000","last_became_peered":"2024-09-21T06:34:32.648965+0000","last_unstale":"2024-09-21T06:36:05.835310+0000","last_undegraded":"2024-09-21T06:36:05.835310+0000","last_fullsized":"2024-09-21T06:36:05.835310+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:51.335949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834838+0000","last_change":"2024-09-21T06:34:32.637261+0000","last_active":"2024-09-21T06:36:05.834838+0000","last_peered":"2024-09-21T06:36:05.834838+0000","last_clean":"2024-09-21T06:36:05.834838+0000","last_became_active":"2024-09-21T06:34:32.636861+0000","last_became_peered":"2024-09-21T06:34:32.636861+0000","last_unstale":"2024-09-21T06:36:05.834838+0000","last_undegraded":"2024-09-21T06:36:05.834838+0000","last_fullsized":"2024-09-21T06:36:05.834838+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:55.271027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834070+0000","last_change":"2024-09-21T06:34:32.635156+0000","last_active":"2024-09-21T06:36:05.834070+0000","last_peered":"2024-09-21T06:36:05.834070+0000","last_clean":"2024-09-21T06:36:05.834070+0000","last_became_active":"2024-09-21T06:34:32.634802+0000","last_became_peered":"2024-09-21T06:34:32.634802+0000","last_unstale":"2024-09-21T06:36:05.834070+0000","last_undegraded":"2024-09-21T06:36:05.834070+0000","last_fullsized":"2024-09-21T06:36:05.834070+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:16.139829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622299+0000","last_change":"2024-09-21T06:34:32.637509+0000","last_active":"2024-09-21T06:36:05.622299+0000","last_peered":"2024-09-21T06:36:05.622299+0000","last_clean":"2024-09-21T06:36:05.622299+0000","last_became_active":"2024-09-21T06:34:32.637136+0000","last_became_peered":"2024-09-21T06:34:32.637136+0000","last_unstale":"2024-09-21T06:36:05.622299+0000","last_undegraded":"2024-09-21T06:36:05.622299+0000","last_fullsized":"2024-09-21T06:36:05.622299+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:48.088988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834524+0000","last_change":"2024-09-21T06:35:05.114819+0000","last_active":"2024-09-21T06:36:05.834524+0000","last_peered":"2024-09-21T06:36:05.834524+0000","last_clean":"2024-09-21T06:36:05.834524+0000","last_became_active":"2024-09-21T06:35:05.114424+0000","last_became_peered":"2024-09-21T06:35:05.114424+0000","last_unstale":"2024-09-21T06:36:05.834524+0000","last_undegraded":"2024-09-21T06:36:05.834524+0000","last_fullsized":"2024-09-21T06:36:05.834524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:59.323643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624873+0000","last_change":"2024-09-21T06:34:32.636103+0000","last_active":"2024-09-21T06:36:05.624873+0000","last_peered":"2024-09-21T06:36:05.624873+0000","last_clean":"2024-09-21T06:36:05.624873+0000","last_became_active":"2024-09-21T06:34:32.635728+0000","last_became_peered":"2024-09-21T06:34:32.635728+0000","last_unstale":"2024-09-21T06:36:05.624873+0000","last_undegraded":"2024-09-21T06:36:05.624873+0000","last_fullsized":"2024-09-21T06:36:05.624873+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:16.731402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.772051+0000","last_change":"2024-09-21T06:34:32.635654+0000","last_active":"2024-09-21T06:36:05.772051+0000","last_peered":"2024-09-21T06:36:05.772051+0000","last_clean":"2024-09-21T06:36:05.772051+0000","last_became_active":"2024-09-21T06:34:32.635265+0000","last_became_peered":"2024-09-21T06:34:32.635265+0000","last_unstale":"2024-09-21T06:36:05.772051+0000","last_undegraded":"2024-09-21T06:36:05.772051+0000","last_fullsized":"2024-09-21T06:36:05.772051+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:28.169625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.628625+0000","last_change":"2024-09-21T06:34:32.644640+0000","last_active":"2024-09-21T06:36:05.628625+0000","last_peered":"2024-09-21T06:36:05.628625+0000","last_clean":"2024-09-21T06:36:05.628625+0000","last_became_active":"2024-09-21T06:34:32.644334+0000","last_became_peered":"2024-09-21T06:34:32.644334+0000","last_unstale":"2024-09-21T06:36:05.628625+0000","last_undegraded":"2024-09-21T06:36:05.628625+0000","last_fullsized":"2024-09-21T06:36:05.628625+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:25.572178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.629082+0000","last_change":"2024-09-21T06:34:32.643705+0000","last_active":"2024-09-21T06:36:05.629082+0000","last_peered":"2024-09-21T06:36:05.629082+0000","last_clean":"2024-09-21T06:36:05.629082+0000","last_became_active":"2024-09-21T06:34:32.642705+0000","last_became_peered":"2024-09-21T06:34:32.642705+0000","last_unstale":"2024-09-21T06:36:05.629082+0000","last_undegraded":"2024-09-21T06:36:05.629082+0000","last_fullsized":"2024-09-21T06:36:05.629082+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:22.542559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624987+0000","last_change":"2024-09-21T06:34:32.646539+0000","last_active":"2024-09-21T06:36:05.624987+0000","last_peered":"2024-09-21T06:36:05.624987+0000","last_clean":"2024-09-21T06:36:05.624987+0000","last_became_active":"2024-09-21T06:34:32.645992+0000","last_became_peered":"2024-09-21T06:34:32.645992+0000","last_unstale":"2024-09-21T06:36:05.624987+0000","last_undegraded":"2024-09-21T06:36:05.624987+0000","last_fullsized":"2024-09-21T06:36:05.624987+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:38.856001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69583,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":147,"num_read_kb":143,"num_write":85,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":208749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71,"ondisk_log_size":71,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282016,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":929792,"data_stored":485699,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939687,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1064960,"data_stored":1140950,"data_compressed":65460,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103212,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":937984,"data_stored":490377,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299442,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":937984,"data_stored":486303,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495670,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":987136,"data_stored":1080745,"data_compressed":65460,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659196,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":925696,"data_stored":481603,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855425,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":991232,"data_stored":1076249,"data_compressed":65460,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018950,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":933888,"data_stored":485721,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215179,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":933888,"data_stored":481707,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1007616,"data_stored":550422,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1003520,"data_stored":550388,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":929792,"data_stored":481693,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":68819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:36:17.736 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T06:36:17.736 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T06:36:17.736 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-21T06:36:17.736 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph health --format=json 2024-09-21T06:36:17.996 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:36:18.262 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:17 smithi067 ceph-mon[31232]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:36:18.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:17 smithi100 ceph-mon[33401]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:36:18.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:17 smithi203 ceph-mon[33735]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:36:18.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.492+0000 7f37ce8cc640 1 -- 172.21.15.67:0/686389307 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37c8109df0 msgr2=0x7f37c8110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:18.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.493+0000 7f37ce8cc640 1 --2- 172.21.15.67:0/686389307 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37c8109df0 0x7f37c8110620 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f37c0009ca0 tx=0x7f37c002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:36:18.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.493+0000 7f37ce8cc640 1 -- 172.21.15.67:0/686389307 shutdown_connections 2024-09-21T06:36:18.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.493+0000 7f37ce8cc640 1 --2- 172.21.15.67:0/686389307 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37c8109df0 0x7f37c8110620 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:18.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.493+0000 7f37ce8cc640 1 --2- 172.21.15.67:0/686389307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37c81094a0 0x7f37c81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:18.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.493+0000 7f37ce8cc640 1 --2- 172.21.15.67:0/686389307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37c8103b60 0x7f37c8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:18.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.493+0000 7f37ce8cc640 1 -- 172.21.15.67:0/686389307 >> 172.21.15.67:0/686389307 conn(0x7f37c807b850 msgr2=0x7f37c807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:18.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.493+0000 7f37ce8cc640 1 -- 172.21.15.67:0/686389307 shutdown_connections 2024-09-21T06:36:18.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.493+0000 7f37ce8cc640 1 -- 172.21.15.67:0/686389307 wait complete. 2024-09-21T06:36:18.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.493+0000 7f37ce8cc640 1 Processor -- start 2024-09-21T06:36:18.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.493+0000 7f37ce8cc640 1 -- start start 2024-09-21T06:36:18.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.494+0000 7f37ce8cc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37c8103b60 0x7f37c819a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:18.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.494+0000 7f37ce8cc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37c81094a0 0x7f37c819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:18.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.494+0000 7f37ce8cc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37c8109df0 0x7f37c819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:18.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.494+0000 7f37ce8cc640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f37c81139b0 con 0x7f37c8109df0 2024-09-21T06:36:18.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.494+0000 7f37ce8cc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f37c8113830 con 0x7f37c8103b60 2024-09-21T06:36:18.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.494+0000 7f37ce8cc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f37c8113b30 con 0x7f37c81094a0 2024-09-21T06:36:18.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.494+0000 7f37cce69640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37c8109df0 0x7f37c819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:18.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.494+0000 7f37cce69640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37c8109df0 0x7f37c819fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59340/0 (socket says 172.21.15.67:59340) 2024-09-21T06:36:18.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.494+0000 7f37cce69640 1 -- 172.21.15.67:0/1908009310 learned_addr learned my addr 172.21.15.67:0/1908009310 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:18.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.494+0000 7f37c77fe640 1 --2- 172.21.15.67:0/1908009310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37c81094a0 0x7f37c819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:18.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.494+0000 7f37c7fff640 1 --2- 172.21.15.67:0/1908009310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37c8103b60 0x7f37c819a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:18.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.494+0000 7f37cce69640 1 -- 172.21.15.67:0/1908009310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37c81094a0 msgr2=0x7f37c819a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:18.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.494+0000 7f37cce69640 1 --2- 172.21.15.67:0/1908009310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37c81094a0 0x7f37c819a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:18.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.494+0000 7f37cce69640 1 -- 172.21.15.67:0/1908009310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37c8103b60 msgr2=0x7f37c819a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:18.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.494+0000 7f37cce69640 1 --2- 172.21.15.67:0/1908009310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37c8103b60 0x7f37c819a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:18.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.494+0000 7f37cce69640 1 -- 172.21.15.67:0/1908009310 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c81a0250 con 0x7f37c8109df0 2024-09-21T06:36:18.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.495+0000 7f37cce69640 1 --2- 172.21.15.67:0/1908009310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37c8109df0 0x7f37c819fbe0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f37c0005cc0 tx=0x7f37c0032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:18.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.495+0000 7f37c57fa640 1 -- 172.21.15.67:0/1908009310 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c0032e00 con 0x7f37c8109df0 2024-09-21T06:36:18.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.495+0000 7f37ce8cc640 1 -- 172.21.15.67:0/1908009310 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c81a04e0 con 0x7f37c8109df0 2024-09-21T06:36:18.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.495+0000 7f37c57fa640 1 -- 172.21.15.67:0/1908009310 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37c00044d0 con 0x7f37c8109df0 2024-09-21T06:36:18.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.495+0000 7f37c57fa640 1 -- 172.21.15.67:0/1908009310 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c0030b70 con 0x7f37c8109df0 2024-09-21T06:36:18.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.495+0000 7f37ce8cc640 1 -- 172.21.15.67:0/1908009310 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c81a0a20 con 0x7f37c8109df0 2024-09-21T06:36:18.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.496+0000 7f37ce8cc640 1 -- 172.21.15.67:0/1908009310 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37c8103f70 con 0x7f37c8109df0 2024-09-21T06:36:18.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.498+0000 7f37c57fa640 1 -- 172.21.15.67:0/1908009310 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f37c0030d10 con 0x7f37c8109df0 2024-09-21T06:36:18.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.498+0000 7f37c57fa640 1 --2- 172.21.15.67:0/1908009310 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f37b00b65d0 0x7f37b00b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:18.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.498+0000 7f37c7fff640 1 --2- 172.21.15.67:0/1908009310 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f37b00b65d0 0x7f37b00b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:18.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.498+0000 7f37c57fa640 1 -- 172.21.15.67:0/1908009310 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f37c0108bf0 con 0x7f37c8109df0 2024-09-21T06:36:18.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.499+0000 7f37c7fff640 1 --2- 172.21.15.67:0/1908009310 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f37b00b65d0 0x7f37b00b8ae0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f37b800bbe0 tx=0x7f37b800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:18.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.502+0000 7f37c57fa640 1 -- 172.21.15.67:0/1908009310 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f37c00cefe0 con 0x7f37c8109df0 2024-09-21T06:36:18.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.731+0000 7f37ce8cc640 1 -- 172.21.15.67:0/1908009310 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f37c810bc40 con 0x7f37c8109df0 2024-09-21T06:36:18.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.731+0000 7f37c57fa640 1 -- 172.21.15.67:0/1908009310 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f37c00d33c0 con 0x7f37c8109df0 2024-09-21T06:36:18.733 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:36:18.733 INFO:teuthology.orchestra.run.smithi067.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T06:36:18.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.733+0000 7f37ce8cc640 1 -- 172.21.15.67:0/1908009310 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f37b00b65d0 msgr2=0x7f37b00b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:18.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.733+0000 7f37ce8cc640 1 --2- 172.21.15.67:0/1908009310 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f37b00b65d0 0x7f37b00b8ae0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f37b800bbe0 tx=0x7f37b800b390 comp rx=0 tx=0).stop 2024-09-21T06:36:18.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.733+0000 7f37ce8cc640 1 -- 172.21.15.67:0/1908009310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37c8109df0 msgr2=0x7f37c819fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:18.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.733+0000 7f37ce8cc640 1 --2- 172.21.15.67:0/1908009310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37c8109df0 0x7f37c819fbe0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f37c0005cc0 tx=0x7f37c0032a10 comp rx=0 tx=0).stop 2024-09-21T06:36:18.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.733+0000 7f37ce8cc640 1 -- 172.21.15.67:0/1908009310 shutdown_connections 2024-09-21T06:36:18.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.733+0000 7f37ce8cc640 1 --2- 172.21.15.67:0/1908009310 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f37b00b65d0 0x7f37b00b8ae0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:18.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.733+0000 7f37ce8cc640 1 --2- 172.21.15.67:0/1908009310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37c8109df0 0x7f37c819fbe0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:18.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.733+0000 7f37ce8cc640 1 --2- 172.21.15.67:0/1908009310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37c81094a0 0x7f37c819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:18.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.733+0000 7f37ce8cc640 1 --2- 172.21.15.67:0/1908009310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37c8103b60 0x7f37c819a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:18.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.733+0000 7f37ce8cc640 1 -- 172.21.15.67:0/1908009310 >> 172.21.15.67:0/1908009310 conn(0x7f37c807b850 msgr2=0x7f37c81054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:18.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.734+0000 7f37ce8cc640 1 -- 172.21.15.67:0/1908009310 shutdown_connections 2024-09-21T06:36:18.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:18.734+0000 7f37ce8cc640 1 -- 172.21.15.67:0/1908009310 wait complete. 2024-09-21T06:36:19.291 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:18 smithi067 ceph-mon[31232]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T06:36:19.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:18 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1908009310' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T06:36:19.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:18 smithi100 ceph-mon[33401]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T06:36:19.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:18 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1908009310' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T06:36:19.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:18 smithi203 ceph-mon[33735]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T06:36:19.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:18 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1908009310' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T06:36:19.577 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-21T06:36:19.577 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:36:19.837 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:36:20.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.354+0000 7fbed68e5640 1 -- 172.21.15.67:0/3824583256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbed0110140 msgr2=0x7fbed0112560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:20.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.354+0000 7fbed68e5640 1 --2- 172.21.15.67:0/3824583256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbed0110140 0x7fbed0112560 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009ca0 tx=0x7fbec802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:36:20.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.354+0000 7fbed68e5640 1 -- 172.21.15.67:0/3824583256 shutdown_connections 2024-09-21T06:36:20.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.354+0000 7fbed68e5640 1 --2- 172.21.15.67:0/3824583256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbed0110140 0x7fbed0112560 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:20.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.355+0000 7fbed68e5640 1 --2- 172.21.15.67:0/3824583256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbed0076080 0x7fbed010fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:20.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.355+0000 7fbed68e5640 1 --2- 172.21.15.67:0/3824583256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbed0075730 0x7fbed0075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:20.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.355+0000 7fbed68e5640 1 -- 172.21.15.67:0/3824583256 >> 172.21.15.67:0/3824583256 conn(0x7fbed007b850 msgr2=0x7fbed007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:20.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.355+0000 7fbed68e5640 1 -- 172.21.15.67:0/3824583256 shutdown_connections 2024-09-21T06:36:20.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.355+0000 7fbed68e5640 1 -- 172.21.15.67:0/3824583256 wait complete. 2024-09-21T06:36:20.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.355+0000 7fbed68e5640 1 Processor -- start 2024-09-21T06:36:20.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.355+0000 7fbed68e5640 1 -- start start 2024-09-21T06:36:20.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.356+0000 7fbed68e5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbed0075730 0x7fbed019c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:20.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.356+0000 7fbed68e5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbed0076080 0x7fbed019cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:20.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.356+0000 7fbed68e5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbed0110140 0x7fbed01a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:20.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.356+0000 7fbed68e5640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbed0115b80 con 0x7fbed0076080 2024-09-21T06:36:20.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.356+0000 7fbed68e5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbed0115a00 con 0x7fbed0075730 2024-09-21T06:36:20.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.356+0000 7fbed68e5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbed0115d00 con 0x7fbed0110140 2024-09-21T06:36:20.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.356+0000 7fbecf7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbed0076080 0x7fbed019cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:20.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.356+0000 7fbecf7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbed0076080 0x7fbed019cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59358/0 (socket says 172.21.15.67:59358) 2024-09-21T06:36:20.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.356+0000 7fbecf7fe640 1 -- 172.21.15.67:0/2458424863 learned_addr learned my addr 172.21.15.67:0/2458424863 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:20.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.356+0000 7fbecffff640 1 --2- 172.21.15.67:0/2458424863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbed0075730 0x7fbed019c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:20.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.356+0000 7fbed4e82640 1 --2- 172.21.15.67:0/2458424863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbed0110140 0x7fbed01a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:20.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.356+0000 7fbecf7fe640 1 -- 172.21.15.67:0/2458424863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbed0110140 msgr2=0x7fbed01a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:20.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.356+0000 7fbecf7fe640 1 --2- 172.21.15.67:0/2458424863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbed0110140 0x7fbed01a1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:20.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.356+0000 7fbecf7fe640 1 -- 172.21.15.67:0/2458424863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbed0075730 msgr2=0x7fbed019c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:20.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.356+0000 7fbecf7fe640 1 --2- 172.21.15.67:0/2458424863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbed0075730 0x7fbed019c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:20.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.356+0000 7fbecf7fe640 1 -- 172.21.15.67:0/2458424863 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed01a2460 con 0x7fbed0076080 2024-09-21T06:36:20.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.356+0000 7fbecf7fe640 1 --2- 172.21.15.67:0/2458424863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbed0076080 0x7fbed019cb50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbec400c8c0 tx=0x7fbec400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:20.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.357+0000 7fbecd7fa640 1 -- 172.21.15.67:0/2458424863 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec4007bb0 con 0x7fbed0076080 2024-09-21T06:36:20.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.357+0000 7fbecd7fa640 1 -- 172.21.15.67:0/2458424863 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbec4007d50 con 0x7fbed0076080 2024-09-21T06:36:20.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.357+0000 7fbed68e5640 1 -- 172.21.15.67:0/2458424863 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbed01a2750 con 0x7fbed0076080 2024-09-21T06:36:20.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.357+0000 7fbecd7fa640 1 -- 172.21.15.67:0/2458424863 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec40186a0 con 0x7fbed0076080 2024-09-21T06:36:20.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.357+0000 7fbed68e5640 1 -- 172.21.15.67:0/2458424863 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbed01a8f70 con 0x7fbed0076080 2024-09-21T06:36:20.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.358+0000 7fbed68e5640 1 -- 172.21.15.67:0/2458424863 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbed0075b40 con 0x7fbed0076080 2024-09-21T06:36:20.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.359+0000 7fbecd7fa640 1 -- 172.21.15.67:0/2458424863 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fbec400fa00 con 0x7fbed0076080 2024-09-21T06:36:20.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.360+0000 7fbecd7fa640 1 --2- 172.21.15.67:0/2458424863 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fbeb80b65d0 0x7fbeb80b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:20.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.360+0000 7fbecffff640 1 --2- 172.21.15.67:0/2458424863 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fbeb80b65d0 0x7fbeb80b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:20.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.360+0000 7fbecd7fa640 1 -- 172.21.15.67:0/2458424863 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fbec40e7b40 con 0x7fbed0076080 2024-09-21T06:36:20.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.361+0000 7fbecffff640 1 --2- 172.21.15.67:0/2458424863 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fbeb80b65d0 0x7fbeb80b8ae0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbec000bbe0 tx=0x7fbec000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:20.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.364+0000 7fbecd7fa640 1 -- 172.21.15.67:0/2458424863 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbec4014030 con 0x7fbed0076080 2024-09-21T06:36:20.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.588+0000 7fbed68e5640 1 -- 172.21.15.67:0/2458424863 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbed019d6e0 con 0x7fbed0076080 2024-09-21T06:36:20.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.589+0000 7fbecd7fa640 1 -- 172.21.15.67:0/2458424863 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7fbec40adf30 con 0x7fbed0076080 2024-09-21T06:36:20.590 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:36:20.590 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:36:20.591 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:36:20.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.591+0000 7fbed68e5640 1 -- 172.21.15.67:0/2458424863 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fbeb80b65d0 msgr2=0x7fbeb80b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:20.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.591+0000 7fbed68e5640 1 --2- 172.21.15.67:0/2458424863 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fbeb80b65d0 0x7fbeb80b8ae0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbec000bbe0 tx=0x7fbec000b390 comp rx=0 tx=0).stop 2024-09-21T06:36:20.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.591+0000 7fbed68e5640 1 -- 172.21.15.67:0/2458424863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbed0076080 msgr2=0x7fbed019cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:20.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.591+0000 7fbed68e5640 1 --2- 172.21.15.67:0/2458424863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbed0076080 0x7fbed019cb50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbec400c8c0 tx=0x7fbec400cd80 comp rx=0 tx=0).stop 2024-09-21T06:36:20.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.591+0000 7fbed68e5640 1 -- 172.21.15.67:0/2458424863 shutdown_connections 2024-09-21T06:36:20.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.591+0000 7fbed68e5640 1 --2- 172.21.15.67:0/2458424863 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fbeb80b65d0 0x7fbeb80b8ae0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:20.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.591+0000 7fbed68e5640 1 --2- 172.21.15.67:0/2458424863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbed0110140 0x7fbed01a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:20.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.591+0000 7fbed68e5640 1 --2- 172.21.15.67:0/2458424863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbed0076080 0x7fbed019cb50 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:20.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.591+0000 7fbed68e5640 1 --2- 172.21.15.67:0/2458424863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbed0075730 0x7fbed019c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:20.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.591+0000 7fbed68e5640 1 -- 172.21.15.67:0/2458424863 >> 172.21.15.67:0/2458424863 conn(0x7fbed007b850 msgr2=0x7fbed01018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:20.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.591+0000 7fbed68e5640 1 -- 172.21.15.67:0/2458424863 shutdown_connections 2024-09-21T06:36:20.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:20.591+0000 7fbed68e5640 1 -- 172.21.15.67:0/2458424863 wait complete. 2024-09-21T06:36:21.287 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:20 smithi067 ceph-mon[31232]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T06:36:21.287 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:20 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2458424863' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:36:21.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:20 smithi100 ceph-mon[33401]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T06:36:21.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:20 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2458424863' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:36:21.317 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:36:21.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:20 smithi203 ceph-mon[33735]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T06:36:21.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:20 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2458424863' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:36:21.576 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:36:22.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.090+0000 7f746a6b5640 1 -- 172.21.15.67:0/3342221219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7464103b80 msgr2=0x7f7464103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:22.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.090+0000 7f746a6b5640 1 --2- 172.21.15.67:0/3342221219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7464103b80 0x7f7464103f90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7454009900 tx=0x7f745402a410 comp rx=0 tx=0).stop 2024-09-21T06:36:22.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.090+0000 7f746a6b5640 1 -- 172.21.15.67:0/3342221219 shutdown_connections 2024-09-21T06:36:22.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.090+0000 7f746a6b5640 1 --2- 172.21.15.67:0/3342221219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7464109e10 0x7f7464110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:22.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.090+0000 7f746a6b5640 1 --2- 172.21.15.67:0/3342221219 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74641094c0 0x7f74641098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:22.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.090+0000 7f746a6b5640 1 --2- 172.21.15.67:0/3342221219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7464103b80 0x7f7464103f90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:22.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.090+0000 7f746a6b5640 1 -- 172.21.15.67:0/3342221219 >> 172.21.15.67:0/3342221219 conn(0x7f74640ff610 msgr2=0x7f7464101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:22.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.090+0000 7f746a6b5640 1 -- 172.21.15.67:0/3342221219 shutdown_connections 2024-09-21T06:36:22.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.090+0000 7f746a6b5640 1 -- 172.21.15.67:0/3342221219 wait complete. 2024-09-21T06:36:22.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.091+0000 7f746a6b5640 1 Processor -- start 2024-09-21T06:36:22.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.091+0000 7f746a6b5640 1 -- start start 2024-09-21T06:36:22.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.091+0000 7f746a6b5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7464103b80 0x7f746419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:22.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.091+0000 7f746a6b5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74641094c0 0x7f746419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:22.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.091+0000 7f746a6b5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7464109e10 0x7f746419fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:22.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.091+0000 7f746a6b5640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7464113980 con 0x7f7464103b80 2024-09-21T06:36:22.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.091+0000 7f746a6b5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7464113800 con 0x7f7464109e10 2024-09-21T06:36:22.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.091+0000 7f746a6b5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7464113b00 con 0x7f74641094c0 2024-09-21T06:36:22.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.091+0000 7f7463fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7464103b80 0x7f746419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:22.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.091+0000 7f7463fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7464103b80 0x7f746419a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59372/0 (socket says 172.21.15.67:59372) 2024-09-21T06:36:22.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.091+0000 7f7463fff640 1 -- 172.21.15.67:0/447151829 learned_addr learned my addr 172.21.15.67:0/447151829 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:22.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.092+0000 7f7468c52640 1 --2- 172.21.15.67:0/447151829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7464109e10 0x7f746419fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:22.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.092+0000 7f745bfff640 1 --2- 172.21.15.67:0/447151829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74641094c0 0x7f746419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:22.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.092+0000 7f7463fff640 1 -- 172.21.15.67:0/447151829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74641094c0 msgr2=0x7f746419aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:22.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.092+0000 7f7463fff640 1 --2- 172.21.15.67:0/447151829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74641094c0 0x7f746419aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:22.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.092+0000 7f7463fff640 1 -- 172.21.15.67:0/447151829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7464109e10 msgr2=0x7f746419fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:22.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.092+0000 7f7463fff640 1 --2- 172.21.15.67:0/447151829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7464109e10 0x7f746419fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:22.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.092+0000 7f7463fff640 1 -- 172.21.15.67:0/447151829 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74641a03e0 con 0x7f7464103b80 2024-09-21T06:36:22.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.092+0000 7f745bfff640 1 --2- 172.21.15.67:0/447151829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74641094c0 0x7f746419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:36:22.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.092+0000 7f7468c52640 1 --2- 172.21.15.67:0/447151829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7464109e10 0x7f746419fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:36:22.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.092+0000 7f7463fff640 1 --2- 172.21.15.67:0/447151829 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7464103b80 0x7f746419a470 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7454009230 tx=0x7f7454004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:22.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.092+0000 7f7461ffb640 1 -- 172.21.15.67:0/447151829 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745403f030 con 0x7f7464103b80 2024-09-21T06:36:22.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.092+0000 7f7461ffb640 1 -- 172.21.15.67:0/447151829 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7454003a80 con 0x7f7464103b80 2024-09-21T06:36:22.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.092+0000 7f746a6b5640 1 -- 172.21.15.67:0/447151829 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74641a0670 con 0x7f7464103b80 2024-09-21T06:36:22.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.092+0000 7f7461ffb640 1 -- 172.21.15.67:0/447151829 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7454035070 con 0x7f7464103b80 2024-09-21T06:36:22.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.092+0000 7f746a6b5640 1 -- 172.21.15.67:0/447151829 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74641a6e30 con 0x7f7464103b80 2024-09-21T06:36:22.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.093+0000 7f746a6b5640 1 -- 172.21.15.67:0/447151829 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7464104e00 con 0x7f7464103b80 2024-09-21T06:36:22.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.095+0000 7f7461ffb640 1 -- 172.21.15.67:0/447151829 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f7454042030 con 0x7f7464103b80 2024-09-21T06:36:22.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.096+0000 7f7461ffb640 1 --2- 172.21.15.67:0/447151829 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f74500b65a0 0x7f74500b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:22.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.096+0000 7f745bfff640 1 --2- 172.21.15.67:0/447151829 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f74500b65a0 0x7f74500b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:22.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.096+0000 7f7461ffb640 1 -- 172.21.15.67:0/447151829 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f74541084a0 con 0x7f7464103b80 2024-09-21T06:36:22.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.096+0000 7f745bfff640 1 --2- 172.21.15.67:0/447151829 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f74500b65a0 0x7f74500b8ab0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7464108f00 tx=0x7f744c005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:22.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.099+0000 7f7461ffb640 1 -- 172.21.15.67:0/447151829 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f74540ce800 con 0x7f7464103b80 2024-09-21T06:36:22.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.325+0000 7f746a6b5640 1 -- 172.21.15.67:0/447151829 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7464103f90 con 0x7f7464103b80 2024-09-21T06:36:22.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.326+0000 7f7461ffb640 1 -- 172.21.15.67:0/447151829 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7f7464103f90 con 0x7f7464103b80 2024-09-21T06:36:22.327 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:36:22.327 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:36:22.328 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:36:22.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.328+0000 7f746a6b5640 1 -- 172.21.15.67:0/447151829 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f74500b65a0 msgr2=0x7f74500b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:22.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.328+0000 7f746a6b5640 1 --2- 172.21.15.67:0/447151829 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f74500b65a0 0x7f74500b8ab0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7464108f00 tx=0x7f744c005e50 comp rx=0 tx=0).stop 2024-09-21T06:36:22.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.328+0000 7f746a6b5640 1 -- 172.21.15.67:0/447151829 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7464103b80 msgr2=0x7f746419a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:22.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.328+0000 7f746a6b5640 1 --2- 172.21.15.67:0/447151829 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7464103b80 0x7f746419a470 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7454009230 tx=0x7f7454004270 comp rx=0 tx=0).stop 2024-09-21T06:36:22.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.328+0000 7f746a6b5640 1 -- 172.21.15.67:0/447151829 shutdown_connections 2024-09-21T06:36:22.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.328+0000 7f746a6b5640 1 --2- 172.21.15.67:0/447151829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7464109e10 0x7f746419fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:22.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.328+0000 7f746a6b5640 1 --2- 172.21.15.67:0/447151829 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f74500b65a0 0x7f74500b8ab0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:22.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.328+0000 7f746a6b5640 1 --2- 172.21.15.67:0/447151829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74641094c0 0x7f746419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:22.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.328+0000 7f746a6b5640 1 --2- 172.21.15.67:0/447151829 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7464103b80 0x7f746419a470 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:22.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.328+0000 7f746a6b5640 1 -- 172.21.15.67:0/447151829 >> 172.21.15.67:0/447151829 conn(0x7f74640ff610 msgr2=0x7f7464101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:22.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.328+0000 7f746a6b5640 1 -- 172.21.15.67:0/447151829 shutdown_connections 2024-09-21T06:36:22.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:22.328+0000 7f746a6b5640 1 -- 172.21.15.67:0/447151829 wait complete. 2024-09-21T06:36:23.145 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:36:23.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:22 smithi067 ceph-mon[31232]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T06:36:23.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:22 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/447151829' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:36:23.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:22 smithi100 ceph-mon[33401]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T06:36:23.304 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:22 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/447151829' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:36:23.404 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:36:23.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:22 smithi203 ceph-mon[33735]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T06:36:23.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:22 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/447151829' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:36:23.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.899+0000 7efd08274640 1 -- 172.21.15.67:0/3584320740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd00109df0 msgr2=0x7efd00110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:23.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.899+0000 7efd08274640 1 --2- 172.21.15.67:0/3584320740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd00109df0 0x7efd00110620 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7efcf0009ca0 tx=0x7efcf002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:36:23.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.899+0000 7efd08274640 1 -- 172.21.15.67:0/3584320740 shutdown_connections 2024-09-21T06:36:23.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.899+0000 7efd08274640 1 --2- 172.21.15.67:0/3584320740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd00109df0 0x7efd00110620 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:23.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.899+0000 7efd08274640 1 --2- 172.21.15.67:0/3584320740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd001094a0 0x7efd001098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:23.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.899+0000 7efd08274640 1 --2- 172.21.15.67:0/3584320740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd00103b60 0x7efd00103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:23.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.899+0000 7efd08274640 1 -- 172.21.15.67:0/3584320740 >> 172.21.15.67:0/3584320740 conn(0x7efd0007b850 msgr2=0x7efd0007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:23.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.899+0000 7efd08274640 1 -- 172.21.15.67:0/3584320740 shutdown_connections 2024-09-21T06:36:23.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.899+0000 7efd08274640 1 -- 172.21.15.67:0/3584320740 wait complete. 2024-09-21T06:36:23.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.900+0000 7efd08274640 1 Processor -- start 2024-09-21T06:36:23.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.900+0000 7efd08274640 1 -- start start 2024-09-21T06:36:23.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.900+0000 7efd08274640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd00103b60 0x7efd0019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:23.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.900+0000 7efd08274640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd001094a0 0x7efd0019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:23.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.900+0000 7efd08274640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd00109df0 0x7efd0019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:23.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.900+0000 7efd08274640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efd001139b0 con 0x7efd001094a0 2024-09-21T06:36:23.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.900+0000 7efd08274640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efd00113830 con 0x7efd00109df0 2024-09-21T06:36:23.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.900+0000 7efd08274640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7efd00113b30 con 0x7efd00103b60 2024-09-21T06:36:23.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.900+0000 7efd0580f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd001094a0 0x7efd0019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:23.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.900+0000 7efd0580f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd001094a0 0x7efd0019aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59388/0 (socket says 172.21.15.67:59388) 2024-09-21T06:36:23.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.900+0000 7efd0580f640 1 -- 172.21.15.67:0/2978139276 learned_addr learned my addr 172.21.15.67:0/2978139276 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:23.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.900+0000 7efd06010640 1 --2- 172.21.15.67:0/2978139276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd00103b60 0x7efd0019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:23.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.900+0000 7efd06811640 1 --2- 172.21.15.67:0/2978139276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd00109df0 0x7efd0019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:23.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.901+0000 7efd0580f640 1 -- 172.21.15.67:0/2978139276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd00103b60 msgr2=0x7efd0019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:23.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.901+0000 7efd0580f640 1 --2- 172.21.15.67:0/2978139276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd00103b60 0x7efd0019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:23.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.901+0000 7efd0580f640 1 -- 172.21.15.67:0/2978139276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd00109df0 msgr2=0x7efd0019fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:23.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.901+0000 7efd0580f640 1 --2- 172.21.15.67:0/2978139276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd00109df0 0x7efd0019fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:23.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.901+0000 7efd0580f640 1 -- 172.21.15.67:0/2978139276 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd001a0370 con 0x7efd001094a0 2024-09-21T06:36:23.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.901+0000 7efd0580f640 1 --2- 172.21.15.67:0/2978139276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd001094a0 0x7efd0019aa60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7efcf400d750 tx=0x7efcf400db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:23.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.901+0000 7efcff7fe640 1 -- 172.21.15.67:0/2978139276 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf4017070 con 0x7efd001094a0 2024-09-21T06:36:23.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.901+0000 7efd08274640 1 -- 172.21.15.67:0/2978139276 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd001a0660 con 0x7efd001094a0 2024-09-21T06:36:23.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.901+0000 7efcff7fe640 1 -- 172.21.15.67:0/2978139276 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efcf4009820 con 0x7efd001094a0 2024-09-21T06:36:23.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.901+0000 7efcff7fe640 1 -- 172.21.15.67:0/2978139276 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf4004bc0 con 0x7efd001094a0 2024-09-21T06:36:23.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.901+0000 7efd08274640 1 -- 172.21.15.67:0/2978139276 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd001a6e80 con 0x7efd001094a0 2024-09-21T06:36:23.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.902+0000 7efd08274640 1 -- 172.21.15.67:0/2978139276 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd00103f70 con 0x7efd001094a0 2024-09-21T06:36:23.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.904+0000 7efcff7fe640 1 -- 172.21.15.67:0/2978139276 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7efcf40099d0 con 0x7efd001094a0 2024-09-21T06:36:23.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.905+0000 7efcff7fe640 1 --2- 172.21.15.67:0/2978139276 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7efce80b65d0 0x7efce80b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:23.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.905+0000 7efd06010640 1 --2- 172.21.15.67:0/2978139276 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7efce80b65d0 0x7efce80b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:23.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.905+0000 7efcff7fe640 1 -- 172.21.15.67:0/2978139276 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7efcf40e74a0 con 0x7efd001094a0 2024-09-21T06:36:23.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.905+0000 7efd06010640 1 --2- 172.21.15.67:0/2978139276 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7efce80b65d0 0x7efce80b8ae0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efcf800abe0 tx=0x7efcf800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:23.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:23.908+0000 7efcff7fe640 1 -- 172.21.15.67:0/2978139276 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efcf40ad800 con 0x7efd001094a0 2024-09-21T06:36:24.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:24.100+0000 7efd08274640 1 -- 172.21.15.67:0/2978139276 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd0019b5f0 con 0x7efd001094a0 2024-09-21T06:36:24.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:24.102+0000 7efcff7fe640 1 -- 172.21.15.67:0/2978139276 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24954 (secure 0 0 0) 0x7efcf40b1be0 con 0x7efd001094a0 2024-09-21T06:36:24.103 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:36:24.104 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":85,"fsid":"2a791768-77e2-11ef-baf5-efdc52797490","created":"2024-09-21T06:26:15.222283+0000","modified":"2024-09-21T06:36:05.617384+0000","last_up_change":"2024-09-21T06:33:40.886310+0000","last_in_change":"2024-09-21T06:33:17.339486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:29:50.212825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:34:29.082881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:34:31.301094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:34:37.681356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"18778364-82ed-41c0-92da-ce44cdad7358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6803","nonce":2310759946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6805","nonce":2310759946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6809","nonce":2310759946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6807","nonce":2310759946}]},"public_addr":"172.21.15.67:6803/2310759946","cluster_addr":"172.21.15.67:6805/2310759946","heartbeat_back_addr":"172.21.15.67:6809/2310759946","heartbeat_front_addr":"172.21.15.67:6807/2310759946","state":["exists","up"]},{"osd":1,"uuid":"84e8d0c2-1c45-4673-ad49-f61df667d7f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6801","nonce":161807364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6803","nonce":161807364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6807","nonce":161807364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6805","nonce":161807364}]},"public_addr":"172.21.15.100:6801/161807364","cluster_addr":"172.21.15.100:6803/161807364","heartbeat_back_addr":"172.21.15.100:6807/161807364","heartbeat_front_addr":"172.21.15.100:6805/161807364","state":["exists","up"]},{"osd":2,"uuid":"e9a4d919-a779-4a1b-9b5b-d03a65f28ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6801","nonce":2754787169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6803","nonce":2754787169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6807","nonce":2754787169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6805","nonce":2754787169}]},"public_addr":"172.21.15.203:6801/2754787169","cluster_addr":"172.21.15.203:6803/2754787169","heartbeat_back_addr":"172.21.15.203:6807/2754787169","heartbeat_front_addr":"172.21.15.203:6805/2754787169","state":["exists","up"]},{"osd":3,"uuid":"3f7d9abd-587c-47ad-bf43-11f7dcdf71c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6811","nonce":4007517951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6813","nonce":4007517951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6817","nonce":4007517951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6815","nonce":4007517951}]},"public_addr":"172.21.15.67:6811/4007517951","cluster_addr":"172.21.15.67:6813/4007517951","heartbeat_back_addr":"172.21.15.67:6817/4007517951","heartbeat_front_addr":"172.21.15.67:6815/4007517951","state":["exists","up"]},{"osd":4,"uuid":"94f3a075-6d8a-4907-8418-a8513b35779a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6809","nonce":3773646501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6811","nonce":3773646501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6815","nonce":3773646501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6813","nonce":3773646501}]},"public_addr":"172.21.15.100:6809/3773646501","cluster_addr":"172.21.15.100:6811/3773646501","heartbeat_back_addr":"172.21.15.100:6815/3773646501","heartbeat_front_addr":"172.21.15.100:6813/3773646501","state":["exists","up"]},{"osd":5,"uuid":"5d842edf-06ad-49ae-ac71-863e5657270c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6809","nonce":2096171166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6811","nonce":2096171166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6815","nonce":2096171166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6813","nonce":2096171166}]},"public_addr":"172.21.15.203:6809/2096171166","cluster_addr":"172.21.15.203:6811/2096171166","heartbeat_back_addr":"172.21.15.203:6815/2096171166","heartbeat_front_addr":"172.21.15.203:6813/2096171166","state":["exists","up"]},{"osd":6,"uuid":"85b9beed-c93d-48ba-bc7a-c4cd816fbdff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6819","nonce":3229936280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6821","nonce":3229936280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6825","nonce":3229936280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6823","nonce":3229936280}]},"public_addr":"172.21.15.67:6819/3229936280","cluster_addr":"172.21.15.67:6821/3229936280","heartbeat_back_addr":"172.21.15.67:6825/3229936280","heartbeat_front_addr":"172.21.15.67:6823/3229936280","state":["exists","up"]},{"osd":7,"uuid":"86de30c2-b8ee-4c9d-9c17-8a56517934ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6817","nonce":1622917909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6819","nonce":1622917909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6823","nonce":1622917909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6821","nonce":1622917909}]},"public_addr":"172.21.15.100:6817/1622917909","cluster_addr":"172.21.15.100:6819/1622917909","heartbeat_back_addr":"172.21.15.100:6823/1622917909","heartbeat_front_addr":"172.21.15.100:6821/1622917909","state":["exists","up"]},{"osd":8,"uuid":"0e96cdad-a135-4115-9e2d-9a28306700e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6817","nonce":3452445943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6819","nonce":3452445943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6823","nonce":3452445943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6821","nonce":3452445943}]},"public_addr":"172.21.15.203:6817/3452445943","cluster_addr":"172.21.15.203:6819/3452445943","heartbeat_back_addr":"172.21.15.203:6823/3452445943","heartbeat_front_addr":"172.21.15.203:6821/3452445943","state":["exists","up"]},{"osd":9,"uuid":"95704ed6-6286-4094-9c6d-23919c7005ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6827","nonce":2998307533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6829","nonce":2998307533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6833","nonce":2998307533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6831","nonce":2998307533}]},"public_addr":"172.21.15.67:6827/2998307533","cluster_addr":"172.21.15.67:6829/2998307533","heartbeat_back_addr":"172.21.15.67:6833/2998307533","heartbeat_front_addr":"172.21.15.67:6831/2998307533","state":["exists","up"]},{"osd":10,"uuid":"bd2a7d33-b935-46c8-95c3-d76f28242ffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6825","nonce":2819057142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6827","nonce":2819057142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6831","nonce":2819057142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6829","nonce":2819057142}]},"public_addr":"172.21.15.100:6825/2819057142","cluster_addr":"172.21.15.100:6827/2819057142","heartbeat_back_addr":"172.21.15.100:6831/2819057142","heartbeat_front_addr":"172.21.15.100:6829/2819057142","state":["exists","up"]},{"osd":11,"uuid":"ea591507-10e2-4c08-8d29-d38ba9be96ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6825","nonce":566502600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6827","nonce":566502600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6831","nonce":566502600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6829","nonce":566502600}]},"public_addr":"172.21.15.203:6825/566502600","cluster_addr":"172.21.15.203:6827/566502600","heartbeat_back_addr":"172.21.15.203:6831/566502600","heartbeat_front_addr":"172.21.15.203:6829/566502600","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:28:58.945733+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:29:23.597118+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:29:48.303563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:30:14.139139+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:30:39.021508+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:04.981241+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:30.853841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:56.452967+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:32:22.239036+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:32:47.998803+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:33:13.850759+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:33:39.282150+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6800/1181476774":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/3100321138":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/2309364298":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/3458717076":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/4110206171":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6801/1181476774":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/2910038982":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6801/2622729103":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6800/2622729103":"2024-09-22T06:26:37.041377+0000","172.21.15.67:0/3346784528":"2024-09-22T06:26:37.041377+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:36:24.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:24.104+0000 7efd08274640 1 -- 172.21.15.67:0/2978139276 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7efce80b65d0 msgr2=0x7efce80b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:24.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:24.104+0000 7efd08274640 1 --2- 172.21.15.67:0/2978139276 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7efce80b65d0 0x7efce80b8ae0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efcf800abe0 tx=0x7efcf800a2d0 comp rx=0 tx=0).stop 2024-09-21T06:36:24.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:24.104+0000 7efd08274640 1 -- 172.21.15.67:0/2978139276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd001094a0 msgr2=0x7efd0019aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:24.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:24.104+0000 7efd08274640 1 --2- 172.21.15.67:0/2978139276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd001094a0 0x7efd0019aa60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7efcf400d750 tx=0x7efcf400db50 comp rx=0 tx=0).stop 2024-09-21T06:36:24.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:24.104+0000 7efd08274640 1 -- 172.21.15.67:0/2978139276 shutdown_connections 2024-09-21T06:36:24.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:24.104+0000 7efd08274640 1 --2- 172.21.15.67:0/2978139276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd00109df0 0x7efd0019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:24.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:24.104+0000 7efd08274640 1 --2- 172.21.15.67:0/2978139276 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7efce80b65d0 0x7efce80b8ae0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:24.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:24.104+0000 7efd08274640 1 --2- 172.21.15.67:0/2978139276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd001094a0 0x7efd0019aa60 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:24.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:24.104+0000 7efd08274640 1 --2- 172.21.15.67:0/2978139276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd00103b60 0x7efd0019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:24.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:24.104+0000 7efd08274640 1 -- 172.21.15.67:0/2978139276 >> 172.21.15.67:0/2978139276 conn(0x7efd0007b850 msgr2=0x7efd00107690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:24.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:24.104+0000 7efd08274640 1 -- 172.21.15.67:0/2978139276 shutdown_connections 2024-09-21T06:36:24.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:24.104+0000 7efd08274640 1 -- 172.21.15.67:0/2978139276 wait complete. 2024-09-21T06:36:24.825 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:36:25.082 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:36:25.203 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:25 smithi067 ceph-mon[31232]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:36:25.204 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:25 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2978139276' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:36:25.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:25 smithi100 ceph-mon[33401]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:36:25.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:25 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2978139276' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:36:25.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:25 smithi203 ceph-mon[33735]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:36:25.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:25 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2978139276' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:36:25.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.599+0000 7fb11451e640 1 -- 172.21.15.67:0/4214633614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb10c108d20 msgr2=0x7fb10c10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:25.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.599+0000 7fb11451e640 1 --2- 172.21.15.67:0/4214633614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb10c108d20 0x7fb10c10f550 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009ca0 tx=0x7fb0fc02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:36:25.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.599+0000 7fb11451e640 1 -- 172.21.15.67:0/4214633614 shutdown_connections 2024-09-21T06:36:25.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.599+0000 7fb11451e640 1 --2- 172.21.15.67:0/4214633614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb10c108d20 0x7fb10c10f550 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:25.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.599+0000 7fb11451e640 1 --2- 172.21.15.67:0/4214633614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb10c1083d0 0x7fb10c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:25.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.599+0000 7fb11451e640 1 --2- 172.21.15.67:0/4214633614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb10c102ae0 0x7fb10c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:25.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.599+0000 7fb11451e640 1 -- 172.21.15.67:0/4214633614 >> 172.21.15.67:0/4214633614 conn(0x7fb10c079760 msgr2=0x7fb10c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:25.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.599+0000 7fb11451e640 1 -- 172.21.15.67:0/4214633614 shutdown_connections 2024-09-21T06:36:25.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.599+0000 7fb11451e640 1 -- 172.21.15.67:0/4214633614 wait complete. 2024-09-21T06:36:25.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.600+0000 7fb11451e640 1 Processor -- start 2024-09-21T06:36:25.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.600+0000 7fb11451e640 1 -- start start 2024-09-21T06:36:25.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.600+0000 7fb11451e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb10c102ae0 0x7fb10c19b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:25.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.600+0000 7fb11451e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb10c1083d0 0x7fb10c19b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:25.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.600+0000 7fb11451e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb10c108d20 0x7fb10c19bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:25.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.600+0000 7fb11451e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb10c112910 con 0x7fb10c102ae0 2024-09-21T06:36:25.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.600+0000 7fb11451e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb10c112790 con 0x7fb10c108d20 2024-09-21T06:36:25.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.600+0000 7fb11451e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb10c112a90 con 0x7fb10c1083d0 2024-09-21T06:36:25.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.600+0000 7fb1122ba640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb10c102ae0 0x7fb10c19b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:25.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.600+0000 7fb1122ba640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb10c102ae0 0x7fb10c19b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59394/0 (socket says 172.21.15.67:59394) 2024-09-21T06:36:25.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.600+0000 7fb1122ba640 1 -- 172.21.15.67:0/3807968348 learned_addr learned my addr 172.21.15.67:0/3807968348 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:25.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.601+0000 7fb111ab9640 1 --2- 172.21.15.67:0/3807968348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb10c1083d0 0x7fb10c19b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:25.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.601+0000 7fb1122ba640 1 -- 172.21.15.67:0/3807968348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb10c1083d0 msgr2=0x7fb10c19b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:25.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.601+0000 7fb1122ba640 1 --2- 172.21.15.67:0/3807968348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb10c1083d0 0x7fb10c19b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:25.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.601+0000 7fb1122ba640 1 -- 172.21.15.67:0/3807968348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb10c108d20 msgr2=0x7fb10c19bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:25.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.601+0000 7fb1122ba640 1 --2- 172.21.15.67:0/3807968348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb10c108d20 0x7fb10c19bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:25.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.601+0000 7fb1122ba640 1 -- 172.21.15.67:0/3807968348 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb10c1a6430 con 0x7fb10c102ae0 2024-09-21T06:36:25.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.601+0000 7fb1122ba640 1 --2- 172.21.15.67:0/3807968348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb10c102ae0 0x7fb10c19b1a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb10400b4a0 tx=0x7fb10400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:25.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.601+0000 7fb10b7fe640 1 -- 172.21.15.67:0/3807968348 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb104014070 con 0x7fb10c102ae0 2024-09-21T06:36:25.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.601+0000 7fb10b7fe640 1 -- 172.21.15.67:0/3807968348 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb104010aa0 con 0x7fb10c102ae0 2024-09-21T06:36:25.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.601+0000 7fb11451e640 1 -- 172.21.15.67:0/3807968348 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb10c1a6720 con 0x7fb10c102ae0 2024-09-21T06:36:25.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.601+0000 7fb10b7fe640 1 -- 172.21.15.67:0/3807968348 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb104017b70 con 0x7fb10c102ae0 2024-09-21T06:36:25.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.601+0000 7fb11451e640 1 -- 172.21.15.67:0/3807968348 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb10c1a6c60 con 0x7fb10c102ae0 2024-09-21T06:36:25.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.602+0000 7fb11451e640 1 -- 172.21.15.67:0/3807968348 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb10c103d10 con 0x7fb10c102ae0 2024-09-21T06:36:25.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.604+0000 7fb10b7fe640 1 -- 172.21.15.67:0/3807968348 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fb104017070 con 0x7fb10c102ae0 2024-09-21T06:36:25.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.604+0000 7fb10b7fe640 1 --2- 172.21.15.67:0/3807968348 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb0f40b65d0 0x7fb0f40b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:25.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.605+0000 7fb111ab9640 1 --2- 172.21.15.67:0/3807968348 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb0f40b65d0 0x7fb0f40b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:25.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.605+0000 7fb10b7fe640 1 -- 172.21.15.67:0/3807968348 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fb1040e6e50 con 0x7fb10c102ae0 2024-09-21T06:36:25.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.605+0000 7fb111ab9640 1 --2- 172.21.15.67:0/3807968348 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb0f40b65d0 0x7fb0f40b8ae0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb10c107040 tx=0x7fb10000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:25.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.608+0000 7fb10b7fe640 1 -- 172.21.15.67:0/3807968348 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb1040ad1b0 con 0x7fb10c102ae0 2024-09-21T06:36:25.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.834+0000 7fb11451e640 1 -- 172.21.15.67:0/3807968348 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb10c06bf50 con 0x7fb10c102ae0 2024-09-21T06:36:25.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.835+0000 7fb10b7fe640 1 -- 172.21.15.67:0/3807968348 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7fb1040a8070 con 0x7fb10c102ae0 2024-09-21T06:36:25.836 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:36:25.836 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:36:25.836 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:36:25.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.837+0000 7fb11451e640 1 -- 172.21.15.67:0/3807968348 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb0f40b65d0 msgr2=0x7fb0f40b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:25.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.837+0000 7fb11451e640 1 --2- 172.21.15.67:0/3807968348 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb0f40b65d0 0x7fb0f40b8ae0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb10c107040 tx=0x7fb10000b390 comp rx=0 tx=0).stop 2024-09-21T06:36:25.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.837+0000 7fb11451e640 1 -- 172.21.15.67:0/3807968348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb10c102ae0 msgr2=0x7fb10c19b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:25.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.837+0000 7fb11451e640 1 --2- 172.21.15.67:0/3807968348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb10c102ae0 0x7fb10c19b1a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb10400b4a0 tx=0x7fb10400b960 comp rx=0 tx=0).stop 2024-09-21T06:36:25.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.837+0000 7fb11451e640 1 -- 172.21.15.67:0/3807968348 shutdown_connections 2024-09-21T06:36:25.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.837+0000 7fb11451e640 1 --2- 172.21.15.67:0/3807968348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb10c108d20 0x7fb10c19bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:25.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.837+0000 7fb11451e640 1 --2- 172.21.15.67:0/3807968348 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb0f40b65d0 0x7fb0f40b8ae0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:25.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.837+0000 7fb11451e640 1 --2- 172.21.15.67:0/3807968348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb10c1083d0 0x7fb10c19b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:25.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.837+0000 7fb11451e640 1 --2- 172.21.15.67:0/3807968348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb10c102ae0 0x7fb10c19b1a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:25.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.837+0000 7fb11451e640 1 -- 172.21.15.67:0/3807968348 >> 172.21.15.67:0/3807968348 conn(0x7fb10c079760 msgr2=0x7fb10c105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:25.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.837+0000 7fb11451e640 1 -- 172.21.15.67:0/3807968348 shutdown_connections 2024-09-21T06:36:25.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:25.837+0000 7fb11451e640 1 -- 172.21.15.67:0/3807968348 wait complete. 2024-09-21T06:36:26.147 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:26 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3807968348' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:36:26.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:26 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3807968348' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:36:26.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:26 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3807968348' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:36:26.567 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T06:34:33.306959+0000', 'modified': '2024-09-21T06:35:09.805211+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24671}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24641': {'gid': 24641, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.100:6837/259395872', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6836', 'nonce': 259395872}, {'type': 'v1', 'addr': '172.21.15.100:6837', 'nonce': 259395872}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24671': {'gid': 24671, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.100:6839/778060936', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6838', 'nonce': 778060936}, {'type': 'v1', 'addr': '172.21.15.100:6839', 'nonce': 778060936}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24671, 'qdb_cluster': [24671]} 2024-09-21T06:36:26.567 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T06:36:26.567 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T06:36:26.827 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:36:27.198 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:27 smithi067 ceph-mon[31232]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:36:27.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.286+0000 7fa874ed0640 1 -- 172.21.15.67:0/3396534398 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa87010de70 msgr2=0x7fa870110340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:27.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.286+0000 7fa874ed0640 1 --2- 172.21.15.67:0/3396534398 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa87010de70 0x7fa870110340 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa868009ca0 tx=0x7fa86802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.286+0000 7fa874ed0640 1 -- 172.21.15.67:0/3396534398 shutdown_connections 2024-09-21T06:36:27.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.286+0000 7fa874ed0640 1 --2- 172.21.15.67:0/3396534398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8701052a0 0x7fa87010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.286+0000 7fa874ed0640 1 --2- 172.21.15.67:0/3396534398 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa87010de70 0x7fa870110340 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.286+0000 7fa874ed0640 1 --2- 172.21.15.67:0/3396534398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8701048c0 0x7fa870104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.286+0000 7fa874ed0640 1 -- 172.21.15.67:0/3396534398 >> 172.21.15.67:0/3396534398 conn(0x7fa870079760 msgr2=0x7fa870079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:27.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.286+0000 7fa874ed0640 1 -- 172.21.15.67:0/3396534398 shutdown_connections 2024-09-21T06:36:27.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.286+0000 7fa874ed0640 1 -- 172.21.15.67:0/3396534398 wait complete. 2024-09-21T06:36:27.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.287+0000 7fa874ed0640 1 Processor -- start 2024-09-21T06:36:27.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.288+0000 7fa874ed0640 1 -- start start 2024-09-21T06:36:27.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.288+0000 7fa874ed0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8701048c0 0x7fa8701027d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:27.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.288+0000 7fa874ed0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8701052a0 0x7fa870102d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:27.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.288+0000 7fa874ed0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa87010de70 0x7fa870103250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:27.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.288+0000 7fa874ed0640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa8701139d0 con 0x7fa8701048c0 2024-09-21T06:36:27.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.288+0000 7fa874ed0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa870113850 con 0x7fa8701052a0 2024-09-21T06:36:27.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.288+0000 7fa874ed0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa870113b50 con 0x7fa87010de70 2024-09-21T06:36:27.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.288+0000 7fa86e59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8701048c0 0x7fa8701027d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:27.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.288+0000 7fa86e59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8701048c0 0x7fa8701027d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59410/0 (socket says 172.21.15.67:59410) 2024-09-21T06:36:27.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.288+0000 7fa86e59c640 1 -- 172.21.15.67:0/3369034008 learned_addr learned my addr 172.21.15.67:0/3369034008 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:27.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.288+0000 7fa865d9b640 1 --2- 172.21.15.67:0/3369034008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8701052a0 0x7fa870102d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:27.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.288+0000 7fa86ed9d640 1 --2- 172.21.15.67:0/3369034008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa87010de70 0x7fa870103250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:27.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.288+0000 7fa86e59c640 1 -- 172.21.15.67:0/3369034008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa87010de70 msgr2=0x7fa870103250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:27.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.289+0000 7fa86e59c640 1 --2- 172.21.15.67:0/3369034008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa87010de70 0x7fa870103250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.289+0000 7fa86e59c640 1 -- 172.21.15.67:0/3369034008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8701052a0 msgr2=0x7fa870102d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:27.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.289+0000 7fa86e59c640 1 --2- 172.21.15.67:0/3369034008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8701052a0 0x7fa870102d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.289+0000 7fa86e59c640 1 -- 172.21.15.67:0/3369034008 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8701a86c0 con 0x7fa8701048c0 2024-09-21T06:36:27.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.289+0000 7fa86e59c640 1 --2- 172.21.15.67:0/3369034008 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8701048c0 0x7fa8701027d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa86000e9c0 tx=0x7fa86000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:27.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.289+0000 7fa867fff640 1 -- 172.21.15.67:0/3369034008 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86000cd80 con 0x7fa8701048c0 2024-09-21T06:36:27.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.289+0000 7fa874ed0640 1 -- 172.21.15.67:0/3369034008 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8701a89b0 con 0x7fa8701048c0 2024-09-21T06:36:27.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.289+0000 7fa867fff640 1 -- 172.21.15.67:0/3369034008 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8600035f0 con 0x7fa8701048c0 2024-09-21T06:36:27.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.289+0000 7fa867fff640 1 -- 172.21.15.67:0/3369034008 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8600044c0 con 0x7fa8701048c0 2024-09-21T06:36:27.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.289+0000 7fa874ed0640 1 -- 172.21.15.67:0/3369034008 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8701a8ef0 con 0x7fa8701048c0 2024-09-21T06:36:27.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.291+0000 7fa87b444740 1 Processor -- start 2024-09-21T06:36:27.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.292+0000 7fa87b444740 1 -- start start 2024-09-21T06:36:27.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.292+0000 7fa87b444740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5621181fbf60 0x5621185a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:27.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.292+0000 7fa87b444740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621181d30a0 0x5621185a2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:27.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.292+0000 7fa87b444740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5621181db280 0x5621185a2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:27.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.292+0000 7fa87b444740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x562118519c10 con 0x5621181fbf60 2024-09-21T06:36:27.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.292+0000 7fa87b444740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562118509040 con 0x5621181d30a0 2024-09-21T06:36:27.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.292+0000 7fa87b444740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56211850ce50 con 0x5621181db280 2024-09-21T06:36:27.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.292+0000 7fa86ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5621181fbf60 0x5621185a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:27.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.292+0000 7fa86ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5621181fbf60 0x5621185a1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59414/0 (socket says 172.21.15.67:59414) 2024-09-21T06:36:27.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.292+0000 7fa86ed9d640 1 -- 172.21.15.67:0/2437531853 learned_addr learned my addr 172.21.15.67:0/2437531853 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:27.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.292+0000 7fa867fff640 1 -- 172.21.15.67:0/3369034008 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fa860004030 con 0x7fa8701048c0 2024-09-21T06:36:27.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.292+0000 7fa865d9b640 1 --2- 172.21.15.67:0/2437531853 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5621181db280 0x5621185a2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:27.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.293+0000 7fa86ed9d640 1 -- 172.21.15.67:0/2437531853 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5621181db280 msgr2=0x5621185a2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:36:27.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.293+0000 7fa86ed9d640 1 --2- 172.21.15.67:0/2437531853 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5621181db280 0x5621185a2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.293+0000 7fa86ed9d640 1 -- 172.21.15.67:0/2437531853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621181d30a0 msgr2=0x5621185a2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:36:27.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.293+0000 7fa86ed9d640 1 --2- 172.21.15.67:0/2437531853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621181d30a0 0x5621185a2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.293+0000 7fa86ed9d640 1 -- 172.21.15.67:0/2437531853 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562118267540 con 0x5621181fbf60 2024-09-21T06:36:27.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.293+0000 7fa86ed9d640 1 --2- 172.21.15.67:0/2437531853 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5621181fbf60 0x5621185a1f00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa868005cc0 tx=0x7fa868009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b824b9f685c9aed4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:27.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.293+0000 7fa864d99640 1 -- 172.21.15.67:0/2437531853 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa868031750 con 0x5621181fbf60 2024-09-21T06:36:27.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.293+0000 7fa864d99640 1 -- 172.21.15.67:0/2437531853 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa868036040 con 0x5621181fbf60 2024-09-21T06:36:27.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.293+0000 7fa864d99640 1 -- 172.21.15.67:0/2437531853 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8680042a0 con 0x5621181fbf60 2024-09-21T06:36:27.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.293+0000 7fa867fff640 1 --2- 172.21.15.67:0/3369034008 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fa85c0b65a0 0x7fa85c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:27.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.294+0000 7fa86e59c640 1 --2- 172.21.15.67:0/3369034008 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fa85c0b65a0 0x7fa85c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:27.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.294+0000 7fa867fff640 1 -- 172.21.15.67:0/3369034008 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fa8600e7cb0 con 0x7fa8701048c0 2024-09-21T06:36:27.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.294+0000 7fa87b444740 1 -- 172.21.15.67:0/2437531853 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5621181fbf60 msgr2=0x5621185a1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:27.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.294+0000 7fa86e59c640 1 --2- 172.21.15.67:0/3369034008 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fa85c0b65a0 0x7fa85c0b8ab0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x5621185a0b80 tx=0x7fa86007a610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:27.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.294+0000 7fa87b444740 1 --2- 172.21.15.67:0/2437531853 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5621181fbf60 0x5621185a1f00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa868005cc0 tx=0x7fa868009a20 comp rx=0 tx=0).stop 2024-09-21T06:36:27.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.294+0000 7fa87b444740 1 -- 172.21.15.67:0/2437531853 shutdown_connections 2024-09-21T06:36:27.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.294+0000 7fa87b444740 1 --2- 172.21.15.67:0/2437531853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621181d30a0 0x5621185a2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.294+0000 7fa87b444740 1 --2- 172.21.15.67:0/2437531853 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5621181db280 0x5621185a2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.294+0000 7fa87b444740 1 --2- 172.21.15.67:0/2437531853 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5621181fbf60 0x5621185a1f00 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.294+0000 7fa87b444740 1 -- 172.21.15.67:0/2437531853 >> 172.21.15.67:0/2437531853 conn(0x5621181f0350 msgr2=0x56211859fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:27.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.294+0000 7fa87b444740 1 -- 172.21.15.67:0/2437531853 shutdown_connections 2024-09-21T06:36:27.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.294+0000 7fa87b444740 1 -- 172.21.15.67:0/2437531853 wait complete. 2024-09-21T06:36:27.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.295+0000 7fa87b444740 1 -- start start 2024-09-21T06:36:27.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.295+0000 7fa87b444740 1 Processor -- start 2024-09-21T06:36:27.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.295+0000 7fa87b444740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5621181d30a0 0x5621185ad7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:27.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.295+0000 7fa87b444740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5621181db280 0x5621185add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:27.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.295+0000 7fa87b444740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621181fbf60 0x5621185b99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:27.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.295+0000 7fa87b444740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56211836a5f0 con 0x5621181db280 2024-09-21T06:36:27.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.295+0000 7fa87b444740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5621184a4010 con 0x5621181fbf60 2024-09-21T06:36:27.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.295+0000 7fa87b444740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5621185b0680 con 0x5621181d30a0 2024-09-21T06:36:27.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.295+0000 7fa87b444740 10 client.0 _ll_register_callbacks cb 0x562118503340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:36:27.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.295+0000 7fa865d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5621181db280 0x5621185add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:27.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.295+0000 7fa865d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5621181db280 0x5621185add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59424/0 (socket says 172.21.15.67:59424) 2024-09-21T06:36:27.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.295+0000 7fa865d9b640 1 -- 172.21.15.67:0/3988916772 learned_addr learned my addr 172.21.15.67:0/3988916772 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:27.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.296+0000 7fa86ed9d640 1 --2- 172.21.15.67:0/3988916772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621181fbf60 0x5621185b99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:27.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.296+0000 7fa86ed9d640 1 --2- 172.21.15.67:0/3988916772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5621181d30a0 0x5621185ad7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:27.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.296+0000 7fa865d9b640 1 -- 172.21.15.67:0/3988916772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5621181d30a0 msgr2=0x5621185ad7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:36:27.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.296+0000 7fa865d9b640 1 --2- 172.21.15.67:0/3988916772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5621181d30a0 0x5621185ad7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.296+0000 7fa865d9b640 1 -- 172.21.15.67:0/3988916772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621181fbf60 msgr2=0x5621185b99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:36:27.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.296+0000 7fa865d9b640 1 --2- 172.21.15.67:0/3988916772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621181fbf60 0x5621185b99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.296+0000 7fa865d9b640 1 -- 172.21.15.67:0/3988916772 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621184d1a00 con 0x5621181db280 2024-09-21T06:36:27.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.296+0000 7fa86ed9d640 1 --2- 172.21.15.67:0/3988916772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5621181d30a0 0x5621185ad7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:36:27.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.296+0000 7fa865d9b640 1 --2- 172.21.15.67:0/3988916772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5621181db280 0x5621185add80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa85800b9f0 tx=0x7fa85800beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63da794329360509 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:27.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.296+0000 7fa855ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:36:27.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.296+0000 7fa855ffb640 1 -- 172.21.15.67:0/3988916772 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa858003e20 con 0x5621181db280 2024-09-21T06:36:27.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.296+0000 7fa855ffb640 1 -- 172.21.15.67:0/3988916772 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fa858005020 con 0x5621181db280 2024-09-21T06:36:27.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.296+0000 7fa87b444740 1 -- 172.21.15.67:0/3988916772 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa868036040 con 0x5621181db280 2024-09-21T06:36:27.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.297+0000 7fa855ffb640 1 -- 172.21.15.67:0/3988916772 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa858005220 con 0x5621181db280 2024-09-21T06:36:27.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.297+0000 7fa87b444740 10 client.15282 fetch_fsmap learned FSMap version 21 2024-09-21T06:36:27.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.297+0000 7fa87b444740 1 -- 172.21.15.67:0/3988916772 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5621185bc5d0 con 0x5621181db280 2024-09-21T06:36:27.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.297+0000 7fa855ffb640 1 -- 172.21.15.67:0/3988916772 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa868036040 con 0x5621181db280 2024-09-21T06:36:27.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.297+0000 7fa855ffb640 1 -- 172.21.15.67:0/3988916772 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa85801b470 con 0x5621181db280 2024-09-21T06:36:27.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.297+0000 7fa87b444740 10 client.15282 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:36:27.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.297+0000 7fa87b444740 10 client.15282 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:36:27.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.297+0000 7fa87b444740 20 client.15282 populate_metadata read hostname 'smithi067' 2024-09-21T06:36:27.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.297+0000 7fa87b444740 1 --2- 172.21.15.67:0/3988916772 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x5621181ba8e0 0x5621185c0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:27.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.297+0000 7fa87b444740 4 client.15282 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:36:27.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.297+0000 7fa87b444740 1 -- 172.21.15.67:0/3988916772 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56211851bb60 con 0x5621181ba8e0 2024-09-21T06:36:27.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.298+0000 7fa86ed9d640 1 --2- 172.21.15.67:0/3988916772 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x5621181ba8e0 0x5621185c0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:27.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.298+0000 7fa86ed9d640 1 --2- 172.21.15.67:0/3988916772 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x5621181ba8e0 0x5621185c0510 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4181d95a62249f server_cookie=31bc37bab0ed41b9 in_seq=0 out_seq=0 2024-09-21T06:36:27.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.298+0000 7fa855ffb640 10 client.15282 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:36:27.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.299+0000 7fa855ffb640 1 -- 172.21.15.67:0/3988916772 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56211851bb60 con 0x5621181ba8e0 2024-09-21T06:36:27.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.299+0000 7fa855ffb640 10 client.15282 handle_command_reply: tid=0 2024-09-21T06:36:27.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.299+0000 7fa87b444740 1 client.15282 shutdown 2024-09-21T06:36:27.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.299+0000 7fa87b444740 1 -- 172.21.15.67:0/3988916772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5621181db280 msgr2=0x5621185add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:27.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.299+0000 7fa87b444740 1 --2- 172.21.15.67:0/3988916772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5621181db280 0x5621185add80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa85800b9f0 tx=0x7fa85800beb0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.299+0000 7fa87b444740 1 -- 172.21.15.67:0/3988916772 shutdown_connections 2024-09-21T06:36:27.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.299+0000 7fa87b444740 1 --2- 172.21.15.67:0/3988916772 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x5621181ba8e0 0x5621185c0510 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.299+0000 7fa87b444740 1 --2- 172.21.15.67:0/3988916772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621181fbf60 0x5621185b99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.299+0000 7fa87b444740 1 --2- 172.21.15.67:0/3988916772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5621181db280 0x5621185add80 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.299+0000 7fa855ffb640 0 client.15282 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:36:27.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.299+0000 7fa87b444740 1 --2- 172.21.15.67:0/3988916772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5621181d30a0 0x5621185ad7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.299+0000 7fa87b444740 1 -- 172.21.15.67:0/3988916772 >> 172.21.15.67:0/3988916772 conn(0x5621181f0350 msgr2=0x5621185a50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:27.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.299+0000 7fa87b444740 1 -- 172.21.15.67:0/3988916772 shutdown_connections 2024-09-21T06:36:27.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.299+0000 7fa87b444740 1 -- 172.21.15.67:0/3988916772 wait complete. 2024-09-21T06:36:27.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.299+0000 7fa87b444740 20 client.15282 trim_cache size 0 max 16384 2024-09-21T06:36:27.307 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:27 smithi100 ceph-mon[33401]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:36:27.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.317+0000 7fa87b444740 1 Processor -- start 2024-09-21T06:36:27.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.318+0000 7fa87b444740 1 -- start start 2024-09-21T06:36:27.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.318+0000 7fa87b444740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5621181d30a0 0x5621185a8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:27.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.318+0000 7fa87b444740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5621181db280 0x5621185b9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:27.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.318+0000 7fa87b444740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621181fbf60 0x5621185b9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:27.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.318+0000 7fa87b444740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5621185be850 con 0x5621181d30a0 2024-09-21T06:36:27.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.318+0000 7fa87b444740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5621185bef20 con 0x5621181fbf60 2024-09-21T06:36:27.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.318+0000 7fa87b444740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5621185bf5f0 con 0x5621181db280 2024-09-21T06:36:27.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.318+0000 7fa86ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5621181d30a0 0x5621185a8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:27.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.318+0000 7fa86ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5621181d30a0 0x5621185a8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59428/0 (socket says 172.21.15.67:59428) 2024-09-21T06:36:27.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.318+0000 7fa86ed9d640 1 -- 172.21.15.67:0/3204691644 learned_addr learned my addr 172.21.15.67:0/3204691644 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:27.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.318+0000 7fa865d9b640 1 --2- 172.21.15.67:0/3204691644 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5621181db280 0x5621185b9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:27.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.318+0000 7fa86ed9d640 1 --2- 172.21.15.67:0/3204691644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621181fbf60 0x5621185b9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:27.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.318+0000 7fa86ed9d640 1 -- 172.21.15.67:0/3204691644 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5621181db280 msgr2=0x5621185b9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:36:27.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.318+0000 7fa86ed9d640 1 --2- 172.21.15.67:0/3204691644 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5621181db280 0x5621185b9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.318+0000 7fa86ed9d640 1 -- 172.21.15.67:0/3204691644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621181fbf60 msgr2=0x5621185b9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:36:27.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.318+0000 7fa86ed9d640 1 --2- 172.21.15.67:0/3204691644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621181fbf60 0x5621185b9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.319+0000 7fa86ed9d640 1 -- 172.21.15.67:0/3204691644 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621185aea60 con 0x5621181d30a0 2024-09-21T06:36:27.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.319+0000 7fa86ed9d640 1 --2- 172.21.15.67:0/3204691644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5621181d30a0 0x5621185a8020 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa868003e00 tx=0x7fa868003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c4206faf4bc0fe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:27.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.319+0000 7fa857fff640 1 -- 172.21.15.67:0/3204691644 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa868032c10 con 0x5621181d30a0 2024-09-21T06:36:27.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.319+0000 7fa857fff640 1 -- 172.21.15.67:0/3204691644 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa868036070 con 0x5621181d30a0 2024-09-21T06:36:27.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.319+0000 7fa857fff640 1 -- 172.21.15.67:0/3204691644 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8680305c0 con 0x5621181d30a0 2024-09-21T06:36:27.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.320+0000 7fa87b444740 1 -- 172.21.15.67:0/3204691644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5621181d30a0 msgr2=0x5621185a8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:27.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.320+0000 7fa87b444740 1 --2- 172.21.15.67:0/3204691644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5621181d30a0 0x5621185a8020 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa868003e00 tx=0x7fa868003e30 comp rx=0 tx=0).stop 2024-09-21T06:36:27.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.320+0000 7fa87b444740 1 -- 172.21.15.67:0/3204691644 shutdown_connections 2024-09-21T06:36:27.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.320+0000 7fa87b444740 1 --2- 172.21.15.67:0/3204691644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621181fbf60 0x5621185b9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.320+0000 7fa87b444740 1 --2- 172.21.15.67:0/3204691644 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5621181db280 0x5621185b9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.320+0000 7fa87b444740 1 --2- 172.21.15.67:0/3204691644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5621181d30a0 0x5621185a8020 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.320+0000 7fa87b444740 1 -- 172.21.15.67:0/3204691644 >> 172.21.15.67:0/3204691644 conn(0x5621181f0350 msgr2=0x5621185ac020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:27.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.320+0000 7fa87b444740 1 -- 172.21.15.67:0/3204691644 shutdown_connections 2024-09-21T06:36:27.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.320+0000 7fa87b444740 1 -- 172.21.15.67:0/3204691644 wait complete. 2024-09-21T06:36:27.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.320+0000 7fa87b444740 1 -- start start 2024-09-21T06:36:27.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.320+0000 7fa87b444740 1 Processor -- start 2024-09-21T06:36:27.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.321+0000 7fa87b444740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5621181d30a0 0x5621185c1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:27.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.321+0000 7fa87b444740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5621181db280 0x5621185c24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:27.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.321+0000 7fa87b444740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621181fbf60 0x5621185c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:27.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.321+0000 7fa87b444740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5621185c0390 con 0x5621181db280 2024-09-21T06:36:27.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.321+0000 7fa87b444740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5621185bfcc0 con 0x5621181fbf60 2024-09-21T06:36:27.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.321+0000 7fa87b444740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5621185c0a60 con 0x5621181d30a0 2024-09-21T06:36:27.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.321+0000 7fa87b444740 10 client.0 _ll_register_callbacks cb 0x7fa858004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:36:27.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.321+0000 7fa865d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5621181db280 0x5621185c24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:27.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.321+0000 7fa865d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5621181db280 0x5621185c24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59434/0 (socket says 172.21.15.67:59434) 2024-09-21T06:36:27.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.321+0000 7fa865d9b640 1 -- 172.21.15.67:0/4055935580 learned_addr learned my addr 172.21.15.67:0/4055935580 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:27.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.321+0000 7fa86ed9d640 1 --2- 172.21.15.67:0/4055935580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5621181d30a0 0x5621185c1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:27.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.321+0000 7fa86ed9d640 1 --2- 172.21.15.67:0/4055935580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621181fbf60 0x5621185c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:27.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.321+0000 7fa865d9b640 1 -- 172.21.15.67:0/4055935580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5621181d30a0 msgr2=0x5621185c1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:36:27.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.321+0000 7fa865d9b640 1 --2- 172.21.15.67:0/4055935580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5621181d30a0 0x5621185c1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.321+0000 7fa865d9b640 1 -- 172.21.15.67:0/4055935580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621181fbf60 msgr2=0x5621185c73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:36:27.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.321+0000 7fa865d9b640 1 --2- 172.21.15.67:0/4055935580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621181fbf60 0x5621185c73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.321+0000 7fa865d9b640 1 -- 172.21.15.67:0/4055935580 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621185c7a20 con 0x5621181db280 2024-09-21T06:36:27.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.321+0000 7fa865d9b640 1 --2- 172.21.15.67:0/4055935580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5621181db280 0x5621185c24c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa85801b910 tx=0x7fa858004510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783e7ec8bcf344b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:27.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.321+0000 7fa855ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:36:27.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.322+0000 7fa855ffb640 1 -- 172.21.15.67:0/4055935580 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85801bb00 con 0x5621181db280 2024-09-21T06:36:27.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.322+0000 7fa855ffb640 1 -- 172.21.15.67:0/4055935580 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fa85801bca0 con 0x5621181db280 2024-09-21T06:36:27.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.322+0000 7fa87b444740 1 -- 172.21.15.67:0/4055935580 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa868036070 con 0x5621181db280 2024-09-21T06:36:27.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.322+0000 7fa855ffb640 1 -- 172.21.15.67:0/4055935580 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa8580354c0 con 0x5621181db280 2024-09-21T06:36:27.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.322+0000 7fa87b444740 10 client.15288 fetch_fsmap learned FSMap version 21 2024-09-21T06:36:27.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.322+0000 7fa87b444740 1 -- 172.21.15.67:0/4055935580 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5621185c8e90 con 0x5621181db280 2024-09-21T06:36:27.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.322+0000 7fa855ffb640 1 -- 172.21.15.67:0/4055935580 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa868036070 con 0x5621181db280 2024-09-21T06:36:27.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.322+0000 7fa855ffb640 1 -- 172.21.15.67:0/4055935580 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa858004cf0 con 0x5621181db280 2024-09-21T06:36:27.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.322+0000 7fa87b444740 10 client.15288 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:36:27.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.322+0000 7fa87b444740 10 client.15288 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:36:27.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.322+0000 7fa87b444740 20 client.15288 populate_metadata read hostname 'smithi067' 2024-09-21T06:36:27.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.322+0000 7fa87b444740 1 --2- 172.21.15.67:0/4055935580 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x56211818fc90 0x56211862e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:27.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.322+0000 7fa87b444740 4 client.15288 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:36:27.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.322+0000 7fa87b444740 1 -- 172.21.15.67:0/4055935580 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "status"}) -- 0x562118517610 con 0x56211818fc90 2024-09-21T06:36:27.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.323+0000 7fa86ed9d640 1 --2- 172.21.15.67:0/4055935580 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x56211818fc90 0x56211862e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:27.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.323+0000 7fa86ed9d640 1 --2- 172.21.15.67:0/4055935580 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x56211818fc90 0x56211862e9e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1a9da04a5ad0c69 server_cookie=2a9a3ffc3f5521d7 in_seq=0 out_seq=0 2024-09-21T06:36:27.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.323+0000 7fa855ffb640 10 client.15288 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:36:27.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.323+0000 7fa855ffb640 1 -- 172.21.15.67:0/4055935580 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x562118517610 con 0x56211818fc90 2024-09-21T06:36:27.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.323+0000 7fa855ffb640 10 client.15288 handle_command_reply: tid=0 2024-09-21T06:36:27.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.323+0000 7fa87b444740 1 client.15288 shutdown 2024-09-21T06:36:27.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.324+0000 7fa87b444740 1 -- 172.21.15.67:0/4055935580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5621181db280 msgr2=0x5621185c24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:27.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.324+0000 7fa87b444740 1 --2- 172.21.15.67:0/4055935580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5621181db280 0x5621185c24c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa85801b910 tx=0x7fa858004510 comp rx=0 tx=0).stop 2024-09-21T06:36:27.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.324+0000 7fa87b444740 1 -- 172.21.15.67:0/4055935580 shutdown_connections 2024-09-21T06:36:27.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.324+0000 7fa87b444740 1 --2- 172.21.15.67:0/4055935580 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x56211818fc90 0x56211862e9e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.324+0000 7fa87b444740 1 --2- 172.21.15.67:0/4055935580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621181fbf60 0x5621185c73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.324+0000 7fa87b444740 1 --2- 172.21.15.67:0/4055935580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5621181db280 0x5621185c24c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.324+0000 7fa855ffb640 0 client.15288 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:36:27.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.324+0000 7fa87b444740 1 --2- 172.21.15.67:0/4055935580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5621181d30a0 0x5621185c1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.324+0000 7fa87b444740 1 -- 172.21.15.67:0/4055935580 >> 172.21.15.67:0/4055935580 conn(0x5621181f0350 msgr2=0x56211859f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:27.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.324+0000 7fa87b444740 1 -- 172.21.15.67:0/4055935580 shutdown_connections 2024-09-21T06:36:27.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.324+0000 7fa87b444740 1 -- 172.21.15.67:0/4055935580 wait complete. 2024-09-21T06:36:27.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.324+0000 7fa87b444740 20 client.15288 trim_cache size 0 max 16384 2024-09-21T06:36:27.335 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-21T06:36:27.335 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "2a791768-77e2-11ef-baf5-efdc52797490", 2024-09-21T06:36:27.336 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-09-21T06:36:27.336 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24671, 2024-09-21T06:36:27.336 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-09-21T06:36:27.336 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-09-21T06:36:27.336 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-09-21T06:36:27.336 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 113.675239902, 2024-09-21T06:36:27.336 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 21, 2024-09-21T06:36:27.336 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 85, 2024-09-21T06:36:27.336 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 83, 2024-09-21T06:36:27.336 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 125.255147033 2024-09-21T06:36:27.336 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-21T06:36:27.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.324+0000 7fa874ed0640 1 -- 172.21.15.67:0/3369034008 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fa85c0b65a0 msgr2=0x7fa85c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:27.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.324+0000 7fa874ed0640 1 --2- 172.21.15.67:0/3369034008 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fa85c0b65a0 0x7fa85c0b8ab0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x5621185a0b80 tx=0x7fa86007a610 comp rx=0 tx=0).stop 2024-09-21T06:36:27.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.324+0000 7fa874ed0640 1 -- 172.21.15.67:0/3369034008 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8701048c0 msgr2=0x7fa8701027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:27.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.324+0000 7fa874ed0640 1 --2- 172.21.15.67:0/3369034008 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8701048c0 0x7fa8701027d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa86000e9c0 tx=0x7fa86000ee80 comp rx=0 tx=0).stop 2024-09-21T06:36:27.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.324+0000 7fa874ed0640 1 -- 172.21.15.67:0/3369034008 shutdown_connections 2024-09-21T06:36:27.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.324+0000 7fa874ed0640 1 --2- 172.21.15.67:0/3369034008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8701052a0 0x7fa870102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.324+0000 7fa874ed0640 1 --2- 172.21.15.67:0/3369034008 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fa85c0b65a0 0x7fa85c0b8ab0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.324+0000 7fa874ed0640 1 --2- 172.21.15.67:0/3369034008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa87010de70 0x7fa870103250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.324+0000 7fa874ed0640 1 --2- 172.21.15.67:0/3369034008 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8701048c0 0x7fa8701027d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:27.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.324+0000 7fa874ed0640 1 -- 172.21.15.67:0/3369034008 >> 172.21.15.67:0/3369034008 conn(0x7fa870079760 msgr2=0x7fa870106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:27.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.325+0000 7fa874ed0640 1 -- 172.21.15.67:0/3369034008 shutdown_connections 2024-09-21T06:36:27.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:27.325+0000 7fa874ed0640 1 -- 172.21.15.67:0/3369034008 wait complete. 2024-09-21T06:36:27.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:27 smithi203 ceph-mon[33735]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:36:28.134 INFO:tasks.cephadm:Setup complete, yielding 2024-09-21T06:36:28.134 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T06:36:28.145 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi067.front.sepia.ceph.com 2024-09-21T06:36:28.145 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- bash -c 'ceph orch status' 2024-09-21T06:36:28.407 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:36:28.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.906+0000 7f86ad977640 1 -- 172.21.15.67:0/3933153297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86a8102d30 msgr2=0x7f86a8105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:28.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.906+0000 7f86ad977640 1 --2- 172.21.15.67:0/3933153297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86a8102d30 0x7f86a8105150 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8698009900 tx=0x7f869802a410 comp rx=0 tx=0).stop 2024-09-21T06:36:28.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.906+0000 7f86ad977640 1 -- 172.21.15.67:0/3933153297 shutdown_connections 2024-09-21T06:36:28.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.906+0000 7f86ad977640 1 --2- 172.21.15.67:0/3933153297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86a8105790 0x7f86a8107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:28.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.906+0000 7f86ad977640 1 --2- 172.21.15.67:0/3933153297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86a81083b0 0x7f86a810a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:28.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.906+0000 7f86ad977640 1 --2- 172.21.15.67:0/3933153297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86a8102d30 0x7f86a8105150 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:28.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.906+0000 7f86ad977640 1 -- 172.21.15.67:0/3933153297 >> 172.21.15.67:0/3933153297 conn(0x7f86a80fd770 msgr2=0x7f86a80ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:28.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.907+0000 7f86ad977640 1 -- 172.21.15.67:0/3933153297 shutdown_connections 2024-09-21T06:36:28.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.907+0000 7f86ad977640 1 -- 172.21.15.67:0/3933153297 wait complete. 2024-09-21T06:36:28.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.907+0000 7f86ad977640 1 Processor -- start 2024-09-21T06:36:28.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.907+0000 7f86ad977640 1 -- start start 2024-09-21T06:36:28.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.907+0000 7f86ad977640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86a8102d30 0x7f86a81021d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:28.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.907+0000 7f86ad977640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86a8105790 0x7f86a819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:28.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.907+0000 7f86ad977640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86a81083b0 0x7f86a81a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:28.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.907+0000 7f86ad977640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f86a810e1b0 con 0x7f86a8105790 2024-09-21T06:36:28.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.907+0000 7f86ad977640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f86a810e030 con 0x7f86a81083b0 2024-09-21T06:36:28.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.907+0000 7f86ad977640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f86a810e330 con 0x7f86a8102d30 2024-09-21T06:36:28.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.908+0000 7f869ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86a8105790 0x7f86a819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:28.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.908+0000 7f869ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86a8105790 0x7f86a819ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37964/0 (socket says 172.21.15.67:37964) 2024-09-21T06:36:28.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.908+0000 7f869ffff640 1 -- 172.21.15.67:0/3504445186 learned_addr learned my addr 172.21.15.67:0/3504445186 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:28.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.908+0000 7f86a6ffd640 1 --2- 172.21.15.67:0/3504445186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86a8102d30 0x7f86a81021d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:28.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.908+0000 7f86a77fe640 1 --2- 172.21.15.67:0/3504445186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86a81083b0 0x7f86a81a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:28.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.908+0000 7f869ffff640 1 -- 172.21.15.67:0/3504445186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86a8102d30 msgr2=0x7f86a81021d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:28.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.908+0000 7f869ffff640 1 --2- 172.21.15.67:0/3504445186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86a8102d30 0x7f86a81021d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:28.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.908+0000 7f869ffff640 1 -- 172.21.15.67:0/3504445186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86a81083b0 msgr2=0x7f86a81a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:28.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.908+0000 7f869ffff640 1 --2- 172.21.15.67:0/3504445186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86a81083b0 0x7f86a81a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:28.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.908+0000 7f869ffff640 1 -- 172.21.15.67:0/3504445186 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a81a19a0 con 0x7f86a8105790 2024-09-21T06:36:28.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.908+0000 7f869ffff640 1 --2- 172.21.15.67:0/3504445186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86a8105790 0x7f86a819ec50 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f869000d950 tx=0x7f869000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:28.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.908+0000 7f86ac975640 1 -- 172.21.15.67:0/3504445186 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8690017070 con 0x7f86a8105790 2024-09-21T06:36:28.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.908+0000 7f86ad977640 1 -- 172.21.15.67:0/3504445186 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86a81a1c90 con 0x7f86a8105790 2024-09-21T06:36:28.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.908+0000 7f86ac975640 1 -- 172.21.15.67:0/3504445186 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8690009c30 con 0x7f86a8105790 2024-09-21T06:36:28.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.908+0000 7f86ac975640 1 -- 172.21.15.67:0/3504445186 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869000f620 con 0x7f86a8105790 2024-09-21T06:36:28.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.908+0000 7f86ad977640 1 -- 172.21.15.67:0/3504445186 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86a81a2170 con 0x7f86a8105790 2024-09-21T06:36:28.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.909+0000 7f86ad977640 1 -- 172.21.15.67:0/3504445186 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86a80ff290 con 0x7f86a8105790 2024-09-21T06:36:28.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.912+0000 7f86ac975640 1 -- 172.21.15.67:0/3504445186 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f8690010460 con 0x7f86a8105790 2024-09-21T06:36:28.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.913+0000 7f86ac975640 1 --2- 172.21.15.67:0/3504445186 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f86940b65a0 0x7f86940b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:28.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.913+0000 7f86a6ffd640 1 --2- 172.21.15.67:0/3504445186 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f86940b65a0 0x7f86940b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:28.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.913+0000 7f86ac975640 1 -- 172.21.15.67:0/3504445186 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f86900e7540 con 0x7f86a8105790 2024-09-21T06:36:28.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.914+0000 7f86a6ffd640 1 --2- 172.21.15.67:0/3504445186 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f86940b65a0 0x7f86940b8ab0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8698006010 tx=0x7f8698005720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:28.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:28.916+0000 7f86ac975640 1 -- 172.21.15.67:0/3504445186 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f86900ad8a0 con 0x7f86a8105790 2024-09-21T06:36:29.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:29.123+0000 7f86ad977640 1 -- 172.21.15.67:0/3504445186 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f86a8063800 con 0x7f86940b65a0 2024-09-21T06:36:29.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:29 smithi067 ceph-mon[31232]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:36:29.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:29.124+0000 7f86ac975640 1 -- 172.21.15.67:0/3504445186 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f86a8063800 con 0x7f86940b65a0 2024-09-21T06:36:29.126 INFO:teuthology.orchestra.run.smithi067.stdout:Backend: cephadm 2024-09-21T06:36:29.126 INFO:teuthology.orchestra.run.smithi067.stdout:Available: Yes 2024-09-21T06:36:29.126 INFO:teuthology.orchestra.run.smithi067.stdout:Paused: No 2024-09-21T06:36:29.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:29.126+0000 7f86ad977640 1 -- 172.21.15.67:0/3504445186 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f86940b65a0 msgr2=0x7f86940b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:29.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:29.126+0000 7f86ad977640 1 --2- 172.21.15.67:0/3504445186 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f86940b65a0 0x7f86940b8ab0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8698006010 tx=0x7f8698005720 comp rx=0 tx=0).stop 2024-09-21T06:36:29.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:29.126+0000 7f86ad977640 1 -- 172.21.15.67:0/3504445186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86a8105790 msgr2=0x7f86a819ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:29.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:29.126+0000 7f86ad977640 1 --2- 172.21.15.67:0/3504445186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86a8105790 0x7f86a819ec50 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f869000d950 tx=0x7f869000de10 comp rx=0 tx=0).stop 2024-09-21T06:36:29.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:29.127+0000 7f86ad977640 1 -- 172.21.15.67:0/3504445186 shutdown_connections 2024-09-21T06:36:29.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:29.127+0000 7f86ad977640 1 --2- 172.21.15.67:0/3504445186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86a81083b0 0x7f86a81a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:29.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:29.127+0000 7f86ad977640 1 --2- 172.21.15.67:0/3504445186 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f86940b65a0 0x7f86940b8ab0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:29.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:29.127+0000 7f86ad977640 1 --2- 172.21.15.67:0/3504445186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86a8105790 0x7f86a819ec50 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:29.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:29.127+0000 7f86ad977640 1 --2- 172.21.15.67:0/3504445186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86a8102d30 0x7f86a81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:29.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:29.127+0000 7f86ad977640 1 -- 172.21.15.67:0/3504445186 >> 172.21.15.67:0/3504445186 conn(0x7f86a80fd770 msgr2=0x7f86a8103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:29.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:29.127+0000 7f86ad977640 1 -- 172.21.15.67:0/3504445186 shutdown_connections 2024-09-21T06:36:29.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:29.127+0000 7f86ad977640 1 -- 172.21.15.67:0/3504445186 wait complete. 2024-09-21T06:36:29.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:29 smithi100 ceph-mon[33401]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:36:29.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:29 smithi203 ceph-mon[33735]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:36:30.070 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ps' 2024-09-21T06:36:30.291 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:30 smithi067 ceph-mon[31232]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:36:30.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:30 smithi100 ceph-mon[33401]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:36:30.334 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:36:30.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:30 smithi203 ceph-mon[33735]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:36:30.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.842+0000 7f5488968640 1 -- 172.21.15.67:0/922698100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5484104090 msgr2=0x7f548410aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:30.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.842+0000 7f5488968640 1 --2- 172.21.15.67:0/922698100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5484104090 0x7f548410aa30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f547c009ca0 tx=0x7f547c02a950 comp rx=0 tx=0).stop 2024-09-21T06:36:30.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.842+0000 7f5488968640 1 -- 172.21.15.67:0/922698100 shutdown_connections 2024-09-21T06:36:30.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.842+0000 7f5488968640 1 --2- 172.21.15.67:0/922698100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54841036f0 0x7f5484103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:30.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.842+0000 7f5488968640 1 --2- 172.21.15.67:0/922698100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5484104090 0x7f548410aa30 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:30.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.842+0000 7f5488968640 1 --2- 172.21.15.67:0/922698100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5484102660 0x7f5484102ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:30.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.842+0000 7f5488968640 1 -- 172.21.15.67:0/922698100 >> 172.21.15.67:0/922698100 conn(0x7f54840fdfd0 msgr2=0x7f5484100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:30.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.842+0000 7f5488968640 1 -- 172.21.15.67:0/922698100 shutdown_connections 2024-09-21T06:36:30.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.842+0000 7f5488968640 1 -- 172.21.15.67:0/922698100 wait complete. 2024-09-21T06:36:30.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.843+0000 7f5488968640 1 Processor -- start 2024-09-21T06:36:30.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.843+0000 7f5488968640 1 -- start start 2024-09-21T06:36:30.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.843+0000 7f5488968640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5484102660 0x7f548410cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:30.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.843+0000 7f5488968640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54841036f0 0x7f548410b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:30.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.843+0000 7f5488968640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5484104090 0x7f548410bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:30.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.843+0000 7f5488968640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f548410f280 con 0x7f5484104090 2024-09-21T06:36:30.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.843+0000 7f5488968640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f548410f100 con 0x7f5484102660 2024-09-21T06:36:30.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.843+0000 7f5488968640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f548410f400 con 0x7f54841036f0 2024-09-21T06:36:30.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.843+0000 7f5482d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5484104090 0x7f548410bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:30.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.843+0000 7f5482d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5484104090 0x7f548410bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37972/0 (socket says 172.21.15.67:37972) 2024-09-21T06:36:30.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.844+0000 7f5482d9d640 1 -- 172.21.15.67:0/775710568 learned_addr learned my addr 172.21.15.67:0/775710568 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:30.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.843+0000 7f548259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5484102660 0x7f548410cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:30.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.844+0000 7f5481d9b640 1 --2- 172.21.15.67:0/775710568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54841036f0 0x7f548410b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:30.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.844+0000 7f5482d9d640 1 -- 172.21.15.67:0/775710568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54841036f0 msgr2=0x7f548410b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:30.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.844+0000 7f5482d9d640 1 --2- 172.21.15.67:0/775710568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54841036f0 0x7f548410b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:30.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.844+0000 7f5482d9d640 1 -- 172.21.15.67:0/775710568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5484102660 msgr2=0x7f548410cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:30.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.844+0000 7f5482d9d640 1 --2- 172.21.15.67:0/775710568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5484102660 0x7f548410cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:30.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.844+0000 7f5482d9d640 1 -- 172.21.15.67:0/775710568 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f548410c2b0 con 0x7f5484104090 2024-09-21T06:36:30.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.844+0000 7f548259c640 1 --2- 172.21.15.67:0/775710568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5484102660 0x7f548410cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:36:30.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.844+0000 7f5482d9d640 1 --2- 172.21.15.67:0/775710568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5484104090 0x7f548410bbb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f547c02ae70 tx=0x7f547c003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:30.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.844+0000 7f547b7fe640 1 -- 172.21.15.67:0/775710568 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c034040 con 0x7f5484104090 2024-09-21T06:36:30.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.844+0000 7f547b7fe640 1 -- 172.21.15.67:0/775710568 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f547c030850 con 0x7f5484104090 2024-09-21T06:36:30.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.844+0000 7f547b7fe640 1 -- 172.21.15.67:0/775710568 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c004650 con 0x7f5484104090 2024-09-21T06:36:30.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.844+0000 7f5488968640 1 -- 172.21.15.67:0/775710568 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54841a3f30 con 0x7f5484104090 2024-09-21T06:36:30.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.844+0000 7f5488968640 1 -- 172.21.15.67:0/775710568 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54841a42d0 con 0x7f5484104090 2024-09-21T06:36:30.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.846+0000 7f5488968640 1 -- 172.21.15.67:0/775710568 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5484102ad0 con 0x7f5484104090 2024-09-21T06:36:30.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.847+0000 7f547b7fe640 1 -- 172.21.15.67:0/775710568 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f547c037020 con 0x7f5484104090 2024-09-21T06:36:30.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.848+0000 7f547b7fe640 1 --2- 172.21.15.67:0/775710568 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f54700b6620 0x7f54700b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:30.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.848+0000 7f548259c640 1 --2- 172.21.15.67:0/775710568 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f54700b6620 0x7f54700b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:30.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.848+0000 7f547b7fe640 1 -- 172.21.15.67:0/775710568 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f547c1087f0 con 0x7f5484104090 2024-09-21T06:36:30.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.849+0000 7f548259c640 1 --2- 172.21.15.67:0/775710568 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f54700b6620 0x7f54700b8b30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5474009fc0 tx=0x7f5474005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:30.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:30.851+0000 7f547b7fe640 1 -- 172.21.15.67:0/775710568 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f547c083710 con 0x7f5484104090 2024-09-21T06:36:31.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:31.071+0000 7f5488968640 1 -- 172.21.15.67:0/775710568 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f54840638c0 con 0x7f54700b6620 2024-09-21T06:36:31.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:31.084+0000 7f547b7fe640 1 -- 172.21.15.67:0/775710568 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f54840638c0 con 0x7f54700b6620 2024-09-21T06:36:31.085 INFO:teuthology.orchestra.run.smithi067.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T06:36:31.085 INFO:teuthology.orchestra.run.smithi067.stdout:mds.a smithi067 running (2m) 61s ago 2m 81.8M - 19.3.0-5120-g18432218 ddce307e1f32 3ecf1f8453a6 2024-09-21T06:36:31.085 INFO:teuthology.orchestra.run.smithi067.stdout:mds.b smithi100 running (2m) 63s ago 2m 93.4M - 19.3.0-5120-g18432218 ddce307e1f32 1e4cb69455a6 2024-09-21T06:36:31.086 INFO:teuthology.orchestra.run.smithi067.stdout:mds.c smithi203 running (2m) 63s ago 2m 81.9M - 19.3.0-5120-g18432218 ddce307e1f32 f580d722c33d 2024-09-21T06:36:31.086 INFO:teuthology.orchestra.run.smithi067.stdout:mds.d smithi067 running (2m) 61s ago 2m 82.2M - 19.3.0-5120-g18432218 ddce307e1f32 4744b69f185d 2024-09-21T06:36:31.086 INFO:teuthology.orchestra.run.smithi067.stdout:mds.e smithi100 running (2m) 63s ago 2m 81.5M - 19.3.0-5120-g18432218 ddce307e1f32 21813274ce3b 2024-09-21T06:36:31.086 INFO:teuthology.orchestra.run.smithi067.stdout:mds.f smithi203 running (2m) 63s ago 2m 82.2M - 19.3.0-5120-g18432218 ddce307e1f32 34dd868b4ddb 2024-09-21T06:36:31.086 INFO:teuthology.orchestra.run.smithi067.stdout:mds.g smithi067 running (2m) 61s ago 2m 82.0M - 19.3.0-5120-g18432218 ddce307e1f32 c8ff47271516 2024-09-21T06:36:31.086 INFO:teuthology.orchestra.run.smithi067.stdout:mds.h smithi100 running (2m) 63s ago 2m 84.6M - 19.3.0-5120-g18432218 ddce307e1f32 fcaaf2d7561b 2024-09-21T06:36:31.086 INFO:teuthology.orchestra.run.smithi067.stdout:mds.i smithi203 running (2m) 63s ago 2m 81.5M - 19.3.0-5120-g18432218 ddce307e1f32 440a27cc8a72 2024-09-21T06:36:31.086 INFO:teuthology.orchestra.run.smithi067.stdout:mds.j smithi067 running (2m) 61s ago 2m 84.5M - 19.3.0-5120-g18432218 ddce307e1f32 97060575a224 2024-09-21T06:36:31.086 INFO:teuthology.orchestra.run.smithi067.stdout:mds.k smithi100 running (2m) 63s ago 2m 82.1M - 19.3.0-5120-g18432218 ddce307e1f32 9195a712aedb 2024-09-21T06:36:31.086 INFO:teuthology.orchestra.run.smithi067.stdout:mds.l smithi203 running (2m) 63s ago 2m 81.7M - 19.3.0-5120-g18432218 ddce307e1f32 1e20586e4e96 2024-09-21T06:36:31.086 INFO:teuthology.orchestra.run.smithi067.stdout:mgr.x smithi067 *:9283,8765 running (10m) 61s ago 10m 592M - 19.3.0-5120-g18432218 ddce307e1f32 5f64d4ee2103 2024-09-21T06:36:31.086 INFO:teuthology.orchestra.run.smithi067.stdout:mgr.y smithi100 *:8443,8765 running (8m) 63s ago 8m 523M - 19.3.0-5120-g18432218 ddce307e1f32 ffd7841c3d54 2024-09-21T06:36:31.086 INFO:teuthology.orchestra.run.smithi067.stdout:mgr.z smithi203 *:8443,8765 running (8m) 63s ago 8m 526M - 19.3.0-5120-g18432218 ddce307e1f32 37bfc5ac89ca 2024-09-21T06:36:31.086 INFO:teuthology.orchestra.run.smithi067.stdout:mon.a smithi067 running (10m) 61s ago 10m 119M 2048M 19.3.0-5120-g18432218 ddce307e1f32 853c94eacb0a 2024-09-21T06:36:31.087 INFO:teuthology.orchestra.run.smithi067.stdout:mon.b smithi100 running (8m) 63s ago 8m 110M 2048M 19.3.0-5120-g18432218 ddce307e1f32 f935de8304a8 2024-09-21T06:36:31.087 INFO:teuthology.orchestra.run.smithi067.stdout:mon.c smithi203 running (8m) 63s ago 8m 110M 2048M 19.3.0-5120-g18432218 ddce307e1f32 258ab452f69c 2024-09-21T06:36:31.087 INFO:teuthology.orchestra.run.smithi067.stdout:osd.0 smithi067 running (7m) 61s ago 7m 122M 4272M 19.3.0-5120-g18432218 ddce307e1f32 ac4a3c1547e9 2024-09-21T06:36:31.087 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1 smithi100 running (7m) 63s ago 7m 113M 4271M 19.3.0-5120-g18432218 ddce307e1f32 4930c2512656 2024-09-21T06:36:31.087 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2 smithi203 running (6m) 63s ago 6m 113M 4272M 19.3.0-5120-g18432218 ddce307e1f32 0419b3bdb938 2024-09-21T06:36:31.087 INFO:teuthology.orchestra.run.smithi067.stdout:osd.3 smithi067 running (6m) 61s ago 6m 124M 4272M 19.3.0-5120-g18432218 ddce307e1f32 5f55c49290db 2024-09-21T06:36:31.087 INFO:teuthology.orchestra.run.smithi067.stdout:osd.4 smithi100 running (5m) 63s ago 5m 110M 4271M 19.3.0-5120-g18432218 ddce307e1f32 f422eeeb192c 2024-09-21T06:36:31.087 INFO:teuthology.orchestra.run.smithi067.stdout:osd.5 smithi203 running (5m) 63s ago 5m 112M 4272M 19.3.0-5120-g18432218 ddce307e1f32 efc20c9e6e70 2024-09-21T06:36:31.087 INFO:teuthology.orchestra.run.smithi067.stdout:osd.6 smithi067 running (5m) 61s ago 5m 110M 4272M 19.3.0-5120-g18432218 ddce307e1f32 328db6064d00 2024-09-21T06:36:31.087 INFO:teuthology.orchestra.run.smithi067.stdout:osd.7 smithi100 running (4m) 63s ago 4m 121M 4271M 19.3.0-5120-g18432218 ddce307e1f32 0f9b5b2533c3 2024-09-21T06:36:31.088 INFO:teuthology.orchestra.run.smithi067.stdout:osd.8 smithi203 running (4m) 63s ago 4m 119M 4272M 19.3.0-5120-g18432218 ddce307e1f32 c24b9ef02b98 2024-09-21T06:36:31.088 INFO:teuthology.orchestra.run.smithi067.stdout:osd.9 smithi067 running (3m) 61s ago 3m 109M 4272M 19.3.0-5120-g18432218 ddce307e1f32 908e132948bb 2024-09-21T06:36:31.088 INFO:teuthology.orchestra.run.smithi067.stdout:osd.10 smithi100 running (3m) 63s ago 3m 122M 4271M 19.3.0-5120-g18432218 ddce307e1f32 670a59e162e9 2024-09-21T06:36:31.088 INFO:teuthology.orchestra.run.smithi067.stdout:osd.11 smithi203 running (2m) 63s ago 2m 120M 4272M 19.3.0-5120-g18432218 ddce307e1f32 ff44622e2f6d 2024-09-21T06:36:31.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:31.086+0000 7f5488968640 1 -- 172.21.15.67:0/775710568 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f54700b6620 msgr2=0x7f54700b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:31.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:31.086+0000 7f5488968640 1 --2- 172.21.15.67:0/775710568 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f54700b6620 0x7f54700b8b30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5474009fc0 tx=0x7f5474005f90 comp rx=0 tx=0).stop 2024-09-21T06:36:31.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:31.086+0000 7f5488968640 1 -- 172.21.15.67:0/775710568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5484104090 msgr2=0x7f548410bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:31.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:31.086+0000 7f5488968640 1 --2- 172.21.15.67:0/775710568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5484104090 0x7f548410bbb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f547c02ae70 tx=0x7f547c003c90 comp rx=0 tx=0).stop 2024-09-21T06:36:31.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:31.086+0000 7f5488968640 1 -- 172.21.15.67:0/775710568 shutdown_connections 2024-09-21T06:36:31.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:31.086+0000 7f5488968640 1 --2- 172.21.15.67:0/775710568 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f54700b6620 0x7f54700b8b30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:31.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:31.086+0000 7f5488968640 1 --2- 172.21.15.67:0/775710568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5484104090 0x7f548410bbb0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:31.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:31.086+0000 7f5488968640 1 --2- 172.21.15.67:0/775710568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54841036f0 0x7f548410b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:31.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:31.086+0000 7f5488968640 1 --2- 172.21.15.67:0/775710568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5484102660 0x7f548410cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:31.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:31.086+0000 7f5488968640 1 -- 172.21.15.67:0/775710568 >> 172.21.15.67:0/775710568 conn(0x7f54840fdfd0 msgr2=0x7f54841069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:31.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:31.086+0000 7f5488968640 1 -- 172.21.15.67:0/775710568 shutdown_connections 2024-09-21T06:36:31.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:31.086+0000 7f5488968640 1 -- 172.21.15.67:0/775710568 wait complete. 2024-09-21T06:36:31.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:31 smithi067 ceph-mon[31232]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:36:31.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:31 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:36:31.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:31 smithi100 ceph-mon[33401]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:36:31.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:31 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:36:31.306 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:31 smithi203 ceph-mon[33735]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:36:31.306 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:31 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:36:32.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:32 smithi067 ceph-mon[31232]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:36:32.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:32 smithi100 ceph-mon[33401]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:36:32.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:32 smithi203 ceph-mon[33735]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:36:32.505 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ls' 2024-09-21T06:36:32.790 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:36:33.196 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:33 smithi067 ceph-mon[31232]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:36:33.196 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:33 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:36:33.197 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:33 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:36:33.197 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:33 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:36:33.197 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:33 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:36:33.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.291+0000 7f4977b07640 1 -- 172.21.15.67:0/3376116863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4970109480 msgr2=0x7f497010b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:33.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.291+0000 7f4977b07640 1 --2- 172.21.15.67:0/3376116863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4970109480 0x7f497010b990 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4968009ca0 tx=0x7f496802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:36:33.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.291+0000 7f4977b07640 1 -- 172.21.15.67:0/3376116863 shutdown_connections 2024-09-21T06:36:33.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.291+0000 7f4977b07640 1 --2- 172.21.15.67:0/3376116863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4970109480 0x7f497010b990 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:33.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.291+0000 7f4977b07640 1 --2- 172.21.15.67:0/3376116863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4970106970 0x7f4970108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:33.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.291+0000 7f4977b07640 1 --2- 172.21.15.67:0/3376116863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f497006b8e0 0x7f4970106430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:33.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.291+0000 7f4977b07640 1 -- 172.21.15.67:0/3376116863 >> 172.21.15.67:0/3376116863 conn(0x7f49700fdfb0 msgr2=0x7f4970100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:33.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.292+0000 7f4977b07640 1 -- 172.21.15.67:0/3376116863 shutdown_connections 2024-09-21T06:36:33.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.292+0000 7f4977b07640 1 -- 172.21.15.67:0/3376116863 wait complete. 2024-09-21T06:36:33.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.292+0000 7f4977b07640 1 Processor -- start 2024-09-21T06:36:33.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.292+0000 7f4977b07640 1 -- start start 2024-09-21T06:36:33.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.293+0000 7f4977b07640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f497006b8e0 0x7f4970103a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:33.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.293+0000 7f4977b07640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4970106970 0x7f4970102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:33.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.293+0000 7f4977b07640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4970109480 0x7f4970102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:33.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.293+0000 7f4977b07640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f497010f290 con 0x7f4970106970 2024-09-21T06:36:33.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.293+0000 7f4977b07640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f497010f110 con 0x7f4970109480 2024-09-21T06:36:33.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.293+0000 7f4977b07640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f497010f410 con 0x7f497006b8e0 2024-09-21T06:36:33.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.293+0000 7f49750a2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4970106970 0x7f4970102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:33.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.293+0000 7f49750a2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4970106970 0x7f4970102330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37988/0 (socket says 172.21.15.67:37988) 2024-09-21T06:36:33.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.293+0000 7f49750a2640 1 -- 172.21.15.67:0/621998091 learned_addr learned my addr 172.21.15.67:0/621998091 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:33.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.293+0000 7f49758a3640 1 --2- 172.21.15.67:0/621998091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f497006b8e0 0x7f4970103a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:33.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.293+0000 7f49760a4640 1 --2- 172.21.15.67:0/621998091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4970109480 0x7f4970102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:33.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.293+0000 7f49750a2640 1 -- 172.21.15.67:0/621998091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f497006b8e0 msgr2=0x7f4970103a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:33.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.293+0000 7f49750a2640 1 --2- 172.21.15.67:0/621998091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f497006b8e0 0x7f4970103a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:33.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.293+0000 7f49750a2640 1 -- 172.21.15.67:0/621998091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4970109480 msgr2=0x7f4970102870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:33.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.293+0000 7f49750a2640 1 --2- 172.21.15.67:0/621998091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4970109480 0x7f4970102870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:33.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.293+0000 7f49750a2640 1 -- 172.21.15.67:0/621998091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4970102f70 con 0x7f4970106970 2024-09-21T06:36:33.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.293+0000 7f49758a3640 1 --2- 172.21.15.67:0/621998091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f497006b8e0 0x7f4970103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:36:33.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.293+0000 7f49760a4640 1 --2- 172.21.15.67:0/621998091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4970109480 0x7f4970102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:36:33.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.294+0000 7f49750a2640 1 --2- 172.21.15.67:0/621998091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4970106970 0x7f4970102330 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f496000e9f0 tx=0x7f496000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:33.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.294+0000 7f4966ffd640 1 -- 172.21.15.67:0/621998091 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496000cd80 con 0x7f4970106970 2024-09-21T06:36:33.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.294+0000 7f4966ffd640 1 -- 172.21.15.67:0/621998091 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f49600054b0 con 0x7f4970106970 2024-09-21T06:36:33.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.294+0000 7f4977b07640 1 -- 172.21.15.67:0/621998091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49701a3f40 con 0x7f4970106970 2024-09-21T06:36:33.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.294+0000 7f4966ffd640 1 -- 172.21.15.67:0/621998091 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4960010430 con 0x7f4970106970 2024-09-21T06:36:33.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.294+0000 7f4977b07640 1 -- 172.21.15.67:0/621998091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49701a42e0 con 0x7f4970106970 2024-09-21T06:36:33.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.295+0000 7f4977b07640 1 -- 172.21.15.67:0/621998091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4970106d80 con 0x7f4970106970 2024-09-21T06:36:33.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.297+0000 7f4966ffd640 1 -- 172.21.15.67:0/621998091 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f49600046c0 con 0x7f4970106970 2024-09-21T06:36:33.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.298+0000 7f4966ffd640 1 --2- 172.21.15.67:0/621998091 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f495c0b6620 0x7f495c0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:33.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.298+0000 7f49758a3640 1 --2- 172.21.15.67:0/621998091 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f495c0b6620 0x7f495c0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:33.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.298+0000 7f4966ffd640 1 -- 172.21.15.67:0/621998091 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f49600e7ec0 con 0x7f4970106970 2024-09-21T06:36:33.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.299+0000 7f49758a3640 1 --2- 172.21.15.67:0/621998091 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f495c0b6620 0x7f495c0b8b30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f496c009c20 tx=0x7f496c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:33.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.301+0000 7f4966ffd640 1 -- 172.21.15.67:0/621998091 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4960056710 con 0x7f4970106970 2024-09-21T06:36:33.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:33 smithi203 ceph-mon[33735]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:36:33.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:33 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:36:33.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:33 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:36:33.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:33 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:36:33.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:33 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:36:33.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.505+0000 7f4977b07640 1 -- 172.21.15.67:0/621998091 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f4970063800 con 0x7f495c0b6620 2024-09-21T06:36:33.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.512+0000 7f4966ffd640 1 -- 172.21.15.67:0/621998091 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f4970063800 con 0x7f495c0b6620 2024-09-21T06:36:33.513 INFO:teuthology.orchestra.run.smithi067.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-21T06:36:33.513 INFO:teuthology.orchestra.run.smithi067.stdout:mds.all 12/12 1s ago 2m smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2024-09-21T06:36:33.513 INFO:teuthology.orchestra.run.smithi067.stdout:mgr 3/3 1s ago 8m smithi067=x;smithi100=y;smithi203=z;count:3 2024-09-21T06:36:33.513 INFO:teuthology.orchestra.run.smithi067.stdout:mon 3/3 1s ago 8m smithi067:172.21.15.67=a;smithi100:172.21.15.100=b;smithi203:172.21.15.203=c;count:3 2024-09-21T06:36:33.513 INFO:teuthology.orchestra.run.smithi067.stdout:osd 12 1s ago - 2024-09-21T06:36:33.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.514+0000 7f4977b07640 1 -- 172.21.15.67:0/621998091 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f495c0b6620 msgr2=0x7f495c0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:33.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.514+0000 7f4977b07640 1 --2- 172.21.15.67:0/621998091 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f495c0b6620 0x7f495c0b8b30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f496c009c20 tx=0x7f496c005f90 comp rx=0 tx=0).stop 2024-09-21T06:36:33.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.514+0000 7f4977b07640 1 -- 172.21.15.67:0/621998091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4970106970 msgr2=0x7f4970102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:33.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.514+0000 7f4977b07640 1 --2- 172.21.15.67:0/621998091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4970106970 0x7f4970102330 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f496000e9f0 tx=0x7f496000edf0 comp rx=0 tx=0).stop 2024-09-21T06:36:33.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.514+0000 7f4977b07640 1 -- 172.21.15.67:0/621998091 shutdown_connections 2024-09-21T06:36:33.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.514+0000 7f4977b07640 1 --2- 172.21.15.67:0/621998091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4970109480 0x7f4970102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:33.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.514+0000 7f4977b07640 1 --2- 172.21.15.67:0/621998091 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f495c0b6620 0x7f495c0b8b30 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:33.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.514+0000 7f4977b07640 1 --2- 172.21.15.67:0/621998091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4970106970 0x7f4970102330 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:33.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.514+0000 7f4977b07640 1 --2- 172.21.15.67:0/621998091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f497006b8e0 0x7f4970103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:33.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.514+0000 7f4977b07640 1 -- 172.21.15.67:0/621998091 >> 172.21.15.67:0/621998091 conn(0x7f49700fdfb0 msgr2=0x7f4970104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:33.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.514+0000 7f4977b07640 1 -- 172.21.15.67:0/621998091 shutdown_connections 2024-09-21T06:36:33.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:33.514+0000 7f4977b07640 1 -- 172.21.15.67:0/621998091 wait complete. 2024-09-21T06:36:33.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:33 smithi100 ceph-mon[33401]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:36:33.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:33 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:36:33.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:33 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:36:33.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:33 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:36:33.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:33 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:36:34.170 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- bash -c 'ceph orch host ls' 2024-09-21T06:36:34.451 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:36:34.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:34 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:36:34.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:34 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:36:34.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:34 smithi067 ceph-mon[31232]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:36:34.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:34 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:36:34.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:34 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:36:34.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:34 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:36:34.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:34 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:36:34.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:34 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:36:34.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:34 smithi203 ceph-mon[33735]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:36:34.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:34 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:36:34.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:34 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:36:34.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:34 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:36:34.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:34 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:36:34.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:34 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:36:34.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:34 smithi100 ceph-mon[33401]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:36:34.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:34 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:36:34.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:34 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:36:34.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:34 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:36:34.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.964+0000 7f3a1612f640 1 -- 172.21.15.67:0/2432850031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a10101b40 msgr2=0x7f3a10101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:34.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.964+0000 7f3a1612f640 1 --2- 172.21.15.67:0/2432850031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a10101b40 0x7f3a10101f50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3a00009900 tx=0x7f3a0002a410 comp rx=0 tx=0).stop 2024-09-21T06:36:34.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.964+0000 7f3a1612f640 1 -- 172.21.15.67:0/2432850031 shutdown_connections 2024-09-21T06:36:34.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.964+0000 7f3a1612f640 1 --2- 172.21.15.67:0/2432850031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a10109540 0x7f3a1010b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:34.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.964+0000 7f3a1612f640 1 --2- 172.21.15.67:0/2432850031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a10106be0 0x7f3a10109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:34.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.964+0000 7f3a1612f640 1 --2- 172.21.15.67:0/2432850031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a10101b40 0x7f3a10101f50 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:34.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.964+0000 7f3a1612f640 1 -- 172.21.15.67:0/2432850031 >> 172.21.15.67:0/2432850031 conn(0x7f3a100fd770 msgr2=0x7f3a100ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:34.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.964+0000 7f3a1612f640 1 -- 172.21.15.67:0/2432850031 shutdown_connections 2024-09-21T06:36:34.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.964+0000 7f3a1612f640 1 -- 172.21.15.67:0/2432850031 wait complete. 2024-09-21T06:36:34.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.964+0000 7f3a1612f640 1 Processor -- start 2024-09-21T06:36:34.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.965+0000 7f3a1612f640 1 -- start start 2024-09-21T06:36:34.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.965+0000 7f3a1612f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a10101b40 0x7f3a10199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:35.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.965+0000 7f3a1612f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a10106be0 0x7f3a1019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:35.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.965+0000 7f3a1612f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a10109540 0x7f3a1019b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:35.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.965+0000 7f3a1612f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1010f2d0 con 0x7f3a10106be0 2024-09-21T06:36:35.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.965+0000 7f3a1612f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1010f150 con 0x7f3a10109540 2024-09-21T06:36:35.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.965+0000 7f3a1612f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1010f450 con 0x7f3a10101b40 2024-09-21T06:36:35.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.965+0000 7f3a0effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a10106be0 0x7f3a1019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:35.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.965+0000 7f3a0effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a10106be0 0x7f3a1019a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38008/0 (socket says 172.21.15.67:38008) 2024-09-21T06:36:35.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.965+0000 7f3a0effd640 1 -- 172.21.15.67:0/4230324514 learned_addr learned my addr 172.21.15.67:0/4230324514 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:35.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.965+0000 7f3a0ffff640 1 --2- 172.21.15.67:0/4230324514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a10109540 0x7f3a1019b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:35.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.965+0000 7f3a0f7fe640 1 --2- 172.21.15.67:0/4230324514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a10101b40 0x7f3a10199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:35.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.965+0000 7f3a0effd640 1 -- 172.21.15.67:0/4230324514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a10101b40 msgr2=0x7f3a10199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:35.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.965+0000 7f3a0effd640 1 --2- 172.21.15.67:0/4230324514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a10101b40 0x7f3a10199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:35.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.965+0000 7f3a0effd640 1 -- 172.21.15.67:0/4230324514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a10109540 msgr2=0x7f3a1019b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:35.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.965+0000 7f3a0effd640 1 --2- 172.21.15.67:0/4230324514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a10109540 0x7f3a1019b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:35.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.965+0000 7f3a0effd640 1 -- 172.21.15.67:0/4230324514 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a1019b9b0 con 0x7f3a10106be0 2024-09-21T06:36:35.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.965+0000 7f3a0ffff640 1 --2- 172.21.15.67:0/4230324514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a10109540 0x7f3a1019b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:36:35.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.966+0000 7f3a0effd640 1 --2- 172.21.15.67:0/4230324514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a10106be0 0x7f3a1019a5f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f39f800b4a0 tx=0x7f39f800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:35.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.966+0000 7f3a0cff9640 1 -- 172.21.15.67:0/4230324514 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f8014070 con 0x7f3a10106be0 2024-09-21T06:36:35.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.966+0000 7f3a1612f640 1 -- 172.21.15.67:0/4230324514 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a1019bca0 con 0x7f3a10106be0 2024-09-21T06:36:35.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.966+0000 7f3a0cff9640 1 -- 172.21.15.67:0/4230324514 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f39f8010aa0 con 0x7f3a10106be0 2024-09-21T06:36:35.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.966+0000 7f3a0cff9640 1 -- 172.21.15.67:0/4230324514 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f8017b70 con 0x7f3a10106be0 2024-09-21T06:36:35.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.966+0000 7f3a1612f640 1 -- 172.21.15.67:0/4230324514 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a101a23a0 con 0x7f3a10106be0 2024-09-21T06:36:35.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.967+0000 7f3a1612f640 1 -- 172.21.15.67:0/4230324514 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a100ff290 con 0x7f3a10106be0 2024-09-21T06:36:35.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.969+0000 7f3a0cff9640 1 -- 172.21.15.67:0/4230324514 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f39f8017070 con 0x7f3a10106be0 2024-09-21T06:36:35.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.969+0000 7f3a0cff9640 1 --2- 172.21.15.67:0/4230324514 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f39fc0b65a0 0x7f39fc0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:35.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.969+0000 7f3a0f7fe640 1 --2- 172.21.15.67:0/4230324514 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f39fc0b65a0 0x7f39fc0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:35.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.970+0000 7f3a0cff9640 1 -- 172.21.15.67:0/4230324514 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f39f80e6c00 con 0x7f3a10106be0 2024-09-21T06:36:35.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.970+0000 7f3a0f7fe640 1 --2- 172.21.15.67:0/4230324514 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f39fc0b65a0 0x7f39fc0b8ab0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3a00000c00 tx=0x7f3a00005720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:35.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:34.974+0000 7f3a0cff9640 1 -- 172.21.15.67:0/4230324514 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f39f80acf60 con 0x7f3a10106be0 2024-09-21T06:36:35.823 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:35 smithi100 ceph-mon[33401]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:36:35.823 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:35 smithi203 ceph-mon[33735]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:36:35.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:35.167+0000 7f3a1612f640 1 -- 172.21.15.67:0/4230324514 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f3a10063800 con 0x7f39fc0b65a0 2024-09-21T06:36:35.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:35.169+0000 7f3a0cff9640 1 -- 172.21.15.67:0/4230324514 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f3a10063800 con 0x7f39fc0b65a0 2024-09-21T06:36:35.824 INFO:teuthology.orchestra.run.smithi067.stdout:HOST ADDR LABELS STATUS 2024-09-21T06:36:35.824 INFO:teuthology.orchestra.run.smithi067.stdout:smithi067 172.21.15.67 2024-09-21T06:36:35.824 INFO:teuthology.orchestra.run.smithi067.stdout:smithi100 172.21.15.100 2024-09-21T06:36:35.824 INFO:teuthology.orchestra.run.smithi067.stdout:smithi203 172.21.15.203 2024-09-21T06:36:35.824 INFO:teuthology.orchestra.run.smithi067.stdout:3 hosts in cluster 2024-09-21T06:36:35.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:35.172+0000 7f3a1612f640 1 -- 172.21.15.67:0/4230324514 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f39fc0b65a0 msgr2=0x7f39fc0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:35.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:35.172+0000 7f3a1612f640 1 --2- 172.21.15.67:0/4230324514 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f39fc0b65a0 0x7f39fc0b8ab0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3a00000c00 tx=0x7f3a00005720 comp rx=0 tx=0).stop 2024-09-21T06:36:35.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:35.172+0000 7f3a1612f640 1 -- 172.21.15.67:0/4230324514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a10106be0 msgr2=0x7f3a1019a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:35.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:35.172+0000 7f3a1612f640 1 --2- 172.21.15.67:0/4230324514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a10106be0 0x7f3a1019a5f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f39f800b4a0 tx=0x7f39f800b960 comp rx=0 tx=0).stop 2024-09-21T06:36:35.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:35.172+0000 7f3a1612f640 1 -- 172.21.15.67:0/4230324514 shutdown_connections 2024-09-21T06:36:35.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:35.172+0000 7f3a1612f640 1 --2- 172.21.15.67:0/4230324514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a10109540 0x7f3a1019b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:35.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:35.172+0000 7f3a1612f640 1 --2- 172.21.15.67:0/4230324514 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f39fc0b65a0 0x7f39fc0b8ab0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:35.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:35.172+0000 7f3a1612f640 1 --2- 172.21.15.67:0/4230324514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a10106be0 0x7f3a1019a5f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:35.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:35.172+0000 7f3a1612f640 1 --2- 172.21.15.67:0/4230324514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a10101b40 0x7f3a10199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:35.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:35.172+0000 7f3a1612f640 1 -- 172.21.15.67:0/4230324514 >> 172.21.15.67:0/4230324514 conn(0x7f3a100fd770 msgr2=0x7f3a101079f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:35.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:35.172+0000 7f3a1612f640 1 -- 172.21.15.67:0/4230324514 shutdown_connections 2024-09-21T06:36:35.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:35.172+0000 7f3a1612f640 1 -- 172.21.15.67:0/4230324514 wait complete. 2024-09-21T06:36:35.826 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:35 smithi067 ceph-mon[31232]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:36:35.828 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- bash -c 'ceph orch device ls' 2024-09-21T06:36:36.086 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:36:36.498 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:36 smithi067 ceph-mon[31232]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:36:36.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.590+0000 7f7993066640 1 -- 172.21.15.67:0/4094391637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f798c1040a0 msgr2=0x7f798c10aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:36.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.590+0000 7f7993066640 1 --2- 172.21.15.67:0/4094391637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f798c1040a0 0x7f798c10aa40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7984009ca0 tx=0x7f798402a950 comp rx=0 tx=0).stop 2024-09-21T06:36:36.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.591+0000 7f7993066640 1 -- 172.21.15.67:0/4094391637 shutdown_connections 2024-09-21T06:36:36.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.591+0000 7f7993066640 1 --2- 172.21.15.67:0/4094391637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f798c103700 0x7f798c103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:36.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.591+0000 7f7993066640 1 --2- 172.21.15.67:0/4094391637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f798c1040a0 0x7f798c10aa40 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:36.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.591+0000 7f7993066640 1 --2- 172.21.15.67:0/4094391637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f798c102670 0x7f798c102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:36.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.591+0000 7f7993066640 1 -- 172.21.15.67:0/4094391637 >> 172.21.15.67:0/4094391637 conn(0x7f798c0fdfe0 msgr2=0x7f798c100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.591+0000 7f7993066640 1 -- 172.21.15.67:0/4094391637 shutdown_connections 2024-09-21T06:36:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.591+0000 7f7993066640 1 -- 172.21.15.67:0/4094391637 wait complete. 2024-09-21T06:36:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.591+0000 7f7993066640 1 Processor -- start 2024-09-21T06:36:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.591+0000 7f7993066640 1 -- start start 2024-09-21T06:36:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.592+0000 7f7993066640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f798c102670 0x7f798c06c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.592+0000 7f7993066640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f798c103700 0x7f798c19ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.592+0000 7f7993066640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f798c1040a0 0x7f798c1a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.592+0000 7f7993066640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f798c10e1c0 con 0x7f798c1040a0 2024-09-21T06:36:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.592+0000 7f7993066640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f798c10e040 con 0x7f798c103700 2024-09-21T06:36:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.592+0000 7f7993066640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f798c10e340 con 0x7f798c102670 2024-09-21T06:36:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.592+0000 7f7991603640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f798c1040a0 0x7f798c1a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.592+0000 7f7991603640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f798c1040a0 0x7f798c1a11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38018/0 (socket says 172.21.15.67:38018) 2024-09-21T06:36:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.592+0000 7f7991603640 1 -- 172.21.15.67:0/49804349 learned_addr learned my addr 172.21.15.67:0/49804349 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.592+0000 7f798bfff640 1 --2- 172.21.15.67:0/49804349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f798c103700 0x7f798c19ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.592+0000 7f7991603640 1 -- 172.21.15.67:0/49804349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f798c102670 msgr2=0x7f798c06c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.592+0000 7f7990e02640 1 --2- 172.21.15.67:0/49804349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f798c102670 0x7f798c06c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.592+0000 7f7991603640 1 --2- 172.21.15.67:0/49804349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f798c102670 0x7f798c06c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.592+0000 7f7991603640 1 -- 172.21.15.67:0/49804349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f798c103700 msgr2=0x7f798c19ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.592+0000 7f7991603640 1 --2- 172.21.15.67:0/49804349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f798c103700 0x7f798c19ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.592+0000 7f7991603640 1 -- 172.21.15.67:0/49804349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f798c1a19c0 con 0x7f798c1040a0 2024-09-21T06:36:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.592+0000 7f798bfff640 1 --2- 172.21.15.67:0/49804349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f798c103700 0x7f798c19ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:36:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.592+0000 7f7990e02640 1 --2- 172.21.15.67:0/49804349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f798c102670 0x7f798c06c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:36:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.592+0000 7f7991603640 1 --2- 172.21.15.67:0/49804349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f798c1040a0 0x7f798c1a11c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f798402f800 tx=0x7f7984003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.592+0000 7f7989ffb640 1 -- 172.21.15.67:0/49804349 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7984034040 con 0x7f798c1040a0 2024-09-21T06:36:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.593+0000 7f7989ffb640 1 -- 172.21.15.67:0/49804349 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7984030850 con 0x7f798c1040a0 2024-09-21T06:36:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.593+0000 7f7989ffb640 1 -- 172.21.15.67:0/49804349 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7984039430 con 0x7f798c1040a0 2024-09-21T06:36:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.593+0000 7f7993066640 1 -- 172.21.15.67:0/49804349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f798c1a1c50 con 0x7f798c1040a0 2024-09-21T06:36:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.593+0000 7f7993066640 1 -- 172.21.15.67:0/49804349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f798c1a2180 con 0x7f798c1040a0 2024-09-21T06:36:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.594+0000 7f7993066640 1 -- 172.21.15.67:0/49804349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f798c109850 con 0x7f798c1040a0 2024-09-21T06:36:36.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.595+0000 7f7989ffb640 1 -- 172.21.15.67:0/49804349 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f7984037020 con 0x7f798c1040a0 2024-09-21T06:36:36.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.596+0000 7f7989ffb640 1 --2- 172.21.15.67:0/49804349 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f79780b6590 0x7f79780b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:36.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.596+0000 7f7990e02640 1 --2- 172.21.15.67:0/49804349 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f79780b6590 0x7f79780b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:36.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.596+0000 7f7989ffb640 1 -- 172.21.15.67:0/49804349 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f7984108770 con 0x7f798c1040a0 2024-09-21T06:36:36.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.597+0000 7f7990e02640 1 --2- 172.21.15.67:0/49804349 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f79780b6590 0x7f79780b8aa0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7974006fd0 tx=0x7f7974008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:36.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.600+0000 7f7989ffb640 1 -- 172.21.15.67:0/49804349 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f79840098b0 con 0x7f798c1040a0 2024-09-21T06:36:36.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:36 smithi203 ceph-mon[33735]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:36:36.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.798+0000 7f7993066640 1 -- 172.21.15.67:0/49804349 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f798c063800 con 0x7f79780b6590 2024-09-21T06:36:36.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.800+0000 7f7989ffb640 1 -- 172.21.15.67:0/49804349 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f798c063800 con 0x7f79780b6590 2024-09-21T06:36:36.801 INFO:teuthology.orchestra.run.smithi067.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-21T06:36:36.801 INFO:teuthology.orchestra.run.smithi067.stdout:smithi067 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53440042400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-21T06:36:36.801 INFO:teuthology.orchestra.run.smithi067.stdout:smithi100 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400WR400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-21T06:36:36.801 INFO:teuthology.orchestra.run.smithi067.stdout:smithi203 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GK400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-21T06:36:36.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.802+0000 7f7993066640 1 -- 172.21.15.67:0/49804349 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f79780b6590 msgr2=0x7f79780b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:36.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.802+0000 7f7993066640 1 --2- 172.21.15.67:0/49804349 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f79780b6590 0x7f79780b8aa0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7974006fd0 tx=0x7f7974008040 comp rx=0 tx=0).stop 2024-09-21T06:36:36.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.802+0000 7f7993066640 1 -- 172.21.15.67:0/49804349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f798c1040a0 msgr2=0x7f798c1a11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:36.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.802+0000 7f7993066640 1 --2- 172.21.15.67:0/49804349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f798c1040a0 0x7f798c1a11c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f798402f800 tx=0x7f7984003f90 comp rx=0 tx=0).stop 2024-09-21T06:36:36.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.802+0000 7f7993066640 1 -- 172.21.15.67:0/49804349 shutdown_connections 2024-09-21T06:36:36.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.802+0000 7f7993066640 1 --2- 172.21.15.67:0/49804349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f798c103700 0x7f798c19ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:36.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.802+0000 7f7993066640 1 --2- 172.21.15.67:0/49804349 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f79780b6590 0x7f79780b8aa0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:36.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.802+0000 7f7993066640 1 --2- 172.21.15.67:0/49804349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f798c1040a0 0x7f798c1a11c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:36.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.802+0000 7f7993066640 1 --2- 172.21.15.67:0/49804349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f798c102670 0x7f798c06c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:36.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.802+0000 7f7993066640 1 -- 172.21.15.67:0/49804349 >> 172.21.15.67:0/49804349 conn(0x7f798c0fdfe0 msgr2=0x7f798c106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:36.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.802+0000 7f7993066640 1 -- 172.21.15.67:0/49804349 shutdown_connections 2024-09-21T06:36:36.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:36.802+0000 7f7993066640 1 -- 172.21.15.67:0/49804349 wait complete. 2024-09-21T06:36:36.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:36 smithi100 ceph-mon[33401]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:36:37.541 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:37 smithi067 ceph-mon[31232]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T06:36:37.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:37 smithi067 ceph-mon[31232]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-21T06:36:37.583 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T06:36:37.598 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi067.front.sepia.ceph.com 2024-09-21T06:36:37.598 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- bash -c 'ceph fs dump' 2024-09-21T06:36:37.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:37 smithi203 ceph-mon[33735]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T06:36:37.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:37 smithi203 ceph-mon[33735]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-21T06:36:37.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:37 smithi100 ceph-mon[33401]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T06:36:37.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:37 smithi100 ceph-mon[33401]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-21T06:36:37.858 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:36:38.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.386+0000 7f13d3b43640 1 -- 172.21.15.67:0/3134286470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13cc06b980 msgr2=0x7f13cc106390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:38.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.386+0000 7f13d3b43640 1 --2- 172.21.15.67:0/3134286470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13cc06b980 0x7f13cc106390 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f13c8009900 tx=0x7f13c802a410 comp rx=0 tx=0).stop 2024-09-21T06:36:38.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.387+0000 7f13d3b43640 1 -- 172.21.15.67:0/3134286470 shutdown_connections 2024-09-21T06:36:38.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.387+0000 7f13d3b43640 1 --2- 172.21.15.67:0/3134286470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13cc1068d0 0x7f13cc108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:38.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.387+0000 7f13d3b43640 1 --2- 172.21.15.67:0/3134286470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13cc109470 0x7f13cc10b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:38.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.387+0000 7f13d3b43640 1 --2- 172.21.15.67:0/3134286470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13cc06b980 0x7f13cc106390 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:38.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.387+0000 7f13d3b43640 1 -- 172.21.15.67:0/3134286470 >> 172.21.15.67:0/3134286470 conn(0x7f13cc0fdfb0 msgr2=0x7f13cc100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:38.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.387+0000 7f13d3b43640 1 -- 172.21.15.67:0/3134286470 shutdown_connections 2024-09-21T06:36:38.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.387+0000 7f13d3b43640 1 -- 172.21.15.67:0/3134286470 wait complete. 2024-09-21T06:36:38.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.387+0000 7f13d3b43640 1 Processor -- start 2024-09-21T06:36:38.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.387+0000 7f13d3b43640 1 -- start start 2024-09-21T06:36:38.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.388+0000 7f13d3b43640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13cc06b980 0x7f13cc199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:38.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.388+0000 7f13d3b43640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13cc1068d0 0x7f13cc19a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:38.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.388+0000 7f13d3b43640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13cc109470 0x7f13cc19b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:38.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.388+0000 7f13d3b43640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc10f230 con 0x7f13cc06b980 2024-09-21T06:36:38.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.388+0000 7f13d3b43640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc10f0b0 con 0x7f13cc1068d0 2024-09-21T06:36:38.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.388+0000 7f13d3b43640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc10f3b0 con 0x7f13cc109470 2024-09-21T06:36:38.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.388+0000 7f13d18df640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13cc06b980 0x7f13cc199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:38.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.388+0000 7f13d18df640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13cc06b980 0x7f13cc199eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41550/0 (socket says 172.21.15.67:41550) 2024-09-21T06:36:38.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.388+0000 7f13d18df640 1 -- 172.21.15.67:0/42520594 learned_addr learned my addr 172.21.15.67:0/42520594 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:38.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.388+0000 7f13d10de640 1 --2- 172.21.15.67:0/42520594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13cc1068d0 0x7f13cc19a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:38.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.388+0000 7f13d20e0640 1 --2- 172.21.15.67:0/42520594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13cc109470 0x7f13cc19b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:38.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.388+0000 7f13d18df640 1 -- 172.21.15.67:0/42520594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13cc109470 msgr2=0x7f13cc19b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:38.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.388+0000 7f13d18df640 1 --2- 172.21.15.67:0/42520594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13cc109470 0x7f13cc19b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:38.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.388+0000 7f13d18df640 1 -- 172.21.15.67:0/42520594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13cc1068d0 msgr2=0x7f13cc19a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:38.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.388+0000 7f13d18df640 1 --2- 172.21.15.67:0/42520594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13cc1068d0 0x7f13cc19a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:38.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.388+0000 7f13d18df640 1 -- 172.21.15.67:0/42520594 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13cc19b8e0 con 0x7f13cc06b980 2024-09-21T06:36:38.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.388+0000 7f13d20e0640 1 --2- 172.21.15.67:0/42520594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13cc109470 0x7f13cc19b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:36:38.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.389+0000 7f13d18df640 1 --2- 172.21.15.67:0/42520594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13cc06b980 0x7f13cc199eb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f13c8009230 tx=0x7f13c8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:38.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.389+0000 7f13c2ffd640 1 -- 172.21.15.67:0/42520594 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c803f030 con 0x7f13cc06b980 2024-09-21T06:36:38.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.389+0000 7f13d3b43640 1 -- 172.21.15.67:0/42520594 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13cc19bb70 con 0x7f13cc06b980 2024-09-21T06:36:38.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.389+0000 7f13c2ffd640 1 -- 172.21.15.67:0/42520594 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f13c80043d0 con 0x7f13cc06b980 2024-09-21T06:36:38.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.389+0000 7f13c2ffd640 1 -- 172.21.15.67:0/42520594 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c8035070 con 0x7f13cc06b980 2024-09-21T06:36:38.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.389+0000 7f13d3b43640 1 -- 172.21.15.67:0/42520594 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13cc1a22d0 con 0x7f13cc06b980 2024-09-21T06:36:38.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.390+0000 7f13d3b43640 1 -- 172.21.15.67:0/42520594 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13cc0ffad0 con 0x7f13cc06b980 2024-09-21T06:36:38.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.392+0000 7f13c2ffd640 1 -- 172.21.15.67:0/42520594 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f13c8042030 con 0x7f13cc06b980 2024-09-21T06:36:38.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.392+0000 7f13c2ffd640 1 --2- 172.21.15.67:0/42520594 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f13b80b6510 0x7f13b80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:38.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.393+0000 7f13d10de640 1 --2- 172.21.15.67:0/42520594 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f13b80b6510 0x7f13b80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:38.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.393+0000 7f13c2ffd640 1 -- 172.21.15.67:0/42520594 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f13c8108090 con 0x7f13cc06b980 2024-09-21T06:36:38.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.393+0000 7f13d10de640 1 --2- 172.21.15.67:0/42520594 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f13b80b6510 0x7f13b80b8a20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f13bc009b60 tx=0x7f13bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:38.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.396+0000 7f13c2ffd640 1 -- 172.21.15.67:0/42520594 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f13c80ce550 con 0x7f13cc06b980 2024-09-21T06:36:38.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:38 smithi067 ceph-mon[31232]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:36:38.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.623+0000 7f13d3b43640 1 -- 172.21.15.67:0/42520594 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f13cc106ce0 con 0x7f13cc06b980 2024-09-21T06:36:38.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.623+0000 7f13c2ffd640 1 -- 172.21.15.67:0/42520594 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) ==== 76+0+3054 (secure 0 0 0) 0x7f13c80d2930 con 0x7f13cc06b980 2024-09-21T06:36:38.625 INFO:teuthology.orchestra.run.smithi067.stdout:e21 2024-09-21T06:36:38.625 INFO:teuthology.orchestra.run.smithi067.stdout:btime 2024-09-21T06:35:10:013981+0000 2024-09-21T06:36:38.625 INFO:teuthology.orchestra.run.smithi067.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T06:36:38.625 INFO:teuthology.orchestra.run.smithi067.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T06:36:38.625 INFO:teuthology.orchestra.run.smithi067.stdout:legacy client fscid: 1 2024-09-21T06:36:38.625 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:36:38.626 INFO:teuthology.orchestra.run.smithi067.stdout:Filesystem 'cephfs' (1) 2024-09-21T06:36:38.626 INFO:teuthology.orchestra.run.smithi067.stdout:fs_name cephfs 2024-09-21T06:36:38.626 INFO:teuthology.orchestra.run.smithi067.stdout:epoch 21 2024-09-21T06:36:38.626 INFO:teuthology.orchestra.run.smithi067.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-21T06:36:38.626 INFO:teuthology.orchestra.run.smithi067.stdout:created 2024-09-21T06:34:33.306959+0000 2024-09-21T06:36:38.626 INFO:teuthology.orchestra.run.smithi067.stdout:modified 2024-09-21T06:35:09.805211+0000 2024-09-21T06:36:38.626 INFO:teuthology.orchestra.run.smithi067.stdout:tableserver 0 2024-09-21T06:36:38.626 INFO:teuthology.orchestra.run.smithi067.stdout:root 0 2024-09-21T06:36:38.626 INFO:teuthology.orchestra.run.smithi067.stdout:session_timeout 300 2024-09-21T06:36:38.626 INFO:teuthology.orchestra.run.smithi067.stdout:session_autoclose 300 2024-09-21T06:36:38.626 INFO:teuthology.orchestra.run.smithi067.stdout:max_file_size 1099511627776 2024-09-21T06:36:38.626 INFO:teuthology.orchestra.run.smithi067.stdout:max_xattr_size 65536 2024-09-21T06:36:38.626 INFO:teuthology.orchestra.run.smithi067.stdout:required_client_features {} 2024-09-21T06:36:38.626 INFO:teuthology.orchestra.run.smithi067.stdout:last_failure 0 2024-09-21T06:36:38.627 INFO:teuthology.orchestra.run.smithi067.stdout:last_failure_osd_epoch 0 2024-09-21T06:36:38.627 INFO:teuthology.orchestra.run.smithi067.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T06:36:38.627 INFO:teuthology.orchestra.run.smithi067.stdout:max_mds 1 2024-09-21T06:36:38.627 INFO:teuthology.orchestra.run.smithi067.stdout:in 0 2024-09-21T06:36:38.627 INFO:teuthology.orchestra.run.smithi067.stdout:up {0=24671} 2024-09-21T06:36:38.627 INFO:teuthology.orchestra.run.smithi067.stdout:failed 2024-09-21T06:36:38.627 INFO:teuthology.orchestra.run.smithi067.stdout:damaged 2024-09-21T06:36:38.627 INFO:teuthology.orchestra.run.smithi067.stdout:stopped 2024-09-21T06:36:38.627 INFO:teuthology.orchestra.run.smithi067.stdout:data_pools [3,4] 2024-09-21T06:36:38.627 INFO:teuthology.orchestra.run.smithi067.stdout:metadata_pool 2 2024-09-21T06:36:38.627 INFO:teuthology.orchestra.run.smithi067.stdout:inline_data disabled 2024-09-21T06:36:38.627 INFO:teuthology.orchestra.run.smithi067.stdout:balancer 2024-09-21T06:36:38.627 INFO:teuthology.orchestra.run.smithi067.stdout:bal_rank_mask -1 2024-09-21T06:36:38.627 INFO:teuthology.orchestra.run.smithi067.stdout:standby_count_wanted 0 2024-09-21T06:36:38.628 INFO:teuthology.orchestra.run.smithi067.stdout:qdb_cluster leader: 24671 members: 24671 2024-09-21T06:36:38.628 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.b{0:24671} state up:active seq 4 addr [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:36:38.628 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.h{0:24641} state up:standby-replay seq 1 addr [v2:172.21.15.100:6836/259395872,v1:172.21.15.100:6837/259395872] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:36:38.628 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:36:38.628 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:36:38.628 INFO:teuthology.orchestra.run.smithi067.stdout:Standby daemons: 2024-09-21T06:36:38.628 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:36:38.628 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.j{-1:14733} state up:standby seq 1 addr [v2:172.21.15.67:6834/4145954943,v1:172.21.15.67:6835/4145954943] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:36:38.628 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.d{-1:14745} state up:standby seq 1 addr [v2:172.21.15.67:6836/1744346240,v1:172.21.15.67:6837/1744346240] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:36:38.628 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.g{-1:14796} state up:standby seq 1 addr [v2:172.21.15.67:6838/1168518926,v1:172.21.15.67:6839/1168518926] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:36:38.628 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.a{-1:14808} state up:standby seq 1 addr [v2:172.21.15.67:6840/2331956394,v1:172.21.15.67:6841/2331956394] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:36:38.628 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.l{-1:24574} state up:standby seq 1 addr [v2:172.21.15.203:6832/3852485418,v1:172.21.15.203:6833/3852485418] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:36:38.628 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.k{-1:24587} state up:standby seq 1 addr [v2:172.21.15.100:6832/4088716909,v1:172.21.15.100:6833/4088716909] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:36:38.629 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.f{-1:24616} state up:standby seq 1 addr [v2:172.21.15.203:6834/2689490358,v1:172.21.15.203:6835/2689490358] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:36:38.629 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.e{-1:24620} state up:standby seq 1 addr [v2:172.21.15.100:6834/1518007169,v1:172.21.15.100:6835/1518007169] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:36:38.629 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.c{-1:24622} state up:standby seq 1 addr [v2:172.21.15.203:6836/1426013144,v1:172.21.15.203:6837/1426013144] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:36:38.629 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.i{-1:24637} state up:standby seq 1 addr [v2:172.21.15.203:6838/2443174511,v1:172.21.15.203:6839/2443174511] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:36:38.629 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:36:38.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.627+0000 7f13d3b43640 1 -- 172.21.15.67:0/42520594 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f13b80b6510 msgr2=0x7f13b80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:38.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.627+0000 7f13d3b43640 1 --2- 172.21.15.67:0/42520594 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f13b80b6510 0x7f13b80b8a20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f13bc009b60 tx=0x7f13bc005f90 comp rx=0 tx=0).stop 2024-09-21T06:36:38.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.627+0000 7f13d3b43640 1 -- 172.21.15.67:0/42520594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13cc06b980 msgr2=0x7f13cc199eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:38.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.627+0000 7f13d3b43640 1 --2- 172.21.15.67:0/42520594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13cc06b980 0x7f13cc199eb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f13c8009230 tx=0x7f13c8005e80 comp rx=0 tx=0).stop 2024-09-21T06:36:38.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.627+0000 7f13d3b43640 1 -- 172.21.15.67:0/42520594 shutdown_connections 2024-09-21T06:36:38.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.627+0000 7f13d3b43640 1 --2- 172.21.15.67:0/42520594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13cc1068d0 0x7f13cc19a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:38.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.627+0000 7f13d3b43640 1 --2- 172.21.15.67:0/42520594 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f13b80b6510 0x7f13b80b8a20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:38.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.627+0000 7f13d3b43640 1 --2- 172.21.15.67:0/42520594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13cc109470 0x7f13cc19b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:38.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.627+0000 7f13d3b43640 1 --2- 172.21.15.67:0/42520594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13cc06b980 0x7f13cc199eb0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:38.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.627+0000 7f13d3b43640 1 -- 172.21.15.67:0/42520594 >> 172.21.15.67:0/42520594 conn(0x7f13cc0fdfb0 msgr2=0x7f13cc104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:38.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.627+0000 7f13d3b43640 1 -- 172.21.15.67:0/42520594 shutdown_connections 2024-09-21T06:36:38.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:38.627+0000 7f13d3b43640 1 -- 172.21.15.67:0/42520594 wait complete. 2024-09-21T06:36:38.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:38 smithi203 ceph-mon[33735]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:36:38.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:38 smithi100 ceph-mon[33401]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:36:39.369 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- bash -c 'ceph osd dump' 2024-09-21T06:36:39.631 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:36:39.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:39 smithi203 ceph-mon[33735]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-21T06:36:39.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:39 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/42520594' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T06:36:39.752 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:39 smithi067 ceph-mon[31232]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-21T06:36:39.752 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:39 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/42520594' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T06:36:39.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:39 smithi100 ceph-mon[33401]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-21T06:36:39.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:39 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/42520594' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T06:36:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.128+0000 7f956a22e640 1 -- 172.21.15.67:0/444627246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9564103760 msgr2=0x7f9564103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.128+0000 7f956a22e640 1 --2- 172.21.15.67:0/444627246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9564103760 0x7f9564103bd0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9554009900 tx=0x7f955402a480 comp rx=0 tx=0).stop 2024-09-21T06:36:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.128+0000 7f956a22e640 1 -- 172.21.15.67:0/444627246 shutdown_connections 2024-09-21T06:36:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.128+0000 7f956a22e640 1 --2- 172.21.15.67:0/444627246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95641047f0 0x7f9564104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.128+0000 7f956a22e640 1 --2- 172.21.15.67:0/444627246 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9564105190 0x7f956410bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.128+0000 7f956a22e640 1 --2- 172.21.15.67:0/444627246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9564103760 0x7f9564103bd0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.128+0000 7f956a22e640 1 -- 172.21.15.67:0/444627246 >> 172.21.15.67:0/444627246 conn(0x7f95640ff0b0 msgr2=0x7f9564101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.128+0000 7f956a22e640 1 -- 172.21.15.67:0/444627246 shutdown_connections 2024-09-21T06:36:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.128+0000 7f956a22e640 1 -- 172.21.15.67:0/444627246 wait complete. 2024-09-21T06:36:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.129+0000 7f956a22e640 1 Processor -- start 2024-09-21T06:36:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.129+0000 7f956a22e640 1 -- start start 2024-09-21T06:36:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.129+0000 7f956a22e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9564103760 0x7f9564199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.129+0000 7f956a22e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95641047f0 0x7f956419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.129+0000 7f956a22e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9564105190 0x7f956419b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.130+0000 7f956a22e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f956410f230 con 0x7f9564105190 2024-09-21T06:36:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.130+0000 7f956a22e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f956410f0b0 con 0x7f9564103760 2024-09-21T06:36:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.130+0000 7f956a22e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f956410f3b0 con 0x7f95641047f0 2024-09-21T06:36:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.130+0000 7f9563fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9564105190 0x7f956419b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.130+0000 7f9563fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9564105190 0x7f956419b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41564/0 (socket says 172.21.15.67:41564) 2024-09-21T06:36:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.130+0000 7f9563fff640 1 -- 172.21.15.67:0/2011952450 learned_addr learned my addr 172.21.15.67:0/2011952450 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.130+0000 7f9562ffd640 1 --2- 172.21.15.67:0/2011952450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95641047f0 0x7f956419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.130+0000 7f95637fe640 1 --2- 172.21.15.67:0/2011952450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9564103760 0x7f9564199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.130+0000 7f9563fff640 1 -- 172.21.15.67:0/2011952450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95641047f0 msgr2=0x7f956419a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.130+0000 7f9563fff640 1 --2- 172.21.15.67:0/2011952450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95641047f0 0x7f956419a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.130+0000 7f9563fff640 1 -- 172.21.15.67:0/2011952450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9564103760 msgr2=0x7f9564199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.130+0000 7f9563fff640 1 --2- 172.21.15.67:0/2011952450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9564103760 0x7f9564199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.130+0000 7f9563fff640 1 -- 172.21.15.67:0/2011952450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f956419b8e0 con 0x7f9564105190 2024-09-21T06:36:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.130+0000 7f9562ffd640 1 --2- 172.21.15.67:0/2011952450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95641047f0 0x7f956419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:36:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.130+0000 7f9563fff640 1 --2- 172.21.15.67:0/2011952450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9564105190 0x7f956419b0c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f955c00eaa0 tx=0x7f955c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:40.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.130+0000 7f9560ff9640 1 -- 172.21.15.67:0/2011952450 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c00cdb0 con 0x7f9564105190 2024-09-21T06:36:40.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.130+0000 7f9560ff9640 1 -- 172.21.15.67:0/2011952450 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f955c005020 con 0x7f9564105190 2024-09-21T06:36:40.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.130+0000 7f956a22e640 1 -- 172.21.15.67:0/2011952450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f956419bbd0 con 0x7f9564105190 2024-09-21T06:36:40.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.130+0000 7f9560ff9640 1 -- 172.21.15.67:0/2011952450 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c0116d0 con 0x7f9564105190 2024-09-21T06:36:40.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.131+0000 7f956a22e640 1 -- 172.21.15.67:0/2011952450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95641a22d0 con 0x7f9564105190 2024-09-21T06:36:40.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.131+0000 7f956a22e640 1 -- 172.21.15.67:0/2011952450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f956410a940 con 0x7f9564105190 2024-09-21T06:36:40.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.134+0000 7f9560ff9640 1 -- 172.21.15.67:0/2011952450 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f955c0051d0 con 0x7f9564105190 2024-09-21T06:36:40.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.135+0000 7f9560ff9640 1 --2- 172.21.15.67:0/2011952450 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f95500b6510 0x7f95500b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:40.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.135+0000 7f95637fe640 1 --2- 172.21.15.67:0/2011952450 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f95500b6510 0x7f95500b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:40.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.135+0000 7f9560ff9640 1 -- 172.21.15.67:0/2011952450 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f955c0e7d10 con 0x7f9564105190 2024-09-21T06:36:40.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.136+0000 7f95637fe640 1 --2- 172.21.15.67:0/2011952450 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f95500b6510 0x7f95500b8a20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9554000c00 tx=0x7f9554005720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:40.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.138+0000 7f9560ff9640 1 -- 172.21.15.67:0/2011952450 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f955c016030 con 0x7f9564105190 2024-09-21T06:36:40.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.328+0000 7f956a22e640 1 -- 172.21.15.67:0/2011952450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f956402e640 con 0x7f9564105190 2024-09-21T06:36:40.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.329+0000 7f9560ff9640 1 -- 172.21.15.67:0/2011952450 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v85) ==== 56+0+5713 (secure 0 0 0) 0x7f955c0ae1d0 con 0x7f9564105190 2024-09-21T06:36:40.331 INFO:teuthology.orchestra.run.smithi067.stdout:epoch 85 2024-09-21T06:36:40.331 INFO:teuthology.orchestra.run.smithi067.stdout:fsid 2a791768-77e2-11ef-baf5-efdc52797490 2024-09-21T06:36:40.331 INFO:teuthology.orchestra.run.smithi067.stdout:created 2024-09-21T06:26:15.222283+0000 2024-09-21T06:36:40.332 INFO:teuthology.orchestra.run.smithi067.stdout:modified 2024-09-21T06:36:05.617384+0000 2024-09-21T06:36:40.332 INFO:teuthology.orchestra.run.smithi067.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-21T06:36:40.332 INFO:teuthology.orchestra.run.smithi067.stdout:crush_version 27 2024-09-21T06:36:40.332 INFO:teuthology.orchestra.run.smithi067.stdout:full_ratio 0.95 2024-09-21T06:36:40.332 INFO:teuthology.orchestra.run.smithi067.stdout:backfillfull_ratio 0.9 2024-09-21T06:36:40.332 INFO:teuthology.orchestra.run.smithi067.stdout:nearfull_ratio 0.85 2024-09-21T06:36:40.332 INFO:teuthology.orchestra.run.smithi067.stdout:require_min_compat_client luminous 2024-09-21T06:36:40.332 INFO:teuthology.orchestra.run.smithi067.stdout:min_compat_client luminous 2024-09-21T06:36:40.332 INFO:teuthology.orchestra.run.smithi067.stdout:require_osd_release squid 2024-09-21T06:36:40.332 INFO:teuthology.orchestra.run.smithi067.stdout:stretch_mode_enabled false 2024-09-21T06:36:40.333 INFO:teuthology.orchestra.run.smithi067.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-21T06:36:40.333 INFO:teuthology.orchestra.run.smithi067.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-21T06:36:40.333 INFO:teuthology.orchestra.run.smithi067.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-21T06:36:40.333 INFO:teuthology.orchestra.run.smithi067.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 81 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-21T06:36:40.333 INFO:teuthology.orchestra.run.smithi067.stdout:max_osd 12 2024-09-21T06:36:40.333 INFO:teuthology.orchestra.run.smithi067.stdout:osd.0 up in weight 1 up_from 8 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6802/2310759946,v1:172.21.15.67:6803/2310759946] [v2:172.21.15.67:6804/2310759946,v1:172.21.15.67:6805/2310759946] exists,up 18778364-82ed-41c0-92da-ce44cdad7358 2024-09-21T06:36:40.333 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1 up in weight 1 up_from 13 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6800/161807364,v1:172.21.15.100:6801/161807364] [v2:172.21.15.100:6802/161807364,v1:172.21.15.100:6803/161807364] exists,up 84e8d0c2-1c45-4673-ad49-f61df667d7f7 2024-09-21T06:36:40.333 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2 up in weight 1 up_from 18 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6800/2754787169,v1:172.21.15.203:6801/2754787169] [v2:172.21.15.203:6802/2754787169,v1:172.21.15.203:6803/2754787169] exists,up e9a4d919-a779-4a1b-9b5b-d03a65f28ddf 2024-09-21T06:36:40.333 INFO:teuthology.orchestra.run.smithi067.stdout:osd.3 up in weight 1 up_from 24 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6810/4007517951,v1:172.21.15.67:6811/4007517951] [v2:172.21.15.67:6812/4007517951,v1:172.21.15.67:6813/4007517951] exists,up 3f7d9abd-587c-47ad-bf43-11f7dcdf71c9 2024-09-21T06:36:40.333 INFO:teuthology.orchestra.run.smithi067.stdout:osd.4 up in weight 1 up_from 30 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6808/3773646501,v1:172.21.15.100:6809/3773646501] [v2:172.21.15.100:6810/3773646501,v1:172.21.15.100:6811/3773646501] exists,up 94f3a075-6d8a-4907-8418-a8513b35779a 2024-09-21T06:36:40.334 INFO:teuthology.orchestra.run.smithi067.stdout:osd.5 up in weight 1 up_from 35 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6808/2096171166,v1:172.21.15.203:6809/2096171166] [v2:172.21.15.203:6810/2096171166,v1:172.21.15.203:6811/2096171166] exists,up 5d842edf-06ad-49ae-ac71-863e5657270c 2024-09-21T06:36:40.334 INFO:teuthology.orchestra.run.smithi067.stdout:osd.6 up in weight 1 up_from 41 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6818/3229936280,v1:172.21.15.67:6819/3229936280] [v2:172.21.15.67:6820/3229936280,v1:172.21.15.67:6821/3229936280] exists,up 85b9beed-c93d-48ba-bc7a-c4cd816fbdff 2024-09-21T06:36:40.334 INFO:teuthology.orchestra.run.smithi067.stdout:osd.7 up in weight 1 up_from 46 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6816/1622917909,v1:172.21.15.100:6817/1622917909] [v2:172.21.15.100:6818/1622917909,v1:172.21.15.100:6819/1622917909] exists,up 86de30c2-b8ee-4c9d-9c17-8a56517934ee 2024-09-21T06:36:40.334 INFO:teuthology.orchestra.run.smithi067.stdout:osd.8 up in weight 1 up_from 52 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6816/3452445943,v1:172.21.15.203:6817/3452445943] [v2:172.21.15.203:6818/3452445943,v1:172.21.15.203:6819/3452445943] exists,up 0e96cdad-a135-4115-9e2d-9a28306700e9 2024-09-21T06:36:40.334 INFO:teuthology.orchestra.run.smithi067.stdout:osd.9 up in weight 1 up_from 58 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6826/2998307533,v1:172.21.15.67:6827/2998307533] [v2:172.21.15.67:6828/2998307533,v1:172.21.15.67:6829/2998307533] exists,up 95704ed6-6286-4094-9c6d-23919c7005ca 2024-09-21T06:36:40.334 INFO:teuthology.orchestra.run.smithi067.stdout:osd.10 up in weight 1 up_from 63 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6824/2819057142,v1:172.21.15.100:6825/2819057142] [v2:172.21.15.100:6826/2819057142,v1:172.21.15.100:6827/2819057142] exists,up bd2a7d33-b935-46c8-95c3-d76f28242ffe 2024-09-21T06:36:40.334 INFO:teuthology.orchestra.run.smithi067.stdout:osd.11 up in weight 1 up_from 69 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6824/566502600,v1:172.21.15.203:6825/566502600] [v2:172.21.15.203:6826/566502600,v1:172.21.15.203:6827/566502600] exists,up ea591507-10e2-4c08-8d29-d38ba9be96ef 2024-09-21T06:36:40.334 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 2.b [8,11] 2024-09-21T06:36:40.334 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 2.10 [2,1] 2024-09-21T06:36:40.334 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 2.16 [2,4] 2024-09-21T06:36:40.334 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 2.1d [8,11] 2024-09-21T06:36:40.334 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 3.6 [4,7] 2024-09-21T06:36:40.335 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 3.1d [4,8] 2024-09-21T06:36:40.335 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 3.21 [5,9] 2024-09-21T06:36:40.335 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 3.22 [3,9] 2024-09-21T06:36:40.335 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 3.25 [5,9] 2024-09-21T06:36:40.335 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 3.3a [4,6] 2024-09-21T06:36:40.335 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 4.2 [9,1] 2024-09-21T06:36:40.335 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 4.18 [9,8] 2024-09-21T06:36:40.335 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 4.19 [9,8] 2024-09-21T06:36:40.335 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 4.1a [5,10] 2024-09-21T06:36:40.335 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:6800/1181476774 expires 2024-09-22T06:27:03.740475+0000 2024-09-21T06:36:40.335 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/3100321138 expires 2024-09-22T06:27:03.740475+0000 2024-09-21T06:36:40.336 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/2309364298 expires 2024-09-22T06:27:03.740475+0000 2024-09-21T06:36:40.336 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/3458717076 expires 2024-09-22T06:27:03.740475+0000 2024-09-21T06:36:40.336 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/4110206171 expires 2024-09-22T06:26:37.041377+0000 2024-09-21T06:36:40.336 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:6801/1181476774 expires 2024-09-22T06:27:03.740475+0000 2024-09-21T06:36:40.336 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/2910038982 expires 2024-09-22T06:26:37.041377+0000 2024-09-21T06:36:40.336 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:6801/2622729103 expires 2024-09-22T06:26:37.041377+0000 2024-09-21T06:36:40.336 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:6800/2622729103 expires 2024-09-22T06:26:37.041377+0000 2024-09-21T06:36:40.336 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/3346784528 expires 2024-09-22T06:26:37.041377+0000 2024-09-21T06:36:40.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.332+0000 7f956a22e640 1 -- 172.21.15.67:0/2011952450 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f95500b6510 msgr2=0x7f95500b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:40.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.332+0000 7f956a22e640 1 --2- 172.21.15.67:0/2011952450 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f95500b6510 0x7f95500b8a20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9554000c00 tx=0x7f9554005720 comp rx=0 tx=0).stop 2024-09-21T06:36:40.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.332+0000 7f956a22e640 1 -- 172.21.15.67:0/2011952450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9564105190 msgr2=0x7f956419b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:40.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.332+0000 7f956a22e640 1 --2- 172.21.15.67:0/2011952450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9564105190 0x7f956419b0c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f955c00eaa0 tx=0x7f955c00ef60 comp rx=0 tx=0).stop 2024-09-21T06:36:40.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.332+0000 7f956a22e640 1 -- 172.21.15.67:0/2011952450 shutdown_connections 2024-09-21T06:36:40.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.332+0000 7f956a22e640 1 --2- 172.21.15.67:0/2011952450 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f95500b6510 0x7f95500b8a20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:40.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.332+0000 7f956a22e640 1 --2- 172.21.15.67:0/2011952450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9564105190 0x7f956419b0c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:40.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.332+0000 7f956a22e640 1 --2- 172.21.15.67:0/2011952450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95641047f0 0x7f956419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:40.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.332+0000 7f956a22e640 1 --2- 172.21.15.67:0/2011952450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9564103760 0x7f9564199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:40.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.332+0000 7f956a22e640 1 -- 172.21.15.67:0/2011952450 >> 172.21.15.67:0/2011952450 conn(0x7f95640ff0b0 msgr2=0x7f9564107b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:40.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.332+0000 7f956a22e640 1 -- 172.21.15.67:0/2011952450 shutdown_connections 2024-09-21T06:36:40.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:40.332+0000 7f956a22e640 1 -- 172.21.15.67:0/2011952450 wait complete. 2024-09-21T06:36:40.708 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:40 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2011952450' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T06:36:40.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:40 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2011952450' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T06:36:40.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:40 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2011952450' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T06:36:41.066 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-21T06:36:41.323 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:36:41.588 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:36:41.709 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:41 smithi067 ceph-mon[31232]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-21T06:36:41.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:41 smithi203 ceph-mon[33735]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-21T06:36:41.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:41 smithi100 ceph-mon[33401]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-21T06:36:42.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.127+0000 7f9f642c0640 1 -- 172.21.15.67:0/1944813710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f5c105380 msgr2=0x7f9f5c10d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:42.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.127+0000 7f9f642c0640 1 --2- 172.21.15.67:0/1944813710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f5c105380 0x7f9f5c10d8d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c009900 tx=0x7f9f4c02a410 comp rx=0 tx=0).stop 2024-09-21T06:36:42.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.127+0000 7f9f642c0640 1 -- 172.21.15.67:0/1944813710 shutdown_connections 2024-09-21T06:36:42.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.127+0000 7f9f642c0640 1 --2- 172.21.15.67:0/1944813710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f5c10de60 0x7f9f5c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:42.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.127+0000 7f9f642c0640 1 --2- 172.21.15.67:0/1944813710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f5c105380 0x7f9f5c10d8d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:42.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.127+0000 7f9f642c0640 1 --2- 172.21.15.67:0/1944813710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f5c104a30 0x7f9f5c104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:42.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.127+0000 7f9f642c0640 1 -- 172.21.15.67:0/1944813710 >> 172.21.15.67:0/1944813710 conn(0x7f9f5c079760 msgr2=0x7f9f5c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:42.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.127+0000 7f9f642c0640 1 -- 172.21.15.67:0/1944813710 shutdown_connections 2024-09-21T06:36:42.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.127+0000 7f9f642c0640 1 -- 172.21.15.67:0/1944813710 wait complete. 2024-09-21T06:36:42.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.128+0000 7f9f642c0640 1 Processor -- start 2024-09-21T06:36:42.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.128+0000 7f9f642c0640 1 -- start start 2024-09-21T06:36:42.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.128+0000 7f9f642c0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f5c104a30 0x7f9f5c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:42.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.128+0000 7f9f642c0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f5c105380 0x7f9f5c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:42.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.128+0000 7f9f642c0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f5c10de60 0x7f9f5c19fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:42.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.128+0000 7f9f642c0640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c113950 con 0x7f9f5c105380 2024-09-21T06:36:42.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.128+0000 7f9f642c0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c1137d0 con 0x7f9f5c104a30 2024-09-21T06:36:42.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.128+0000 7f9f642c0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c113ad0 con 0x7f9f5c10de60 2024-09-21T06:36:42.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.128+0000 7f9f6185b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f5c105380 0x7f9f5c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:42.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.128+0000 7f9f6185b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f5c105380 0x7f9f5c19aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41578/0 (socket says 172.21.15.67:41578) 2024-09-21T06:36:42.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.128+0000 7f9f6185b640 1 -- 172.21.15.67:0/3574767133 learned_addr learned my addr 172.21.15.67:0/3574767133 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:42.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.129+0000 7f9f6285d640 1 --2- 172.21.15.67:0/3574767133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f5c10de60 0x7f9f5c19fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:42.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.129+0000 7f9f6205c640 1 --2- 172.21.15.67:0/3574767133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f5c104a30 0x7f9f5c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:42.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.129+0000 7f9f6185b640 1 -- 172.21.15.67:0/3574767133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f5c10de60 msgr2=0x7f9f5c19fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:42.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.129+0000 7f9f6185b640 1 --2- 172.21.15.67:0/3574767133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f5c10de60 0x7f9f5c19fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:42.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.129+0000 7f9f6185b640 1 -- 172.21.15.67:0/3574767133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f5c104a30 msgr2=0x7f9f5c19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:42.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.129+0000 7f9f6185b640 1 --2- 172.21.15.67:0/3574767133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f5c104a30 0x7f9f5c19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:42.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.129+0000 7f9f6185b640 1 -- 172.21.15.67:0/3574767133 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5c1a03f0 con 0x7f9f5c105380 2024-09-21T06:36:42.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.129+0000 7f9f6185b640 1 --2- 172.21.15.67:0/3574767133 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f5c105380 0x7f9f5c19aa20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c004770 tx=0x7f9f4c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:42.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.129+0000 7f9f537fe640 1 -- 172.21.15.67:0/3574767133 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c03f030 con 0x7f9f5c105380 2024-09-21T06:36:42.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.129+0000 7f9f642c0640 1 -- 172.21.15.67:0/3574767133 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f5c1a0680 con 0x7f9f5c105380 2024-09-21T06:36:42.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.129+0000 7f9f537fe640 1 -- 172.21.15.67:0/3574767133 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f4c004440 con 0x7f9f5c105380 2024-09-21T06:36:42.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.129+0000 7f9f537fe640 1 -- 172.21.15.67:0/3574767133 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c035070 con 0x7f9f5c105380 2024-09-21T06:36:42.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.129+0000 7f9f642c0640 1 -- 172.21.15.67:0/3574767133 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f5c1a6e40 con 0x7f9f5c105380 2024-09-21T06:36:42.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.131+0000 7f9f642c0640 1 -- 172.21.15.67:0/3574767133 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f5c06c1f0 con 0x7f9f5c105380 2024-09-21T06:36:42.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.132+0000 7f9f537fe640 1 -- 172.21.15.67:0/3574767133 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f9f4c042030 con 0x7f9f5c105380 2024-09-21T06:36:42.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.133+0000 7f9f537fe640 1 --2- 172.21.15.67:0/3574767133 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9f480b6510 0x7f9f480b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:42.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.133+0000 7f9f6205c640 1 --2- 172.21.15.67:0/3574767133 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9f480b6510 0x7f9f480b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:42.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.133+0000 7f9f537fe640 1 -- 172.21.15.67:0/3574767133 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f9f4c107f10 con 0x7f9f5c105380 2024-09-21T06:36:42.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.134+0000 7f9f6205c640 1 --2- 172.21.15.67:0/3574767133 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9f480b6510 0x7f9f480b8a20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c06c8f0 tx=0x7f9f58005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:42.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.137+0000 7f9f537fe640 1 -- 172.21.15.67:0/3574767133 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9f4c0ce370 con 0x7f9f5c105380 2024-09-21T06:36:42.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.363+0000 7f9f642c0640 1 -- 172.21.15.67:0/3574767133 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f5c104e40 con 0x7f9f5c105380 2024-09-21T06:36:42.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.363+0000 7f9f537fe640 1 -- 172.21.15.67:0/3574767133 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7f9f4c0e3e20 con 0x7f9f5c105380 2024-09-21T06:36:42.366 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:36:42.366 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:36:42.366 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:36:42.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.366+0000 7f9f642c0640 1 -- 172.21.15.67:0/3574767133 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9f480b6510 msgr2=0x7f9f480b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:42.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.366+0000 7f9f642c0640 1 --2- 172.21.15.67:0/3574767133 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9f480b6510 0x7f9f480b8a20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c06c8f0 tx=0x7f9f58005f90 comp rx=0 tx=0).stop 2024-09-21T06:36:42.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.366+0000 7f9f642c0640 1 -- 172.21.15.67:0/3574767133 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f5c105380 msgr2=0x7f9f5c19aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:42.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.366+0000 7f9f642c0640 1 --2- 172.21.15.67:0/3574767133 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f5c105380 0x7f9f5c19aa20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c004770 tx=0x7f9f4c004040 comp rx=0 tx=0).stop 2024-09-21T06:36:42.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.366+0000 7f9f642c0640 1 -- 172.21.15.67:0/3574767133 shutdown_connections 2024-09-21T06:36:42.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.366+0000 7f9f642c0640 1 --2- 172.21.15.67:0/3574767133 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9f480b6510 0x7f9f480b8a20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:42.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.366+0000 7f9f642c0640 1 --2- 172.21.15.67:0/3574767133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f5c10de60 0x7f9f5c19fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:42.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.366+0000 7f9f642c0640 1 --2- 172.21.15.67:0/3574767133 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f5c105380 0x7f9f5c19aa20 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:42.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.366+0000 7f9f642c0640 1 --2- 172.21.15.67:0/3574767133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f5c104a30 0x7f9f5c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:42.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.366+0000 7f9f642c0640 1 -- 172.21.15.67:0/3574767133 >> 172.21.15.67:0/3574767133 conn(0x7f9f5c079760 msgr2=0x7f9f5c1009c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:42.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.366+0000 7f9f642c0640 1 -- 172.21.15.67:0/3574767133 shutdown_connections 2024-09-21T06:36:42.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:42.366+0000 7f9f642c0640 1 -- 172.21.15.67:0/3574767133 wait complete. 2024-09-21T06:36:42.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:42 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3574767133' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:36:42.771 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:42 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3574767133' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:36:42.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:42 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3574767133' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:36:43.065 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:36:43.323 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:36:43.705 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:43 smithi067 ceph-mon[31232]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T06:36:43.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:43 smithi203 ceph-mon[33735]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T06:36:43.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:43 smithi100 ceph-mon[33401]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T06:36:43.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.794+0000 7f8dc5fcd640 1 -- 172.21.15.67:0/2244704423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc0102ac0 msgr2=0x7f8dc0102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:43.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.794+0000 7f8dc5fcd640 1 --2- 172.21.15.67:0/2244704423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc0102ac0 0x7f8dc0102ed0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8db0009900 tx=0x7f8db002a410 comp rx=0 tx=0).stop 2024-09-21T06:36:43.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.794+0000 7f8dc5fcd640 1 -- 172.21.15.67:0/2244704423 shutdown_connections 2024-09-21T06:36:43.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.794+0000 7f8dc5fcd640 1 --2- 172.21.15.67:0/2244704423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8dc0108d00 0x7f8dc010f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:43.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.794+0000 7f8dc5fcd640 1 --2- 172.21.15.67:0/2244704423 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dc01083b0 0x7f8dc01087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:43.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.794+0000 7f8dc5fcd640 1 --2- 172.21.15.67:0/2244704423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc0102ac0 0x7f8dc0102ed0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:43.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.794+0000 7f8dc5fcd640 1 -- 172.21.15.67:0/2244704423 >> 172.21.15.67:0/2244704423 conn(0x7f8dc0079720 msgr2=0x7f8dc0079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:43.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.794+0000 7f8dc5fcd640 1 -- 172.21.15.67:0/2244704423 shutdown_connections 2024-09-21T06:36:43.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.794+0000 7f8dc5fcd640 1 -- 172.21.15.67:0/2244704423 wait complete. 2024-09-21T06:36:43.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.795+0000 7f8dc5fcd640 1 Processor -- start 2024-09-21T06:36:43.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.795+0000 7f8dc5fcd640 1 -- start start 2024-09-21T06:36:43.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.795+0000 7f8dc5fcd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8dc0102ac0 0x7f8dc019c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:43.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.796+0000 7f8dc5fcd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc01083b0 0x7f8dc019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:43.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.796+0000 7f8dc5fcd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dc0108d00 0x7f8dc01a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:43.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.796+0000 7f8dc5fcd640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc0113990 con 0x7f8dc01083b0 2024-09-21T06:36:43.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.796+0000 7f8dc5fcd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc0113810 con 0x7f8dc0102ac0 2024-09-21T06:36:43.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.796+0000 7f8dc5fcd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc0113b10 con 0x7f8dc0108d00 2024-09-21T06:36:43.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.796+0000 7f8dbeffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc01083b0 0x7f8dc019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:43.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.796+0000 7f8dbeffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc01083b0 0x7f8dc019cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41594/0 (socket says 172.21.15.67:41594) 2024-09-21T06:36:43.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.796+0000 7f8dbeffd640 1 -- 172.21.15.67:0/1088162583 learned_addr learned my addr 172.21.15.67:0/1088162583 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:43.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.796+0000 7f8dbf7fe640 1 --2- 172.21.15.67:0/1088162583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8dc0102ac0 0x7f8dc019c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:43.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.796+0000 7f8dbffff640 1 --2- 172.21.15.67:0/1088162583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dc0108d00 0x7f8dc01a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:43.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.796+0000 7f8dbeffd640 1 -- 172.21.15.67:0/1088162583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dc0108d00 msgr2=0x7f8dc01a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:43.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.796+0000 7f8dbeffd640 1 --2- 172.21.15.67:0/1088162583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dc0108d00 0x7f8dc01a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:43.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.796+0000 7f8dbeffd640 1 -- 172.21.15.67:0/1088162583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8dc0102ac0 msgr2=0x7f8dc019c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:43.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.796+0000 7f8dbeffd640 1 --2- 172.21.15.67:0/1088162583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8dc0102ac0 0x7f8dc019c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:43.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.796+0000 7f8dbeffd640 1 -- 172.21.15.67:0/1088162583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc01a2590 con 0x7f8dc01083b0 2024-09-21T06:36:43.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.796+0000 7f8dbffff640 1 --2- 172.21.15.67:0/1088162583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dc0108d00 0x7f8dc01a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:36:43.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.796+0000 7f8dbeffd640 1 --2- 172.21.15.67:0/1088162583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc01083b0 0x7f8dc019cbc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8db400d750 tx=0x7f8db400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:43.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.796+0000 7f8dbcff9640 1 -- 172.21.15.67:0/1088162583 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db4017070 con 0x7f8dc01083b0 2024-09-21T06:36:43.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.796+0000 7f8dc5fcd640 1 -- 172.21.15.67:0/1088162583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dc01a2880 con 0x7f8dc01083b0 2024-09-21T06:36:43.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.796+0000 7f8dbcff9640 1 -- 172.21.15.67:0/1088162583 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8db40054b0 con 0x7f8dc01083b0 2024-09-21T06:36:43.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.797+0000 7f8dbcff9640 1 -- 172.21.15.67:0/1088162583 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db4004bc0 con 0x7f8dc01083b0 2024-09-21T06:36:43.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.797+0000 7f8dc5fcd640 1 -- 172.21.15.67:0/1088162583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dc01a8fe0 con 0x7f8dc01083b0 2024-09-21T06:36:43.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.798+0000 7f8dc5fcd640 1 -- 172.21.15.67:0/1088162583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dc0102ed0 con 0x7f8dc01083b0 2024-09-21T06:36:43.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.799+0000 7f8dbcff9640 1 -- 172.21.15.67:0/1088162583 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f8db4009c30 con 0x7f8dc01083b0 2024-09-21T06:36:43.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.800+0000 7f8dbcff9640 1 --2- 172.21.15.67:0/1088162583 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f8da80b6510 0x7f8da80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:43.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.800+0000 7f8dbf7fe640 1 --2- 172.21.15.67:0/1088162583 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f8da80b6510 0x7f8da80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:43.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.800+0000 7f8dbcff9640 1 -- 172.21.15.67:0/1088162583 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f8db40e74d0 con 0x7f8dc01083b0 2024-09-21T06:36:43.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.801+0000 7f8dbf7fe640 1 --2- 172.21.15.67:0/1088162583 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f8da80b6510 0x7f8da80b8a20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8db00057c0 tx=0x7f8db00058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:43.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:43.803+0000 7f8dbcff9640 1 -- 172.21.15.67:0/1088162583 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8db40ad990 con 0x7f8dc01083b0 2024-09-21T06:36:44.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:44.028+0000 7f8dc5fcd640 1 -- 172.21.15.67:0/1088162583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8dc019d750 con 0x7f8dc01083b0 2024-09-21T06:36:44.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:44.030+0000 7f8dbcff9640 1 -- 172.21.15.67:0/1088162583 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7f8dc019d750 con 0x7f8dc01083b0 2024-09-21T06:36:44.032 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:36:44.032 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:36:44.033 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:36:44.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:44.032+0000 7f8dc5fcd640 1 -- 172.21.15.67:0/1088162583 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f8da80b6510 msgr2=0x7f8da80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:44.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:44.032+0000 7f8dc5fcd640 1 --2- 172.21.15.67:0/1088162583 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f8da80b6510 0x7f8da80b8a20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8db00057c0 tx=0x7f8db00058d0 comp rx=0 tx=0).stop 2024-09-21T06:36:44.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:44.032+0000 7f8dc5fcd640 1 -- 172.21.15.67:0/1088162583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc01083b0 msgr2=0x7f8dc019cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:44.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:44.032+0000 7f8dc5fcd640 1 --2- 172.21.15.67:0/1088162583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc01083b0 0x7f8dc019cbc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8db400d750 tx=0x7f8db400dc10 comp rx=0 tx=0).stop 2024-09-21T06:36:44.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:44.032+0000 7f8dc5fcd640 1 -- 172.21.15.67:0/1088162583 shutdown_connections 2024-09-21T06:36:44.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:44.032+0000 7f8dc5fcd640 1 --2- 172.21.15.67:0/1088162583 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f8da80b6510 0x7f8da80b8a20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:44.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:44.032+0000 7f8dc5fcd640 1 --2- 172.21.15.67:0/1088162583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dc0108d00 0x7f8dc01a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:44.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:44.032+0000 7f8dc5fcd640 1 --2- 172.21.15.67:0/1088162583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc01083b0 0x7f8dc019cbc0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:44.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:44.032+0000 7f8dc5fcd640 1 --2- 172.21.15.67:0/1088162583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8dc0102ac0 0x7f8dc019c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:44.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:44.032+0000 7f8dc5fcd640 1 -- 172.21.15.67:0/1088162583 >> 172.21.15.67:0/1088162583 conn(0x7f8dc0079720 msgr2=0x7f8dc0106300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:44.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:44.032+0000 7f8dc5fcd640 1 -- 172.21.15.67:0/1088162583 shutdown_connections 2024-09-21T06:36:44.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:44.032+0000 7f8dc5fcd640 1 -- 172.21.15.67:0/1088162583 wait complete. 2024-09-21T06:36:44.711 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:44 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1088162583' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:36:44.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:44 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1088162583' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:36:44.742 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:36:44.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:44 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1088162583' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:36:44.999 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:36:45.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.497+0000 7f5c5135b640 1 -- 172.21.15.67:0/3293637477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c4c105300 msgr2=0x7f5c4c10d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:45.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.497+0000 7f5c5135b640 1 --2- 172.21.15.67:0/3293637477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c4c105300 0x7f5c4c10d850 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5c34009900 tx=0x7f5c3402a410 comp rx=0 tx=0).stop 2024-09-21T06:36:45.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.497+0000 7f5c5135b640 1 -- 172.21.15.67:0/3293637477 shutdown_connections 2024-09-21T06:36:45.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.497+0000 7f5c5135b640 1 --2- 172.21.15.67:0/3293637477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c4c10de10 0x7f5c4c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:45.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.497+0000 7f5c5135b640 1 --2- 172.21.15.67:0/3293637477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c4c105300 0x7f5c4c10d850 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:45.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.497+0000 7f5c5135b640 1 --2- 172.21.15.67:0/3293637477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c4c1049b0 0x7f5c4c104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:45.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.497+0000 7f5c5135b640 1 -- 172.21.15.67:0/3293637477 >> 172.21.15.67:0/3293637477 conn(0x7f5c4c079730 msgr2=0x7f5c4c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:45.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.497+0000 7f5c5135b640 1 -- 172.21.15.67:0/3293637477 shutdown_connections 2024-09-21T06:36:45.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.497+0000 7f5c5135b640 1 -- 172.21.15.67:0/3293637477 wait complete. 2024-09-21T06:36:45.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.498+0000 7f5c5135b640 1 Processor -- start 2024-09-21T06:36:45.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.498+0000 7f5c5135b640 1 -- start start 2024-09-21T06:36:45.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.498+0000 7f5c5135b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c4c1049b0 0x7f5c4c19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:45.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.498+0000 7f5c5135b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c4c105300 0x7f5c4c19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:45.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.498+0000 7f5c5135b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c4c10de10 0x7f5c4c19fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:45.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.498+0000 7f5c5135b640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4c113920 con 0x7f5c4c10de10 2024-09-21T06:36:45.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.498+0000 7f5c5135b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4c1137a0 con 0x7f5c4c105300 2024-09-21T06:36:45.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.498+0000 7f5c5135b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4c113aa0 con 0x7f5c4c1049b0 2024-09-21T06:36:45.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.498+0000 7f5c4b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c4c10de10 0x7f5c4c19fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:45.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.498+0000 7f5c4b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c4c10de10 0x7f5c4c19fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41618/0 (socket says 172.21.15.67:41618) 2024-09-21T06:36:45.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.498+0000 7f5c4b7fe640 1 -- 172.21.15.67:0/1482972322 learned_addr learned my addr 172.21.15.67:0/1482972322 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:45.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.498+0000 7f5c4affd640 1 --2- 172.21.15.67:0/1482972322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c4c1049b0 0x7f5c4c19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:45.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.498+0000 7f5c4a7fc640 1 --2- 172.21.15.67:0/1482972322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c4c105300 0x7f5c4c19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:45.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.499+0000 7f5c4b7fe640 1 -- 172.21.15.67:0/1482972322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c4c1049b0 msgr2=0x7f5c4c19a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:45.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.499+0000 7f5c4b7fe640 1 --2- 172.21.15.67:0/1482972322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c4c1049b0 0x7f5c4c19a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:45.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.499+0000 7f5c4b7fe640 1 -- 172.21.15.67:0/1482972322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c4c105300 msgr2=0x7f5c4c19a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:45.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.499+0000 7f5c4b7fe640 1 --2- 172.21.15.67:0/1482972322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c4c105300 0x7f5c4c19a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:45.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.499+0000 7f5c4b7fe640 1 -- 172.21.15.67:0/1482972322 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c4c1a0360 con 0x7f5c4c10de10 2024-09-21T06:36:45.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.499+0000 7f5c4b7fe640 1 --2- 172.21.15.67:0/1482972322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c4c10de10 0x7f5c4c19fc30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00eaa0 tx=0x7f5c3c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:45.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.499+0000 7f5c43fff640 1 -- 172.21.15.67:0/1482972322 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c00cdb0 con 0x7f5c4c10de10 2024-09-21T06:36:45.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.499+0000 7f5c5135b640 1 -- 172.21.15.67:0/1482972322 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c4c1a0650 con 0x7f5c4c10de10 2024-09-21T06:36:45.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.499+0000 7f5c43fff640 1 -- 172.21.15.67:0/1482972322 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c3c005020 con 0x7f5c4c10de10 2024-09-21T06:36:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.499+0000 7f5c43fff640 1 -- 172.21.15.67:0/1482972322 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c0116d0 con 0x7f5c4c10de10 2024-09-21T06:36:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.499+0000 7f5c5135b640 1 -- 172.21.15.67:0/1482972322 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c4c1a6db0 con 0x7f5c4c10de10 2024-09-21T06:36:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.500+0000 7f5c5135b640 1 -- 172.21.15.67:0/1482972322 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c4c104dc0 con 0x7f5c4c10de10 2024-09-21T06:36:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.502+0000 7f5c43fff640 1 -- 172.21.15.67:0/1482972322 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f5c3c0051d0 con 0x7f5c4c10de10 2024-09-21T06:36:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.503+0000 7f5c43fff640 1 --2- 172.21.15.67:0/1482972322 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5c380b6510 0x7f5c380b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.503+0000 7f5c4affd640 1 --2- 172.21.15.67:0/1482972322 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5c380b6510 0x7f5c380b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.503+0000 7f5c43fff640 1 -- 172.21.15.67:0/1482972322 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f5c3c0e7be0 con 0x7f5c4c10de10 2024-09-21T06:36:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.504+0000 7f5c4affd640 1 --2- 172.21.15.67:0/1482972322 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5c380b6510 0x7f5c380b8a20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c06bc60 tx=0x7f5c4400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.506+0000 7f5c43fff640 1 -- 172.21.15.67:0/1482972322 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5c3c016030 con 0x7f5c4c10de10 2024-09-21T06:36:45.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.696+0000 7f5c5135b640 1 -- 172.21.15.67:0/1482972322 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5c4c19b520 con 0x7f5c4c10de10 2024-09-21T06:36:45.698 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:45 smithi067 ceph-mon[31232]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-21T06:36:45.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.697+0000 7f5c43fff640 1 -- 172.21.15.67:0/1482972322 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24954 (secure 0 0 0) 0x7f5c4c19b520 con 0x7f5c4c10de10 2024-09-21T06:36:45.700 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:36:45.700 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":85,"fsid":"2a791768-77e2-11ef-baf5-efdc52797490","created":"2024-09-21T06:26:15.222283+0000","modified":"2024-09-21T06:36:05.617384+0000","last_up_change":"2024-09-21T06:33:40.886310+0000","last_in_change":"2024-09-21T06:33:17.339486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:29:50.212825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:34:29.082881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:34:31.301094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:34:37.681356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"18778364-82ed-41c0-92da-ce44cdad7358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6803","nonce":2310759946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6805","nonce":2310759946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6809","nonce":2310759946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6807","nonce":2310759946}]},"public_addr":"172.21.15.67:6803/2310759946","cluster_addr":"172.21.15.67:6805/2310759946","heartbeat_back_addr":"172.21.15.67:6809/2310759946","heartbeat_front_addr":"172.21.15.67:6807/2310759946","state":["exists","up"]},{"osd":1,"uuid":"84e8d0c2-1c45-4673-ad49-f61df667d7f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6801","nonce":161807364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6803","nonce":161807364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6807","nonce":161807364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6805","nonce":161807364}]},"public_addr":"172.21.15.100:6801/161807364","cluster_addr":"172.21.15.100:6803/161807364","heartbeat_back_addr":"172.21.15.100:6807/161807364","heartbeat_front_addr":"172.21.15.100:6805/161807364","state":["exists","up"]},{"osd":2,"uuid":"e9a4d919-a779-4a1b-9b5b-d03a65f28ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6801","nonce":2754787169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6803","nonce":2754787169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6807","nonce":2754787169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6805","nonce":2754787169}]},"public_addr":"172.21.15.203:6801/2754787169","cluster_addr":"172.21.15.203:6803/2754787169","heartbeat_back_addr":"172.21.15.203:6807/2754787169","heartbeat_front_addr":"172.21.15.203:6805/2754787169","state":["exists","up"]},{"osd":3,"uuid":"3f7d9abd-587c-47ad-bf43-11f7dcdf71c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6811","nonce":4007517951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6813","nonce":4007517951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6817","nonce":4007517951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6815","nonce":4007517951}]},"public_addr":"172.21.15.67:6811/4007517951","cluster_addr":"172.21.15.67:6813/4007517951","heartbeat_back_addr":"172.21.15.67:6817/4007517951","heartbeat_front_addr":"172.21.15.67:6815/4007517951","state":["exists","up"]},{"osd":4,"uuid":"94f3a075-6d8a-4907-8418-a8513b35779a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6809","nonce":3773646501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6811","nonce":3773646501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6815","nonce":3773646501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6813","nonce":3773646501}]},"public_addr":"172.21.15.100:6809/3773646501","cluster_addr":"172.21.15.100:6811/3773646501","heartbeat_back_addr":"172.21.15.100:6815/3773646501","heartbeat_front_addr":"172.21.15.100:6813/3773646501","state":["exists","up"]},{"osd":5,"uuid":"5d842edf-06ad-49ae-ac71-863e5657270c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6809","nonce":2096171166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6811","nonce":2096171166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6815","nonce":2096171166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6813","nonce":2096171166}]},"public_addr":"172.21.15.203:6809/2096171166","cluster_addr":"172.21.15.203:6811/2096171166","heartbeat_back_addr":"172.21.15.203:6815/2096171166","heartbeat_front_addr":"172.21.15.203:6813/2096171166","state":["exists","up"]},{"osd":6,"uuid":"85b9beed-c93d-48ba-bc7a-c4cd816fbdff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6819","nonce":3229936280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6821","nonce":3229936280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6825","nonce":3229936280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6823","nonce":3229936280}]},"public_addr":"172.21.15.67:6819/3229936280","cluster_addr":"172.21.15.67:6821/3229936280","heartbeat_back_addr":"172.21.15.67:6825/3229936280","heartbeat_front_addr":"172.21.15.67:6823/3229936280","state":["exists","up"]},{"osd":7,"uuid":"86de30c2-b8ee-4c9d-9c17-8a56517934ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6817","nonce":1622917909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6819","nonce":1622917909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6823","nonce":1622917909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6821","nonce":1622917909}]},"public_addr":"172.21.15.100:6817/1622917909","cluster_addr":"172.21.15.100:6819/1622917909","heartbeat_back_addr":"172.21.15.100:6823/1622917909","heartbeat_front_addr":"172.21.15.100:6821/1622917909","state":["exists","up"]},{"osd":8,"uuid":"0e96cdad-a135-4115-9e2d-9a28306700e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6817","nonce":3452445943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6819","nonce":3452445943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6823","nonce":3452445943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6821","nonce":3452445943}]},"public_addr":"172.21.15.203:6817/3452445943","cluster_addr":"172.21.15.203:6819/3452445943","heartbeat_back_addr":"172.21.15.203:6823/3452445943","heartbeat_front_addr":"172.21.15.203:6821/3452445943","state":["exists","up"]},{"osd":9,"uuid":"95704ed6-6286-4094-9c6d-23919c7005ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6827","nonce":2998307533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6829","nonce":2998307533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6833","nonce":2998307533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6831","nonce":2998307533}]},"public_addr":"172.21.15.67:6827/2998307533","cluster_addr":"172.21.15.67:6829/2998307533","heartbeat_back_addr":"172.21.15.67:6833/2998307533","heartbeat_front_addr":"172.21.15.67:6831/2998307533","state":["exists","up"]},{"osd":10,"uuid":"bd2a7d33-b935-46c8-95c3-d76f28242ffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6825","nonce":2819057142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6827","nonce":2819057142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6831","nonce":2819057142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6829","nonce":2819057142}]},"public_addr":"172.21.15.100:6825/2819057142","cluster_addr":"172.21.15.100:6827/2819057142","heartbeat_back_addr":"172.21.15.100:6831/2819057142","heartbeat_front_addr":"172.21.15.100:6829/2819057142","state":["exists","up"]},{"osd":11,"uuid":"ea591507-10e2-4c08-8d29-d38ba9be96ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6825","nonce":566502600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6827","nonce":566502600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6831","nonce":566502600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6829","nonce":566502600}]},"public_addr":"172.21.15.203:6825/566502600","cluster_addr":"172.21.15.203:6827/566502600","heartbeat_back_addr":"172.21.15.203:6831/566502600","heartbeat_front_addr":"172.21.15.203:6829/566502600","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:28:58.945733+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:29:23.597118+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:29:48.303563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:30:14.139139+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:30:39.021508+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:04.981241+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:30.853841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:56.452967+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:32:22.239036+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:32:47.998803+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:33:13.850759+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:33:39.282150+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6800/1181476774":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/3100321138":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/2309364298":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/3458717076":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/4110206171":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6801/1181476774":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/2910038982":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6801/2622729103":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6800/2622729103":"2024-09-22T06:26:37.041377+0000","172.21.15.67:0/3346784528":"2024-09-22T06:26:37.041377+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:36:45.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.699+0000 7f5c5135b640 1 -- 172.21.15.67:0/1482972322 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5c380b6510 msgr2=0x7f5c380b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:45.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.699+0000 7f5c5135b640 1 --2- 172.21.15.67:0/1482972322 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5c380b6510 0x7f5c380b8a20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c06bc60 tx=0x7f5c4400b390 comp rx=0 tx=0).stop 2024-09-21T06:36:45.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.699+0000 7f5c5135b640 1 -- 172.21.15.67:0/1482972322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c4c10de10 msgr2=0x7f5c4c19fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:45.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.699+0000 7f5c5135b640 1 --2- 172.21.15.67:0/1482972322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c4c10de10 0x7f5c4c19fc30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00eaa0 tx=0x7f5c3c00ef60 comp rx=0 tx=0).stop 2024-09-21T06:36:45.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.699+0000 7f5c5135b640 1 -- 172.21.15.67:0/1482972322 shutdown_connections 2024-09-21T06:36:45.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.699+0000 7f5c5135b640 1 --2- 172.21.15.67:0/1482972322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c4c105300 0x7f5c4c19a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:45.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.699+0000 7f5c5135b640 1 --2- 172.21.15.67:0/1482972322 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5c380b6510 0x7f5c380b8a20 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:45.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.699+0000 7f5c5135b640 1 --2- 172.21.15.67:0/1482972322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c4c10de10 0x7f5c4c19fc30 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:45.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.699+0000 7f5c5135b640 1 --2- 172.21.15.67:0/1482972322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c4c1049b0 0x7f5c4c19a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:45.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.699+0000 7f5c5135b640 1 -- 172.21.15.67:0/1482972322 >> 172.21.15.67:0/1482972322 conn(0x7f5c4c079730 msgr2=0x7f5c4c1008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:45.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.700+0000 7f5c5135b640 1 -- 172.21.15.67:0/1482972322 shutdown_connections 2024-09-21T06:36:45.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:45.700+0000 7f5c5135b640 1 -- 172.21.15.67:0/1482972322 wait complete. 2024-09-21T06:36:45.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:45 smithi203 ceph-mon[33735]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-21T06:36:45.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:45 smithi100 ceph-mon[33401]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-21T06:36:46.429 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T06:34:33.306959+0000', 'modified': '2024-09-21T06:35:09.805211+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24671}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24641': {'gid': 24641, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.100:6837/259395872', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6836', 'nonce': 259395872}, {'type': 'v1', 'addr': '172.21.15.100:6837', 'nonce': 259395872}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24671': {'gid': 24671, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.100:6839/778060936', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6838', 'nonce': 778060936}, {'type': 'v1', 'addr': '172.21.15.100:6839', 'nonce': 778060936}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24671, 'qdb_cluster': [24671]} 2024-09-21T06:36:46.429 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T06:36:46.429 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T06:36:46.691 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:36:46.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:46 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1482972322' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:36:46.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:46 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1482972322' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:36:46.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:46 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1482972322' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:36:47.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.231+0000 7f19f6283640 1 -- 172.21.15.67:0/62006004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19f01083c0 msgr2=0x7f19f01087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:47.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.231+0000 7f19f6283640 1 --2- 172.21.15.67:0/62006004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19f01083c0 0x7f19f01087d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009900 tx=0x7f19d802a410 comp rx=0 tx=0).stop 2024-09-21T06:36:47.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.231+0000 7f19f6283640 1 -- 172.21.15.67:0/62006004 shutdown_connections 2024-09-21T06:36:47.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.231+0000 7f19f6283640 1 --2- 172.21.15.67:0/62006004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19f0108d10 0x7f19f010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.231+0000 7f19f6283640 1 --2- 172.21.15.67:0/62006004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19f01083c0 0x7f19f01087d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.231+0000 7f19f6283640 1 --2- 172.21.15.67:0/62006004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19f0102ad0 0x7f19f0102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.231+0000 7f19f6283640 1 -- 172.21.15.67:0/62006004 >> 172.21.15.67:0/62006004 conn(0x7f19f00796d0 msgr2=0x7f19f0079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:47.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.231+0000 7f19f6283640 1 -- 172.21.15.67:0/62006004 shutdown_connections 2024-09-21T06:36:47.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.231+0000 7f19f6283640 1 -- 172.21.15.67:0/62006004 wait complete. 2024-09-21T06:36:47.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.232+0000 7f19f6283640 1 Processor -- start 2024-09-21T06:36:47.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.232+0000 7f19f6283640 1 -- start start 2024-09-21T06:36:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.232+0000 7f19f6283640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19f0102ad0 0x7f19f019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.232+0000 7f19f6283640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19f01083c0 0x7f19f019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.232+0000 7f19f6283640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19f0108d10 0x7f19f019fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.232+0000 7f19f6283640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f19f01139a0 con 0x7f19f0108d10 2024-09-21T06:36:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.232+0000 7f19f6283640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f19f0113820 con 0x7f19f01083c0 2024-09-21T06:36:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.233+0000 7f19f6283640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f19f0113b20 con 0x7f19f0102ad0 2024-09-21T06:36:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.233+0000 7f19f4820640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19f0108d10 0x7f19f019fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.233+0000 7f19f4820640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19f0108d10 0x7f19f019fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41646/0 (socket says 172.21.15.67:41646) 2024-09-21T06:36:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.233+0000 7f19f4820640 1 -- 172.21.15.67:0/2403540753 learned_addr learned my addr 172.21.15.67:0/2403540753 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.233+0000 7f19effff640 1 --2- 172.21.15.67:0/2403540753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19f0102ad0 0x7f19f019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.233+0000 7f19e7fff640 1 --2- 172.21.15.67:0/2403540753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19f01083c0 0x7f19f019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.233+0000 7f19f4820640 1 -- 172.21.15.67:0/2403540753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19f0102ad0 msgr2=0x7f19f019a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.233+0000 7f19f4820640 1 --2- 172.21.15.67:0/2403540753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19f0102ad0 0x7f19f019a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.233+0000 7f19f4820640 1 -- 172.21.15.67:0/2403540753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19f01083c0 msgr2=0x7f19f019a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.233+0000 7f19f4820640 1 --2- 172.21.15.67:0/2403540753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19f01083c0 0x7f19f019a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.233+0000 7f19f4820640 1 -- 172.21.15.67:0/2403540753 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f01a02e0 con 0x7f19f0108d10 2024-09-21T06:36:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.233+0000 7f19f4820640 1 --2- 172.21.15.67:0/2403540753 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19f0108d10 0x7f19f019fc70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f19e800eaa0 tx=0x7f19e800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.233+0000 7f19edffb640 1 -- 172.21.15.67:0/2403540753 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8004d20 con 0x7f19f0108d10 2024-09-21T06:36:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.233+0000 7f19edffb640 1 -- 172.21.15.67:0/2403540753 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19e8005020 con 0x7f19f0108d10 2024-09-21T06:36:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.233+0000 7f19f6283640 1 -- 172.21.15.67:0/2403540753 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f01a05d0 con 0x7f19f0108d10 2024-09-21T06:36:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.233+0000 7f19edffb640 1 -- 172.21.15.67:0/2403540753 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e80116d0 con 0x7f19f0108d10 2024-09-21T06:36:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.233+0000 7f19f6283640 1 -- 172.21.15.67:0/2403540753 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f01a6df0 con 0x7f19f0108d10 2024-09-21T06:36:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.236+0000 7f19fc7f7740 1 Processor -- start 2024-09-21T06:36:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.237+0000 7f19edffb640 1 -- 172.21.15.67:0/2403540753 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f19e80051d0 con 0x7f19f0108d10 2024-09-21T06:36:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.237+0000 7f19fc7f7740 1 -- start start 2024-09-21T06:36:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.237+0000 7f19fc7f7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564959b79f60 0x564959f1ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.237+0000 7f19fc7f7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564959b510a0 0x564959f20440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.237+0000 7f19fc7f7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564959b59280 0x564959f20980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:47.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.237+0000 7f19fc7f7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x564959e97c10 con 0x564959b59280 2024-09-21T06:36:47.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.237+0000 7f19fc7f7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x564959e87040 con 0x564959b79f60 2024-09-21T06:36:47.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.237+0000 7f19fc7f7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564959e8ae50 con 0x564959b510a0 2024-09-21T06:36:47.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.238+0000 7f19e7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564959b59280 0x564959f20980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:47.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.238+0000 7f19e7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564959b59280 0x564959f20980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41654/0 (socket says 172.21.15.67:41654) 2024-09-21T06:36:47.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.238+0000 7f19e7fff640 1 -- 172.21.15.67:0/1043384644 learned_addr learned my addr 172.21.15.67:0/1043384644 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:47.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.238+0000 7f19e7fff640 1 --2- 172.21.15.67:0/1043384644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564959b79f60 0x564959f1ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:47.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.238+0000 7f19effff640 1 --2- 172.21.15.67:0/1043384644 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564959b510a0 0x564959f20440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:47.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.238+0000 7f19edffb640 1 --2- 172.21.15.67:0/2403540753 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f19dc0b65c0 0x7f19dc0b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:47.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.238+0000 7f19e7fff640 1 -- 172.21.15.67:0/1043384644 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564959b510a0 msgr2=0x564959f20440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:36:47.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.238+0000 7f19e7fff640 1 --2- 172.21.15.67:0/1043384644 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564959b510a0 0x564959f20440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.238+0000 7f19e7fff640 1 -- 172.21.15.67:0/1043384644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564959b79f60 msgr2=0x564959f1ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:36:47.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.238+0000 7f19e7fff640 1 --2- 172.21.15.67:0/1043384644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564959b79f60 0x564959f1ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.238+0000 7f19e7fff640 1 -- 172.21.15.67:0/1043384644 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564959be5540 con 0x564959b59280 2024-09-21T06:36:47.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.238+0000 7f19f4820640 1 --2- 172.21.15.67:0/2403540753 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f19dc0b65c0 0x7f19dc0b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:47.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.238+0000 7f19edffb640 1 -- 172.21.15.67:0/2403540753 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f19e80e7720 con 0x7f19f0108d10 2024-09-21T06:36:47.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.238+0000 7f19e7fff640 1 --2- 172.21.15.67:0/1043384644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564959b59280 0x564959f20980 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f19d8005ab0 tx=0x7f19d8005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c62f5791839e975 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:47.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.239+0000 7f19e67fc640 1 -- 172.21.15.67:0/1043384644 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d8035040 con 0x564959b59280 2024-09-21T06:36:47.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.239+0000 7f19e67fc640 1 -- 172.21.15.67:0/1043384644 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19d80044c0 con 0x564959b59280 2024-09-21T06:36:47.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.239+0000 7f19e67fc640 1 -- 172.21.15.67:0/1043384644 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d80048c0 con 0x564959b59280 2024-09-21T06:36:47.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.239+0000 7f19fc7f7740 1 -- 172.21.15.67:0/1043384644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564959b59280 msgr2=0x564959f20980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:47.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.239+0000 7f19fc7f7740 1 --2- 172.21.15.67:0/1043384644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564959b59280 0x564959f20980 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f19d8005ab0 tx=0x7f19d8005a40 comp rx=0 tx=0).stop 2024-09-21T06:36:47.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.239+0000 7f19f4820640 1 --2- 172.21.15.67:0/2403540753 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f19dc0b65c0 0x7f19dc0b8ad0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f19e80aa750 tx=0x7f19e807a810 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:47.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.239+0000 7f19fc7f7740 1 -- 172.21.15.67:0/1043384644 shutdown_connections 2024-09-21T06:36:47.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.239+0000 7f19fc7f7740 1 --2- 172.21.15.67:0/1043384644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564959b59280 0x564959f20980 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.239+0000 7f19fc7f7740 1 --2- 172.21.15.67:0/1043384644 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564959b510a0 0x564959f20440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.239+0000 7f19fc7f7740 1 --2- 172.21.15.67:0/1043384644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564959b79f60 0x564959f1ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.239+0000 7f19fc7f7740 1 -- 172.21.15.67:0/1043384644 >> 172.21.15.67:0/1043384644 conn(0x564959b6e350 msgr2=0x564959f1ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.240+0000 7f19fc7f7740 1 -- 172.21.15.67:0/1043384644 shutdown_connections 2024-09-21T06:36:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.240+0000 7f19fc7f7740 1 -- 172.21.15.67:0/1043384644 wait complete. 2024-09-21T06:36:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.240+0000 7f19fc7f7740 1 -- start start 2024-09-21T06:36:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.240+0000 7f19fc7f7740 1 Processor -- start 2024-09-21T06:36:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.240+0000 7f19fc7f7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564959b510a0 0x564959f2ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.240+0000 7f19fc7f7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564959b59280 0x564959f2bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.240+0000 7f19fc7f7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564959b79f60 0x564959f37970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.240+0000 7f19fc7f7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x564959ce85f0 con 0x564959b510a0 2024-09-21T06:36:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.240+0000 7f19fc7f7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x564959e22010 con 0x564959b59280 2024-09-21T06:36:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.240+0000 7f19fc7f7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564959f2e680 con 0x564959b79f60 2024-09-21T06:36:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.241+0000 7f19fc7f7740 10 client.0 _ll_register_callbacks cb 0x564959e81340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:36:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.241+0000 7f19e7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564959b510a0 0x564959f2ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.241+0000 7f19e7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564959b510a0 0x564959f2ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41664/0 (socket says 172.21.15.67:41664) 2024-09-21T06:36:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.241+0000 7f19e7fff640 1 -- 172.21.15.67:0/649661363 learned_addr learned my addr 172.21.15.67:0/649661363 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.241+0000 7f19effff640 1 --2- 172.21.15.67:0/649661363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564959b59280 0x564959f2bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:47.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.241+0000 7f19e7fff640 1 -- 172.21.15.67:0/649661363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564959b79f60 msgr2=0x564959f37970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:36:47.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.241+0000 7f19e7fff640 1 --2- 172.21.15.67:0/649661363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564959b79f60 0x564959f37970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.241+0000 7f19e7fff640 1 -- 172.21.15.67:0/649661363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564959b59280 msgr2=0x564959f2bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:36:47.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.241+0000 7f19e7fff640 1 --2- 172.21.15.67:0/649661363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564959b59280 0x564959f2bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.241+0000 7f19e7fff640 1 -- 172.21.15.67:0/649661363 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564959e4fa00 con 0x564959b510a0 2024-09-21T06:36:47.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.241+0000 7f19effff640 1 --2- 172.21.15.67:0/649661363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564959b59280 0x564959f2bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:36:47.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.241+0000 7f19e7fff640 1 --2- 172.21.15.67:0/649661363 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564959b510a0 0x564959f2ba50 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f19d802a450 tx=0x7f19d8005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7fafcea7e1265e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:47.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.241+0000 7f19cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:36:47.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.241+0000 7f19cf7fe640 1 -- 172.21.15.67:0/649661363 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d8035070 con 0x564959b510a0 2024-09-21T06:36:47.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.241+0000 7f19cf7fe640 1 -- 172.21.15.67:0/649661363 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f19d80046c0 con 0x564959b510a0 2024-09-21T06:36:47.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.241+0000 7f19fc7f7740 1 -- 172.21.15.67:0/649661363 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564959e99b60 con 0x564959b510a0 2024-09-21T06:36:47.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.242+0000 7f19cf7fe640 1 -- 172.21.15.67:0/649661363 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f19d8057050 con 0x564959b510a0 2024-09-21T06:36:47.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.242+0000 7f19fc7f7740 10 client.15360 fetch_fsmap learned FSMap version 21 2024-09-21T06:36:47.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.242+0000 7f19fc7f7740 1 -- 172.21.15.67:0/649661363 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564959f3a640 con 0x564959b510a0 2024-09-21T06:36:47.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.242+0000 7f19cf7fe640 1 -- 172.21.15.67:0/649661363 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x564959e99b60 con 0x564959b510a0 2024-09-21T06:36:47.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.242+0000 7f19cf7fe640 1 -- 172.21.15.67:0/649661363 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f19d803a040 con 0x564959b510a0 2024-09-21T06:36:47.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.242+0000 7f19fc7f7740 10 client.15360 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:36:47.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.242+0000 7f19fc7f7740 10 client.15360 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:36:47.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.242+0000 7f19fc7f7740 20 client.15360 populate_metadata read hostname 'smithi067' 2024-09-21T06:36:47.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.242+0000 7f19fc7f7740 1 --2- 172.21.15.67:0/649661363 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x564959b388e0 0x564959f3dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:47.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.242+0000 7f19fc7f7740 4 client.15360 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:36:47.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.242+0000 7f19fc7f7740 1 -- 172.21.15.67:0/649661363 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564959e95610 con 0x564959b388e0 2024-09-21T06:36:47.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.243+0000 7f19effff640 1 --2- 172.21.15.67:0/649661363 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x564959b388e0 0x564959f3dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:47.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.243+0000 7f19cf7fe640 10 client.15360 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:36:47.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.243+0000 7f19effff640 1 --2- 172.21.15.67:0/649661363 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x564959b388e0 0x564959f3dc20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e63bfc27bf62d837 server_cookie=77498bbb9dc5194d in_seq=0 out_seq=0 2024-09-21T06:36:47.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.244+0000 7f19cf7fe640 1 -- 172.21.15.67:0/649661363 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564959e95610 con 0x564959b388e0 2024-09-21T06:36:47.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.244+0000 7f19cf7fe640 10 client.15360 handle_command_reply: tid=0 2024-09-21T06:36:47.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.244+0000 7f19fc7f7740 1 client.15360 shutdown 2024-09-21T06:36:47.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.244+0000 7f19fc7f7740 1 -- 172.21.15.67:0/649661363 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564959b510a0 msgr2=0x564959f2ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:47.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.244+0000 7f19fc7f7740 1 --2- 172.21.15.67:0/649661363 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564959b510a0 0x564959f2ba50 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f19d802a450 tx=0x7f19d8005970 comp rx=0 tx=0).stop 2024-09-21T06:36:47.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.244+0000 7f19fc7f7740 1 -- 172.21.15.67:0/649661363 shutdown_connections 2024-09-21T06:36:47.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.244+0000 7f19fc7f7740 1 --2- 172.21.15.67:0/649661363 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x564959b388e0 0x564959f3dc20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.244+0000 7f19fc7f7740 1 --2- 172.21.15.67:0/649661363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564959b59280 0x564959f2bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.244+0000 7f19cf7fe640 0 client.15360 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:36:47.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.244+0000 7f19fc7f7740 1 --2- 172.21.15.67:0/649661363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564959b79f60 0x564959f37970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.244+0000 7f19fc7f7740 1 --2- 172.21.15.67:0/649661363 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564959b510a0 0x564959f2ba50 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.244+0000 7f19fc7f7740 1 -- 172.21.15.67:0/649661363 >> 172.21.15.67:0/649661363 conn(0x564959b6e350 msgr2=0x564959f230e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:47.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.244+0000 7f19fc7f7740 1 -- 172.21.15.67:0/649661363 shutdown_connections 2024-09-21T06:36:47.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.244+0000 7f19fc7f7740 1 -- 172.21.15.67:0/649661363 wait complete. 2024-09-21T06:36:47.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.244+0000 7f19fc7f7740 20 client.15360 trim_cache size 0 max 16384 2024-09-21T06:36:47.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.263+0000 7f19fc7f7740 1 Processor -- start 2024-09-21T06:36:47.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.263+0000 7f19fc7f7740 1 -- start start 2024-09-21T06:36:47.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.263+0000 7f19fc7f7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564959b388e0 0x564959f21a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:47.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.263+0000 7f19fc7f7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564959b510a0 0x564959f26020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:47.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.263+0000 7f19fc7f7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564959b59280 0x564959f378e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:47.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.263+0000 7f19fc7f7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x564959f3bf60 con 0x564959b388e0 2024-09-21T06:36:47.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.263+0000 7f19fc7f7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x564959f3c630 con 0x564959b59280 2024-09-21T06:36:47.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.263+0000 7f19fc7f7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564959f3cd00 con 0x564959b510a0 2024-09-21T06:36:47.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.263+0000 7f19e7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564959b388e0 0x564959f21a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:47.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.263+0000 7f19e7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564959b388e0 0x564959f21a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41666/0 (socket says 172.21.15.67:41666) 2024-09-21T06:36:47.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.263+0000 7f19e7fff640 1 -- 172.21.15.67:0/1517909173 learned_addr learned my addr 172.21.15.67:0/1517909173 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:47.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.263+0000 7f19effff640 1 --2- 172.21.15.67:0/1517909173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564959b510a0 0x564959f26020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:47.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.264+0000 7f19e7fff640 1 -- 172.21.15.67:0/1517909173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564959b510a0 msgr2=0x564959f26020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:36:47.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.264+0000 7f19e7fff640 1 --2- 172.21.15.67:0/1517909173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564959b510a0 0x564959f26020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.264+0000 7f19e7fff640 1 -- 172.21.15.67:0/1517909173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564959b59280 msgr2=0x564959f378e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:36:47.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.264+0000 7f19e7fff640 1 --2- 172.21.15.67:0/1517909173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564959b59280 0x564959f378e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.264+0000 7f19e7fff640 1 -- 172.21.15.67:0/1517909173 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564959f2cb10 con 0x564959b388e0 2024-09-21T06:36:47.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.264+0000 7f19e7fff640 1 --2- 172.21.15.67:0/1517909173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564959b388e0 0x564959f21a10 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f19d802a930 tx=0x7f19d8032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbfaac481ca2bb10 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:47.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.264+0000 7f19e5ffb640 1 -- 172.21.15.67:0/1517909173 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d8035070 con 0x564959b388e0 2024-09-21T06:36:47.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.264+0000 7f19e5ffb640 1 -- 172.21.15.67:0/1517909173 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19d804e070 con 0x564959b388e0 2024-09-21T06:36:47.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.264+0000 7f19e5ffb640 1 -- 172.21.15.67:0/1517909173 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d80046c0 con 0x564959b388e0 2024-09-21T06:36:47.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.265+0000 7f19fc7f7740 1 -- 172.21.15.67:0/1517909173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564959b388e0 msgr2=0x564959f21a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:47.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.265+0000 7f19fc7f7740 1 --2- 172.21.15.67:0/1517909173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564959b388e0 0x564959f21a10 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f19d802a930 tx=0x7f19d8032fc0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.265+0000 7f19fc7f7740 1 -- 172.21.15.67:0/1517909173 shutdown_connections 2024-09-21T06:36:47.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.265+0000 7f19fc7f7740 1 --2- 172.21.15.67:0/1517909173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564959b59280 0x564959f378e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.265+0000 7f19fc7f7740 1 --2- 172.21.15.67:0/1517909173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564959b510a0 0x564959f26020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.265+0000 7f19fc7f7740 1 --2- 172.21.15.67:0/1517909173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564959b388e0 0x564959f21a10 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.265+0000 7f19fc7f7740 1 -- 172.21.15.67:0/1517909173 >> 172.21.15.67:0/1517909173 conn(0x564959b6e350 msgr2=0x564959f26a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:47.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.265+0000 7f19fc7f7740 1 -- 172.21.15.67:0/1517909173 shutdown_connections 2024-09-21T06:36:47.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.265+0000 7f19fc7f7740 1 -- 172.21.15.67:0/1517909173 wait complete. 2024-09-21T06:36:47.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.265+0000 7f19fc7f7740 1 -- start start 2024-09-21T06:36:47.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.265+0000 7f19fc7f7740 1 Processor -- start 2024-09-21T06:36:47.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.266+0000 7f19fc7f7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564959b388e0 0x564959f462a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:47.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.266+0000 7f19fc7f7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564959b510a0 0x564959f46880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:47.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.266+0000 7f19fc7f7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564959b59280 0x564959f3f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:47.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.266+0000 7f19fc7f7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x564959f3daa0 con 0x564959b388e0 2024-09-21T06:36:47.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.266+0000 7f19fc7f7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x564959f3d3d0 con 0x564959b510a0 2024-09-21T06:36:47.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.266+0000 7f19fc7f7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564959f3e170 con 0x564959b59280 2024-09-21T06:36:47.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.266+0000 7f19fc7f7740 10 client.0 _ll_register_callbacks cb 0x7f19d8032dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:36:47.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.266+0000 7f19e7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564959b388e0 0x564959f462a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:47.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.266+0000 7f19e7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564959b388e0 0x564959f462a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41668/0 (socket says 172.21.15.67:41668) 2024-09-21T06:36:47.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.266+0000 7f19e7fff640 1 -- 172.21.15.67:0/3136712259 learned_addr learned my addr 172.21.15.67:0/3136712259 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:47.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.266+0000 7f19effff640 1 --2- 172.21.15.67:0/3136712259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564959b510a0 0x564959f46880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:47.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.266+0000 7f19e7fff640 1 --2- 172.21.15.67:0/3136712259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564959b59280 0x564959f3f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:47.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.266+0000 7f19e7fff640 1 -- 172.21.15.67:0/3136712259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564959b59280 msgr2=0x564959f3f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:36:47.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.266+0000 7f19e7fff640 1 --2- 172.21.15.67:0/3136712259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564959b59280 0x564959f3f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.266+0000 7f19e7fff640 1 -- 172.21.15.67:0/3136712259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564959b510a0 msgr2=0x564959f46880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:36:47.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.266+0000 7f19e7fff640 1 --2- 172.21.15.67:0/3136712259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564959b510a0 0x564959f46880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.266+0000 7f19e7fff640 1 -- 172.21.15.67:0/3136712259 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564959f40070 con 0x564959b388e0 2024-09-21T06:36:47.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.266+0000 7f19e7fff640 1 --2- 172.21.15.67:0/3136712259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564959b388e0 0x564959f462a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f19d80098e0 tx=0x7f19d8049a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b915d0d5aa53985 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:47.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.266+0000 7f19cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:36:47.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.267+0000 7f19cf7fe640 1 -- 172.21.15.67:0/3136712259 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d8035070 con 0x564959b388e0 2024-09-21T06:36:47.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.267+0000 7f19cf7fe640 1 -- 172.21.15.67:0/3136712259 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f19d8031760 con 0x564959b388e0 2024-09-21T06:36:47.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.267+0000 7f19fc7f7740 1 -- 172.21.15.67:0/3136712259 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19d804e070 con 0x564959b388e0 2024-09-21T06:36:47.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.267+0000 7f19cf7fe640 1 -- 172.21.15.67:0/3136712259 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f19d8005330 con 0x564959b388e0 2024-09-21T06:36:47.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.267+0000 7f19fc7f7740 10 client.15366 fetch_fsmap learned FSMap version 21 2024-09-21T06:36:47.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.267+0000 7f19fc7f7740 1 -- 172.21.15.67:0/3136712259 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564959f45a20 con 0x564959b388e0 2024-09-21T06:36:47.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.267+0000 7f19cf7fe640 1 -- 172.21.15.67:0/3136712259 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19d804e070 con 0x564959b388e0 2024-09-21T06:36:47.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.267+0000 7f19cf7fe640 1 -- 172.21.15.67:0/3136712259 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f19d8064070 con 0x564959b388e0 2024-09-21T06:36:47.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.267+0000 7f19fc7f7740 10 client.15366 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:36:47.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.267+0000 7f19fc7f7740 10 client.15366 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:36:47.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.267+0000 7f19fc7f7740 20 client.15366 populate_metadata read hostname 'smithi067' 2024-09-21T06:36:47.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.267+0000 7f19fc7f7740 1 --2- 172.21.15.67:0/3136712259 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x564959b79f60 0x564959fac0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:47.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.267+0000 7f19fc7f7740 4 client.15366 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:36:47.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.267+0000 7f19fc7f7740 1 -- 172.21.15.67:0/3136712259 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "status"}) -- 0x564959e92a40 con 0x564959b79f60 2024-09-21T06:36:47.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.268+0000 7f19effff640 1 --2- 172.21.15.67:0/3136712259 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x564959b79f60 0x564959fac0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:47.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.268+0000 7f19effff640 1 --2- 172.21.15.67:0/3136712259 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x564959b79f60 0x564959fac0b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59ef105cd47bb4bb server_cookie=5d292fe2816a2521 in_seq=0 out_seq=0 2024-09-21T06:36:47.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.268+0000 7f19cf7fe640 10 client.15366 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:36:47.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.268+0000 7f19cf7fe640 1 -- 172.21.15.67:0/3136712259 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x564959e92a40 con 0x564959b79f60 2024-09-21T06:36:47.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.268+0000 7f19cf7fe640 10 client.15366 handle_command_reply: tid=0 2024-09-21T06:36:47.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.268+0000 7f19fc7f7740 1 client.15366 shutdown 2024-09-21T06:36:47.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.269+0000 7f19fc7f7740 1 -- 172.21.15.67:0/3136712259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564959b388e0 msgr2=0x564959f462a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:47.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.269+0000 7f19fc7f7740 1 --2- 172.21.15.67:0/3136712259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564959b388e0 0x564959f462a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f19d80098e0 tx=0x7f19d8049a10 comp rx=0 tx=0).stop 2024-09-21T06:36:47.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.269+0000 7f19fc7f7740 1 -- 172.21.15.67:0/3136712259 shutdown_connections 2024-09-21T06:36:47.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.269+0000 7f19fc7f7740 1 --2- 172.21.15.67:0/3136712259 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x564959b79f60 0x564959fac0b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.269+0000 7f19fc7f7740 1 --2- 172.21.15.67:0/3136712259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564959b510a0 0x564959f46880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.269+0000 7f19fc7f7740 1 --2- 172.21.15.67:0/3136712259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564959b59280 0x564959f3f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.269+0000 7f19fc7f7740 1 --2- 172.21.15.67:0/3136712259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564959b388e0 0x564959f462a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.269+0000 7f19fc7f7740 1 -- 172.21.15.67:0/3136712259 >> 172.21.15.67:0/3136712259 conn(0x564959b6e350 msgr2=0x564959f1d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:47.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.269+0000 7f19cf7fe640 0 client.15366 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:36:47.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.269+0000 7f19fc7f7740 1 -- 172.21.15.67:0/3136712259 shutdown_connections 2024-09-21T06:36:47.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.269+0000 7f19fc7f7740 1 -- 172.21.15.67:0/3136712259 wait complete. 2024-09-21T06:36:47.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.269+0000 7f19fc7f7740 20 client.15366 trim_cache size 0 max 16384 2024-09-21T06:36:47.276 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-21T06:36:47.276 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "2a791768-77e2-11ef-baf5-efdc52797490", 2024-09-21T06:36:47.276 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-09-21T06:36:47.276 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24671, 2024-09-21T06:36:47.276 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-09-21T06:36:47.276 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-09-21T06:36:47.276 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-09-21T06:36:47.276 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 133.62130267699999, 2024-09-21T06:36:47.276 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 21, 2024-09-21T06:36:47.277 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 85, 2024-09-21T06:36:47.277 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 83, 2024-09-21T06:36:47.277 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 145.201209011 2024-09-21T06:36:47.277 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-21T06:36:47.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.270+0000 7f19f6283640 1 -- 172.21.15.67:0/2403540753 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f19dc0b65c0 msgr2=0x7f19dc0b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:47.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.270+0000 7f19f6283640 1 --2- 172.21.15.67:0/2403540753 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f19dc0b65c0 0x7f19dc0b8ad0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f19e80aa750 tx=0x7f19e807a810 comp rx=0 tx=0).stop 2024-09-21T06:36:47.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.270+0000 7f19f6283640 1 -- 172.21.15.67:0/2403540753 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19f0108d10 msgr2=0x7f19f019fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:47.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.270+0000 7f19f6283640 1 --2- 172.21.15.67:0/2403540753 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19f0108d10 0x7f19f019fc70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f19e800eaa0 tx=0x7f19e800eea0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.270+0000 7f19f6283640 1 -- 172.21.15.67:0/2403540753 shutdown_connections 2024-09-21T06:36:47.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.270+0000 7f19f6283640 1 --2- 172.21.15.67:0/2403540753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19f01083c0 0x7f19f019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.270+0000 7f19f6283640 1 --2- 172.21.15.67:0/2403540753 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f19dc0b65c0 0x7f19dc0b8ad0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.270+0000 7f19f6283640 1 --2- 172.21.15.67:0/2403540753 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19f0108d10 0x7f19f019fc70 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.270+0000 7f19f6283640 1 --2- 172.21.15.67:0/2403540753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19f0102ad0 0x7f19f019a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:47.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.270+0000 7f19f6283640 1 -- 172.21.15.67:0/2403540753 >> 172.21.15.67:0/2403540753 conn(0x7f19f00796d0 msgr2=0x7f19f0106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:47.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.270+0000 7f19f6283640 1 -- 172.21.15.67:0/2403540753 shutdown_connections 2024-09-21T06:36:47.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:47.270+0000 7f19f6283640 1 -- 172.21.15.67:0/2403540753 wait complete. 2024-09-21T06:36:47.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:47 smithi203 ceph-mon[33735]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T06:36:47.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:47 smithi067 ceph-mon[31232]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T06:36:47.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:47 smithi100 ceph-mon[33401]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T06:36:47.959 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-21T06:36:47.982 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-21T06:36:47.982 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-09-21T06:36:47.982 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-09-21T06:36:47.983 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T06:36:47.983 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T06:36:47.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:36:47.983 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-09-21T06:36:48.001 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-21T06:36:48.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:36:48.001 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T06:36:48.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:36:48.069 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link delete ceph-brx 2024-09-21T06:36:48.171 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-21T06:36:48.171 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-21T06:36:48.171 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs ls 2024-09-21T06:36:48.430 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:36:48.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.933+0000 7fa2fe49d640 1 -- 172.21.15.67:0/2186574315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2f8108d20 msgr2=0x7fa2f810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:48.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.933+0000 7fa2fe49d640 1 --2- 172.21.15.67:0/2186574315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2f8108d20 0x7fa2f810f550 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0009ca0 tx=0x7fa2f002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:36:48.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.933+0000 7fa2fe49d640 1 -- 172.21.15.67:0/2186574315 shutdown_connections 2024-09-21T06:36:48.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.933+0000 7fa2fe49d640 1 --2- 172.21.15.67:0/2186574315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2f8108d20 0x7fa2f810f550 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:48.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.933+0000 7fa2fe49d640 1 --2- 172.21.15.67:0/2186574315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2f81083d0 0x7fa2f81087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:48.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.933+0000 7fa2fe49d640 1 --2- 172.21.15.67:0/2186574315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2f8102ae0 0x7fa2f8102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:48.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.933+0000 7fa2fe49d640 1 -- 172.21.15.67:0/2186574315 >> 172.21.15.67:0/2186574315 conn(0x7fa2f8079760 msgr2=0x7fa2f8079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:48.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.934+0000 7fa2fe49d640 1 -- 172.21.15.67:0/2186574315 shutdown_connections 2024-09-21T06:36:48.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.934+0000 7fa2fe49d640 1 -- 172.21.15.67:0/2186574315 wait complete. 2024-09-21T06:36:48.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.934+0000 7fa2fe49d640 1 Processor -- start 2024-09-21T06:36:48.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.934+0000 7fa2fe49d640 1 -- start start 2024-09-21T06:36:48.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.935+0000 7fa2fe49d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2f8102ae0 0x7fa2f8199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:48.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.935+0000 7fa2fe49d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2f81083d0 0x7fa2f8199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:48.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.935+0000 7fa2fe49d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2f8108d20 0x7fa2f819eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:48.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.935+0000 7fa2fe49d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f8112890 con 0x7fa2f81083d0 2024-09-21T06:36:48.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.935+0000 7fa2fe49d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f8112710 con 0x7fa2f8108d20 2024-09-21T06:36:48.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.935+0000 7fa2fe49d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f8112a10 con 0x7fa2f8102ae0 2024-09-21T06:36:48.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.935+0000 7fa2f77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2f81083d0 0x7fa2f8199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:48.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.935+0000 7fa2f77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2f81083d0 0x7fa2f8199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54640/0 (socket says 172.21.15.67:54640) 2024-09-21T06:36:48.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.935+0000 7fa2f77fe640 1 -- 172.21.15.67:0/3437379536 learned_addr learned my addr 172.21.15.67:0/3437379536 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:48.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.935+0000 7fa2f7fff640 1 --2- 172.21.15.67:0/3437379536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2f8102ae0 0x7fa2f8199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:48.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.935+0000 7fa2f77fe640 1 -- 172.21.15.67:0/3437379536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2f8102ae0 msgr2=0x7fa2f8199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:48.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.935+0000 7fa2f77fe640 1 --2- 172.21.15.67:0/3437379536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2f8102ae0 0x7fa2f8199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:48.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.935+0000 7fa2fca3a640 1 --2- 172.21.15.67:0/3437379536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2f8108d20 0x7fa2f819eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:48.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.935+0000 7fa2f77fe640 1 -- 172.21.15.67:0/3437379536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2f8108d20 msgr2=0x7fa2f819eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:48.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.935+0000 7fa2f77fe640 1 --2- 172.21.15.67:0/3437379536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2f8108d20 0x7fa2f819eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:48.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.935+0000 7fa2f77fe640 1 -- 172.21.15.67:0/3437379536 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f819f1d0 con 0x7fa2f81083d0 2024-09-21T06:36:48.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.935+0000 7fa2f7fff640 1 --2- 172.21.15.67:0/3437379536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2f8102ae0 0x7fa2f8199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:36:48.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.935+0000 7fa2fca3a640 1 --2- 172.21.15.67:0/3437379536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2f8108d20 0x7fa2f819eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:36:48.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.935+0000 7fa2f77fe640 1 --2- 172.21.15.67:0/3437379536 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2f81083d0 0x7fa2f8199800 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec005fd0 tx=0x7fa2ec00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:48.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.936+0000 7fa2f57fa640 1 -- 172.21.15.67:0/3437379536 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec017070 con 0x7fa2f81083d0 2024-09-21T06:36:48.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.936+0000 7fa2fe49d640 1 -- 172.21.15.67:0/3437379536 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2f819f4c0 con 0x7fa2f81083d0 2024-09-21T06:36:48.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.936+0000 7fa2f57fa640 1 -- 172.21.15.67:0/3437379536 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2ec009c30 con 0x7fa2f81083d0 2024-09-21T06:36:48.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.936+0000 7fa2f57fa640 1 -- 172.21.15.67:0/3437379536 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec003c70 con 0x7fa2f81083d0 2024-09-21T06:36:48.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.936+0000 7fa2fe49d640 1 -- 172.21.15.67:0/3437379536 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2f81a5c20 con 0x7fa2f81083d0 2024-09-21T06:36:48.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.937+0000 7fa2fe49d640 1 -- 172.21.15.67:0/3437379536 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2f8103d10 con 0x7fa2f81083d0 2024-09-21T06:36:48.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.938+0000 7fa2f57fa640 1 -- 172.21.15.67:0/3437379536 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa2ec003e10 con 0x7fa2f81083d0 2024-09-21T06:36:48.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.939+0000 7fa2f57fa640 1 --2- 172.21.15.67:0/3437379536 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fa2e00b6510 0x7fa2e00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:48.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.939+0000 7fa2f7fff640 1 --2- 172.21.15.67:0/3437379536 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fa2e00b6510 0x7fa2e00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:48.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.939+0000 7fa2f57fa640 1 -- 172.21.15.67:0/3437379536 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fa2ec0e7bb0 con 0x7fa2f81083d0 2024-09-21T06:36:48.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.940+0000 7fa2f7fff640 1 --2- 172.21.15.67:0/3437379536 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fa2e00b6510 0x7fa2e00b8a20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009c20 tx=0x7fa2e8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:48.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:48.942+0000 7fa2f57fa640 1 -- 172.21.15.67:0/3437379536 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa2ec020030 con 0x7fa2f81083d0 2024-09-21T06:36:49.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:49.170+0000 7fa2fe49d640 1 -- 172.21.15.67:0/3437379536 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa2f819a390 con 0x7fa2f81083d0 2024-09-21T06:36:49.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:49.171+0000 7fa2f57fa640 1 -- 172.21.15.67:0/3437379536 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) ==== 53+0+88 (secure 0 0 0) 0x7fa2ec0ae070 con 0x7fa2f81083d0 2024-09-21T06:36:49.173 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T06:36:49.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:49.173+0000 7fa2fe49d640 1 -- 172.21.15.67:0/3437379536 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fa2e00b6510 msgr2=0x7fa2e00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:49.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:49.173+0000 7fa2fe49d640 1 --2- 172.21.15.67:0/3437379536 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fa2e00b6510 0x7fa2e00b8a20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009c20 tx=0x7fa2e8005f90 comp rx=0 tx=0).stop 2024-09-21T06:36:49.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:49.173+0000 7fa2fe49d640 1 -- 172.21.15.67:0/3437379536 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2f81083d0 msgr2=0x7fa2f8199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:49.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:49.173+0000 7fa2fe49d640 1 --2- 172.21.15.67:0/3437379536 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2f81083d0 0x7fa2f8199800 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec005fd0 tx=0x7fa2ec00dde0 comp rx=0 tx=0).stop 2024-09-21T06:36:49.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:49.173+0000 7fa2fe49d640 1 -- 172.21.15.67:0/3437379536 shutdown_connections 2024-09-21T06:36:49.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:49.173+0000 7fa2fe49d640 1 --2- 172.21.15.67:0/3437379536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2f8108d20 0x7fa2f819eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:49.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:49.173+0000 7fa2fe49d640 1 --2- 172.21.15.67:0/3437379536 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fa2e00b6510 0x7fa2e00b8a20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:49.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:49.173+0000 7fa2fe49d640 1 --2- 172.21.15.67:0/3437379536 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2f81083d0 0x7fa2f8199800 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:49.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:49.173+0000 7fa2fe49d640 1 --2- 172.21.15.67:0/3437379536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2f8102ae0 0x7fa2f8199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:49.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:49.173+0000 7fa2fe49d640 1 -- 172.21.15.67:0/3437379536 >> 172.21.15.67:0/3437379536 conn(0x7fa2f8079760 msgr2=0x7fa2f81043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:49.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:49.173+0000 7fa2fe49d640 1 -- 172.21.15.67:0/3437379536 shutdown_connections 2024-09-21T06:36:49.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:49.173+0000 7fa2fe49d640 1 -- 172.21.15.67:0/3437379536 wait complete. 2024-09-21T06:36:49.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:49 smithi203 ceph-mon[33735]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-21T06:36:49.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:49 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3437379536' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T06:36:49.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:49 smithi067 ceph-mon[31232]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-21T06:36:49.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:49 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3437379536' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T06:36:49.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:49 smithi100 ceph-mon[33401]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-21T06:36:49.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:49 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3437379536' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T06:36:49.877 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T06:36:49.877 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T06:36:49.877 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2024-09-21T06:36:49.877 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T06:36:49.877 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T06:36:49.877 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T06:36:49.877 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T06:36:49.877 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T06:36:49.877 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:36:49.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:36:49.878 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2024-09-21T06:36:49.895 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T06:36:49.896 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T06:36:49.896 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T06:36:49.896 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:36:49.896 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2024-09-21T06:36:49.896 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:36:49.896 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:36:49.896 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2024-09-21T06:36:49.896 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f0 2024-09-21T06:36:49.896 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:36:49.896 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2024-09-21T06:36:49.896 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f1 2024-09-21T06:36:49.896 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:36:49.896 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2024-09-21T06:36:49.896 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T06:36:49.897 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2024-09-21T06:36:49.897 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T06:36:49.897 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 42122sec preferred_lft 42122sec 2024-09-21T06:36:49.897 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link noprefixroute 2024-09-21T06:36:49.897 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:36:49.897 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T06:36:49.897 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:36:49.897 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-09-21T06:36:49.897 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add name ceph-brx type bridge 2024-09-21T06:36:49.897 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr flush dev ceph-brx 2024-09-21T06:36:49.897 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set ceph-brx up 2024-09-21T06:36:49.898 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T06:36:49.898 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-09-21T06:36:49.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:36:50.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:36:50.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:36:50.040 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T06:36:50.065 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-09-21T06:36:50.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:36:50.067 DEBUG:teuthology.orchestra.run.smithi067:> route 2024-09-21T06:36:50.124 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2024-09-21T06:36:50.124 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T06:36:50.124 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T06:36:50.124 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T06:36:50.124 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T06:36:50.124 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:36:50.124 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-09-21T06:36:50.125 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T06:36:50.125 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T06:36:50.125 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T06:36:50.125 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-09-21T06:36:50.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:36:50.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:36:50.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:36:50.254 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-09-21T06:36:50.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:36:50.270 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list-id 2024-09-21T06:36:50.326 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:36:50.326 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-09-21T06:36:50.326 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T06:36:50.326 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-21T06:36:50.326 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-09-21T06:36:50.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:36:50.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:36:50.445 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-21T06:36:50.445 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:36:50.446 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-09-21T06:36:50.446 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-21T06:36:50.446 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T06:36:50.446 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-21T06:36:50.446 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-21T06:36:50.446 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-21T06:36:50.446 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-09-21T06:36:50.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:36:50.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:36:50.564 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:36:50.564 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-09-21T06:36:50.564 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2024-09-21T06:36:50.564 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T06:36:50.564 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-09-21T06:36:50.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:36:50.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:36:50.643 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-09-21T06:36:50.644 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:36:50.644 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T06:36:50.659 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T06:36:50.660 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:36:50.660 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:36:50.715 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-09-21T06:36:50.783 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T06:36:50.840 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-09-21T06:36:50.840 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-09-21T06:36:50.840 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-09-21T06:36:50.840 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-09-21T06:36:50.840 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-09-21T06:36:50.840 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-09-21T06:36:50.840 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-09-21T06:36:50.840 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-09-21T06:36:50.840 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-09-21T06:36:50.840 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-09-21T06:36:50.840 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-09-21T06:36:50.840 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-09-21T06:36:50.841 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T06:36:50.841 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-09-21T06:36:50.841 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-09-21T06:36:50.841 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-09-21T06:36:50.841 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-09-21T06:36:50.841 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-09-21T06:36:50.841 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-09-21T06:36:50.841 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T06:36:50.841 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T06:36:50.841 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T06:36:50.841 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T06:36:50.842 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-09-21T06:36:50.842 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-09-21T06:36:50.842 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay 2024-09-21T06:36:50.842 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/29fcd3741f2cc055bd00d5d0fa0c56546c5e775a4a1f53b1e931f43fdf5ff1e0/merged 2024-09-21T06:36:50.842 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/32acf6b7fe983d1dd15da3c2e23dc552a0df99470cc8cb96b4f745c41be94cde/merged 2024-09-21T06:36:50.842 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/0 2024-09-21T06:36:50.842 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T06:36:50.842 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/bafc1e5188868475e941c6d45eee229f72ba6ee7586332805d28b4774bba0dc2/merged 2024-09-21T06:36:50.842 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/39c1ee7a488e1cd808010f93a252b973411dc42fdab768269153bca71d230d48/merged 2024-09-21T06:36:50.842 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/477ce1b7f70d362fd35b8467af3f66a9ab9ddfebe22f2ca05e951c9c7e60264a/merged 2024-09-21T06:36:50.842 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/8363bb5cac1f856dec433b9c1ed2f566cd6bb9affd699fe771e5341a35835dd1/merged 2024-09-21T06:36:50.842 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/b8c3e6584fa2ea8ae7847e9343eb1ddf695ef1ff17a5598fb95d5cf5c8a54492/merged 2024-09-21T06:36:50.842 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/0cf20f5642d1e6f9125be88aaa36a3c5267b9e1ded4255030dff6a66c6178016/merged 2024-09-21T06:36:50.843 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/3d9fd1163c75a40ffd0f5d7f6a53413e9606d02c0f9e9a960e4c0b2f08910a6f/merged 2024-09-21T06:36:50.843 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/4525d5ac138e0d41fde6a3521150af1932abef421e5475cd735b2a2670135f97/merged 2024-09-21T06:36:50.843 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-09-21T06:36:50.843 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T06:36:50.843 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T06:36:50.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:36:50.843 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-09-21T06:36:50.896 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T06:36:50.896 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/d337545a-42eb-483d-9071-3b1bcc855266 --client_fs=cephfs) 2024-09-21T06:36:50.938 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-09-21T06:36:50.964 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T06:36:51.022 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-09-21T06:36:51.022 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-09-21T06:36:51.023 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-09-21T06:36:51.023 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-09-21T06:36:51.023 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-09-21T06:36:51.023 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-09-21T06:36:51.023 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-09-21T06:36:51.023 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-09-21T06:36:51.023 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-09-21T06:36:51.023 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-09-21T06:36:51.023 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-09-21T06:36:51.023 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-09-21T06:36:51.023 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T06:36:51.023 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-09-21T06:36:51.023 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-09-21T06:36:51.023 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-09-21T06:36:51.024 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-09-21T06:36:51.024 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-09-21T06:36:51.024 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-09-21T06:36:51.024 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T06:36:51.024 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T06:36:51.024 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T06:36:51.024 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T06:36:51.024 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-09-21T06:36:51.024 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-09-21T06:36:51.024 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay 2024-09-21T06:36:51.024 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/29fcd3741f2cc055bd00d5d0fa0c56546c5e775a4a1f53b1e931f43fdf5ff1e0/merged 2024-09-21T06:36:51.024 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/32acf6b7fe983d1dd15da3c2e23dc552a0df99470cc8cb96b4f745c41be94cde/merged 2024-09-21T06:36:51.024 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/0 2024-09-21T06:36:51.024 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T06:36:51.025 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/bafc1e5188868475e941c6d45eee229f72ba6ee7586332805d28b4774bba0dc2/merged 2024-09-21T06:36:51.025 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/39c1ee7a488e1cd808010f93a252b973411dc42fdab768269153bca71d230d48/merged 2024-09-21T06:36:51.025 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/477ce1b7f70d362fd35b8467af3f66a9ab9ddfebe22f2ca05e951c9c7e60264a/merged 2024-09-21T06:36:51.025 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/8363bb5cac1f856dec433b9c1ed2f566cd6bb9affd699fe771e5341a35835dd1/merged 2024-09-21T06:36:51.025 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/b8c3e6584fa2ea8ae7847e9343eb1ddf695ef1ff17a5598fb95d5cf5c8a54492/merged 2024-09-21T06:36:51.025 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/0cf20f5642d1e6f9125be88aaa36a3c5267b9e1ded4255030dff6a66c6178016/merged 2024-09-21T06:36:51.025 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/3d9fd1163c75a40ffd0f5d7f6a53413e9606d02c0f9e9a960e4c0b2f08910a6f/merged 2024-09-21T06:36:51.025 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/4525d5ac138e0d41fde6a3521150af1932abef421e5475cd735b2a2670135f97/merged 2024-09-21T06:36:51.025 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-09-21T06:36:51.025 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T06:36:51.025 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T06:36:51.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:36:51.026 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-09-21T06:36:51.033 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:2024-09-21T06:36:51.031+0000 7f62b7012480 -1 init, newargv = 0x7f629c004720 newargc=15 2024-09-21T06:36:51.033 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[84451]: starting ceph client 2024-09-21T06:36:51.042 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[84451]: starting fuse 2024-09-21T06:36:51.049 INFO:teuthology.orchestra.run.smithi067.stdout:120 2024-09-21T06:36:51.050 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-21T06:36:51.050 DEBUG:teuthology.orchestra.run.smithi067:> sudo stdin-killer -- python3 -c ' 2024-09-21T06:36:51.050 DEBUG:teuthology.orchestra.run.smithi067:> import glob 2024-09-21T06:36:51.050 DEBUG:teuthology.orchestra.run.smithi067:> import re 2024-09-21T06:36:51.050 DEBUG:teuthology.orchestra.run.smithi067:> import os 2024-09-21T06:36:51.050 DEBUG:teuthology.orchestra.run.smithi067:> import subprocess 2024-09-21T06:36:51.050 DEBUG:teuthology.orchestra.run.smithi067:> 2024-09-21T06:36:51.050 DEBUG:teuthology.orchestra.run.smithi067:> def _find_admin_socket(client_name): 2024-09-21T06:36:51.050 DEBUG:teuthology.orchestra.run.smithi067:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-21T06:36:51.050 DEBUG:teuthology.orchestra.run.smithi067:> files = glob.glob(asok_path) 2024-09-21T06:36:51.051 DEBUG:teuthology.orchestra.run.smithi067:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-21T06:36:51.051 DEBUG:teuthology.orchestra.run.smithi067:> 2024-09-21T06:36:51.051 DEBUG:teuthology.orchestra.run.smithi067:> # Given a non-glob path, it better be there 2024-09-21T06:36:51.051 DEBUG:teuthology.orchestra.run.smithi067:> if "*" not in asok_path: 2024-09-21T06:36:51.051 DEBUG:teuthology.orchestra.run.smithi067:> assert(len(files) == 1) 2024-09-21T06:36:51.051 DEBUG:teuthology.orchestra.run.smithi067:> return files[0] 2024-09-21T06:36:51.051 DEBUG:teuthology.orchestra.run.smithi067:> 2024-09-21T06:36:51.051 DEBUG:teuthology.orchestra.run.smithi067:> for f in files: 2024-09-21T06:36:51.051 DEBUG:teuthology.orchestra.run.smithi067:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T06:36:51.051 DEBUG:teuthology.orchestra.run.smithi067:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T06:36:51.051 DEBUG:teuthology.orchestra.run.smithi067:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T06:36:51.051 DEBUG:teuthology.orchestra.run.smithi067:> contents = proc_f.read() 2024-09-21T06:36:51.051 DEBUG:teuthology.orchestra.run.smithi067:> if mountpoint in contents: 2024-09-21T06:36:51.051 DEBUG:teuthology.orchestra.run.smithi067:> return f 2024-09-21T06:36:51.051 DEBUG:teuthology.orchestra.run.smithi067:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T06:36:51.051 DEBUG:teuthology.orchestra.run.smithi067:> 2024-09-21T06:36:51.051 DEBUG:teuthology.orchestra.run.smithi067:> print(_find_admin_socket("client.0")) 2024-09-21T06:36:51.052 DEBUG:teuthology.orchestra.run.smithi067:> ' 2024-09-21T06:36:51.170 INFO:teuthology.orchestra.run.smithi067.stdout:/var/run/ceph/ceph-client.0.84451.asok 2024-09-21T06:36:51.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:36:51.180 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.84451.asok 2024-09-21T06:36:51.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:36:51.181 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.84451.asok status 2024-09-21T06:36:51.327 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-21T06:36:51.327 INFO:teuthology.orchestra.run.smithi067.stdout: "metadata": { 2024-09-21T06:36:51.327 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T06:36:51.327 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T06:36:51.327 INFO:teuthology.orchestra.run.smithi067.stdout: "entity_id": "0", 2024-09-21T06:36:51.327 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": "smithi067", 2024-09-21T06:36:51.327 INFO:teuthology.orchestra.run.smithi067.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-21T06:36:51.327 INFO:teuthology.orchestra.run.smithi067.stdout: "pid": "84451", 2024-09-21T06:36:51.327 INFO:teuthology.orchestra.run.smithi067.stdout: "root": "/volumes/qa/sv_0/d337545a-42eb-483d-9071-3b1bcc855266" 2024-09-21T06:36:51.328 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:36:51.328 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_count": 0, 2024-09-21T06:36:51.328 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_pinned_count": 0, 2024-09-21T06:36:51.328 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 15378, 2024-09-21T06:36:51.328 INFO:teuthology.orchestra.run.smithi067.stdout: "inst": { 2024-09-21T06:36:51.328 INFO:teuthology.orchestra.run.smithi067.stdout: "name": { 2024-09-21T06:36:51.328 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "client", 2024-09-21T06:36:51.328 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 15378 2024-09-21T06:36:51.328 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:36:51.328 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-09-21T06:36:51.328 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-09-21T06:36:51.328 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2024-09-21T06:36:51.328 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 967588866 2024-09-21T06:36:51.328 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:36:51.328 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:36:51.328 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-09-21T06:36:51.329 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-09-21T06:36:51.329 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2024-09-21T06:36:51.329 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 967588866 2024-09-21T06:36:51.329 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:36:51.329 INFO:teuthology.orchestra.run.smithi067.stdout: "inst_str": "client.15378 192.168.0.1:0/967588866", 2024-09-21T06:36:51.329 INFO:teuthology.orchestra.run.smithi067.stdout: "addr_str": "192.168.0.1:0/967588866", 2024-09-21T06:36:51.329 INFO:teuthology.orchestra.run.smithi067.stdout: "inode_count": 5, 2024-09-21T06:36:51.329 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_epoch": 21, 2024-09-21T06:36:51.329 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch": 85, 2024-09-21T06:36:51.329 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch_barrier": 0, 2024-09-21T06:36:51.329 INFO:teuthology.orchestra.run.smithi067.stdout: "blocklisted": false, 2024-09-21T06:36:51.329 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs" 2024-09-21T06:36:51.329 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-21T06:36:51.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:36:51.336 DEBUG:teuthology.orchestra.run.smithi067:> stat --file-system '--printf=%T 2024-09-21T06:36:51.336 DEBUG:teuthology.orchestra.run.smithi067:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-21T06:36:51.352 INFO:teuthology.orchestra.run.smithi067.stdout:fuseblk 2024-09-21T06:36:51.352 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-21T06:36:51.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:36:51.353 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T06:36:51.419 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-21T06:36:51.432 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-21T06:36:51.449 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T06:36:51.449 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T06:36:51.717 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:36:51.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:51 smithi203 ceph-mon[33735]: pgmap v368: 129 pgs: 129 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:36:51.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:51 smithi067 ceph-mon[31232]: pgmap v368: 129 pgs: 129 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:36:51.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:51 smithi100 ceph-mon[33401]: pgmap v368: 129 pgs: 129 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:36:52.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.247+0000 7f416e9ba640 1 -- 172.21.15.67:0/3880832171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41681083c0 msgr2=0x7f41681087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:52.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.247+0000 7f416e9ba640 1 --2- 172.21.15.67:0/3880832171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41681083c0 0x7f41681087d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f415c009900 tx=0x7f415c02a410 comp rx=0 tx=0).stop 2024-09-21T06:36:52.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.247+0000 7f416e9ba640 1 -- 172.21.15.67:0/3880832171 shutdown_connections 2024-09-21T06:36:52.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.247+0000 7f416e9ba640 1 --2- 172.21.15.67:0/3880832171 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4168108d10 0x7f416810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:52.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.247+0000 7f416e9ba640 1 --2- 172.21.15.67:0/3880832171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41681083c0 0x7f41681087d0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:52.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.247+0000 7f416e9ba640 1 --2- 172.21.15.67:0/3880832171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4168102a80 0x7f4168102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:52.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.247+0000 7f416e9ba640 1 -- 172.21.15.67:0/3880832171 >> 172.21.15.67:0/3880832171 conn(0x7f4168079740 msgr2=0x7f4168079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:52.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.247+0000 7f416e9ba640 1 -- 172.21.15.67:0/3880832171 shutdown_connections 2024-09-21T06:36:52.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.247+0000 7f416e9ba640 1 -- 172.21.15.67:0/3880832171 wait complete. 2024-09-21T06:36:52.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.247+0000 7f416e9ba640 1 Processor -- start 2024-09-21T06:36:52.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.247+0000 7f416e9ba640 1 -- start start 2024-09-21T06:36:52.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.248+0000 7f416e9ba640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4168102a80 0x7f416819b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:52.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.248+0000 7f416e9ba640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41681083c0 0x7f416819b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:52.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.248+0000 7f416e9ba640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4168108d10 0x7f416819bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:52.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.248+0000 7f416e9ba640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4168112830 con 0x7f4168102a80 2024-09-21T06:36:52.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.248+0000 7f416e9ba640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f41681126b0 con 0x7f41681083c0 2024-09-21T06:36:52.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.248+0000 7f416e9ba640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f41681129b0 con 0x7f4168108d10 2024-09-21T06:36:52.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.248+0000 7f4167fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4168102a80 0x7f416819b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:52.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.248+0000 7f4167fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4168102a80 0x7f416819b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54646/0 (socket says 172.21.15.67:54646) 2024-09-21T06:36:52.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.248+0000 7f4167fff640 1 -- 172.21.15.67:0/2881500433 learned_addr learned my addr 172.21.15.67:0/2881500433 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:52.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.248+0000 7f41677fe640 1 --2- 172.21.15.67:0/2881500433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41681083c0 0x7f416819b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:52.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.248+0000 7f416cf57640 1 --2- 172.21.15.67:0/2881500433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4168108d10 0x7f416819bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:52.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.248+0000 7f4167fff640 1 -- 172.21.15.67:0/2881500433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4168108d10 msgr2=0x7f416819bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:52.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.248+0000 7f4167fff640 1 --2- 172.21.15.67:0/2881500433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4168108d10 0x7f416819bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:52.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.248+0000 7f4167fff640 1 -- 172.21.15.67:0/2881500433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41681083c0 msgr2=0x7f416819b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:52.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.248+0000 7f4167fff640 1 --2- 172.21.15.67:0/2881500433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41681083c0 0x7f416819b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:52.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.248+0000 7f4167fff640 1 -- 172.21.15.67:0/2881500433 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41681a6540 con 0x7f4168102a80 2024-09-21T06:36:52.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.249+0000 7f4167fff640 1 --2- 172.21.15.67:0/2881500433 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4168102a80 0x7f416819b280 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f415800c890 tx=0x7f415800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:52.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.249+0000 7f41657fa640 1 -- 172.21.15.67:0/2881500433 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4158004220 con 0x7f4168102a80 2024-09-21T06:36:52.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.249+0000 7f41657fa640 1 -- 172.21.15.67:0/2881500433 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4158007e40 con 0x7f4168102a80 2024-09-21T06:36:52.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.249+0000 7f416e9ba640 1 -- 172.21.15.67:0/2881500433 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41681a6830 con 0x7f4168102a80 2024-09-21T06:36:52.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.249+0000 7f41657fa640 1 -- 172.21.15.67:0/2881500433 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4158004a00 con 0x7f4168102a80 2024-09-21T06:36:52.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.249+0000 7f416e9ba640 1 -- 172.21.15.67:0/2881500433 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41681a6d90 con 0x7f4168102a80 2024-09-21T06:36:52.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.250+0000 7f416e9ba640 1 -- 172.21.15.67:0/2881500433 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4168103d00 con 0x7f4168102a80 2024-09-21T06:36:52.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.252+0000 7f41657fa640 1 -- 172.21.15.67:0/2881500433 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f41580079b0 con 0x7f4168102a80 2024-09-21T06:36:52.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.252+0000 7f41657fa640 1 --2- 172.21.15.67:0/2881500433 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f41500b6590 0x7f41500b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:52.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.253+0000 7f41677fe640 1 --2- 172.21.15.67:0/2881500433 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f41500b6590 0x7f41500b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:52.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.253+0000 7f41657fa640 1 -- 172.21.15.67:0/2881500433 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f41580e7750 con 0x7f4168102a80 2024-09-21T06:36:52.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.253+0000 7f41677fe640 1 --2- 172.21.15.67:0/2881500433 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f41500b6590 0x7f41500b8aa0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f415c005ec0 tx=0x7f415c0058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:52.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.255+0000 7f41657fa640 1 -- 172.21.15.67:0/2881500433 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4158014030 con 0x7f4168102a80 2024-09-21T06:36:52.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.442+0000 7f416e9ba640 1 -- 172.21.15.67:0/2881500433 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4168063800 con 0x7f41500b6590 2024-09-21T06:36:52.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.448+0000 7f41657fa640 1 -- 172.21.15.67:0/2881500433 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340645 (secure 0 0 0) 0x7f4168063800 con 0x7f41500b6590 2024-09-21T06:36:52.453 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:36:52.454 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-09-21T06:36:52.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.451+0000 7f416e9ba640 1 -- 172.21.15.67:0/2881500433 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f41500b6590 msgr2=0x7f41500b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:52.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.451+0000 7f416e9ba640 1 --2- 172.21.15.67:0/2881500433 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f41500b6590 0x7f41500b8aa0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f415c005ec0 tx=0x7f415c0058d0 comp rx=0 tx=0).stop 2024-09-21T06:36:52.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.452+0000 7f416e9ba640 1 -- 172.21.15.67:0/2881500433 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4168102a80 msgr2=0x7f416819b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:52.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.452+0000 7f416e9ba640 1 --2- 172.21.15.67:0/2881500433 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4168102a80 0x7f416819b280 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f415800c890 tx=0x7f415800cd50 comp rx=0 tx=0).stop 2024-09-21T06:36:52.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.452+0000 7f416e9ba640 1 -- 172.21.15.67:0/2881500433 shutdown_connections 2024-09-21T06:36:52.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.452+0000 7f416e9ba640 1 --2- 172.21.15.67:0/2881500433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41681083c0 0x7f416819b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:52.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.452+0000 7f416e9ba640 1 --2- 172.21.15.67:0/2881500433 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f41500b6590 0x7f41500b8aa0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:52.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.452+0000 7f416e9ba640 1 --2- 172.21.15.67:0/2881500433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4168108d10 0x7f416819bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:52.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.452+0000 7f416e9ba640 1 --2- 172.21.15.67:0/2881500433 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4168102a80 0x7f416819b280 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:52.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.452+0000 7f416e9ba640 1 -- 172.21.15.67:0/2881500433 >> 172.21.15.67:0/2881500433 conn(0x7f4168079740 msgr2=0x7f4168104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:52.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.452+0000 7f416e9ba640 1 -- 172.21.15.67:0/2881500433 shutdown_connections 2024-09-21T06:36:52.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:52.452+0000 7f416e9ba640 1 -- 172.21.15.67:0/2881500433 wait complete. 2024-09-21T06:36:53.156 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2024-09-21T06:36:51.881929+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660434,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":294,"num_read_kb":243,"num_write":215,"num_write_kb":2251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":669,"ondisk_log_size":669,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368880,"kb_used_data":11312,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572944,"statfs":{"total":1151940427776,"available":1151562694656,"internally_reserved":0,"allocated":11583488,"data_stored":7792604,"data_compressed":462771,"data_compressed_allocated":626688,"data_compressed_original":2752512,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003644"},"pg_stats":[{"pgid":"2.1f","version":"75'4","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621812+0000","last_change":"2024-09-21T06:34:30.633366+0000","last_active":"2024-09-21T06:36:05.621812+0000","last_peered":"2024-09-21T06:36:05.621812+0000","last_clean":"2024-09-21T06:36:05.621812+0000","last_became_active":"2024-09-21T06:34:30.632929+0000","last_became_peered":"2024-09-21T06:34:30.632929+0000","last_unstale":"2024-09-21T06:36:05.621812+0000","last_undegraded":"2024-09-21T06:36:05.621812+0000","last_fullsized":"2024-09-21T06:36:05.621812+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:47.722461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.238429+0000","last_change":"2024-09-21T06:34:32.632989+0000","last_active":"2024-09-21T06:36:34.238429+0000","last_peered":"2024-09-21T06:36:34.238429+0000","last_clean":"2024-09-21T06:36:34.238429+0000","last_became_active":"2024-09-21T06:34:32.632024+0000","last_became_peered":"2024-09-21T06:34:32.632024+0000","last_unstale":"2024-09-21T06:36:34.238429+0000","last_undegraded":"2024-09-21T06:36:34.238429+0000","last_fullsized":"2024-09-21T06:36:34.238429+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:13.669613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621767+0000","last_change":"2024-09-21T06:35:05.274434+0000","last_active":"2024-09-21T06:36:05.621767+0000","last_peered":"2024-09-21T06:36:05.621767+0000","last_clean":"2024-09-21T06:36:05.621767+0000","last_became_active":"2024-09-21T06:35:05.273833+0000","last_became_peered":"2024-09-21T06:35:05.273833+0000","last_unstale":"2024-09-21T06:36:05.621767+0000","last_undegraded":"2024-09-21T06:36:05.621767+0000","last_fullsized":"2024-09-21T06:36:05.621767+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:48.847861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835213+0000","last_change":"2024-09-21T06:34:30.946309+0000","last_active":"2024-09-21T06:36:05.835213+0000","last_peered":"2024-09-21T06:36:05.835213+0000","last_clean":"2024-09-21T06:36:05.835213+0000","last_became_active":"2024-09-21T06:34:30.945913+0000","last_became_peered":"2024-09-21T06:34:30.945913+0000","last_unstale":"2024-09-21T06:36:05.835213+0000","last_undegraded":"2024-09-21T06:36:05.835213+0000","last_fullsized":"2024-09-21T06:36:05.835213+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:21.676524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.246270+0000","last_change":"2024-09-21T06:34:32.637636+0000","last_active":"2024-09-21T06:36:34.246270+0000","last_peered":"2024-09-21T06:36:34.246270+0000","last_clean":"2024-09-21T06:36:34.246270+0000","last_became_active":"2024-09-21T06:34:32.636534+0000","last_became_peered":"2024-09-21T06:34:32.636534+0000","last_unstale":"2024-09-21T06:36:34.246270+0000","last_undegraded":"2024-09-21T06:36:34.246270+0000","last_fullsized":"2024-09-21T06:36:34.246270+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:05.802118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771498+0000","last_change":"2024-09-21T06:35:05.274419+0000","last_active":"2024-09-21T06:36:05.771498+0000","last_peered":"2024-09-21T06:36:05.771498+0000","last_clean":"2024-09-21T06:36:05.771498+0000","last_became_active":"2024-09-21T06:35:05.273865+0000","last_became_peered":"2024-09-21T06:35:05.273865+0000","last_unstale":"2024-09-21T06:36:05.771498+0000","last_undegraded":"2024-09-21T06:36:05.771498+0000","last_fullsized":"2024-09-21T06:36:05.771498+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:04.490355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.631709+0000","last_change":"2024-09-21T06:36:05.631098+0000","last_active":"2024-09-21T06:36:05.631709+0000","last_peered":"2024-09-21T06:36:05.631709+0000","last_clean":"2024-09-21T06:36:05.631709+0000","last_became_active":"2024-09-21T06:36:05.627521+0000","last_became_peered":"2024-09-21T06:36:05.627521+0000","last_unstale":"2024-09-21T06:36:05.631709+0000","last_undegraded":"2024-09-21T06:36:05.631709+0000","last_fullsized":"2024-09-21T06:36:05.631709+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:02.467758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.247327+0000","last_change":"2024-09-21T06:34:32.650303+0000","last_active":"2024-09-21T06:36:34.247327+0000","last_peered":"2024-09-21T06:36:34.247327+0000","last_clean":"2024-09-21T06:36:34.247327+0000","last_became_active":"2024-09-21T06:34:32.647189+0000","last_became_peered":"2024-09-21T06:34:32.647189+0000","last_unstale":"2024-09-21T06:36:34.247327+0000","last_undegraded":"2024-09-21T06:36:34.247327+0000","last_fullsized":"2024-09-21T06:36:34.247327+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:14.889384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771714+0000","last_change":"2024-09-21T06:34:39.713139+0000","last_active":"2024-09-21T06:36:05.771714+0000","last_peered":"2024-09-21T06:36:05.771714+0000","last_clean":"2024-09-21T06:36:05.771714+0000","last_became_active":"2024-09-21T06:34:39.712379+0000","last_became_peered":"2024-09-21T06:34:39.712379+0000","last_unstale":"2024-09-21T06:36:05.771714+0000","last_undegraded":"2024-09-21T06:36:05.771714+0000","last_fullsized":"2024-09-21T06:36:05.771714+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:53.100529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.625272+0000","last_change":"2024-09-21T06:34:30.633850+0000","last_active":"2024-09-21T06:36:05.625272+0000","last_peered":"2024-09-21T06:36:05.625272+0000","last_clean":"2024-09-21T06:36:05.625272+0000","last_became_active":"2024-09-21T06:34:30.633468+0000","last_became_peered":"2024-09-21T06:34:30.633468+0000","last_unstale":"2024-09-21T06:36:05.625272+0000","last_undegraded":"2024-09-21T06:36:05.625272+0000","last_fullsized":"2024-09-21T06:36:05.625272+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:59.095803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":56,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.247919+0000","last_change":"2024-09-21T06:35:05.005240+0000","last_active":"2024-09-21T06:36:34.247919+0000","last_peered":"2024-09-21T06:36:34.247919+0000","last_clean":"2024-09-21T06:36:34.247919+0000","last_became_active":"2024-09-21T06:35:05.004849+0000","last_became_peered":"2024-09-21T06:35:05.004849+0000","last_unstale":"2024-09-21T06:36:34.247919+0000","last_undegraded":"2024-09-21T06:36:34.247919+0000","last_fullsized":"2024-09-21T06:36:34.247919+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:28.849604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771556+0000","last_change":"2024-09-21T06:35:05.274437+0000","last_active":"2024-09-21T06:36:05.771556+0000","last_peered":"2024-09-21T06:36:05.771556+0000","last_clean":"2024-09-21T06:36:05.771556+0000","last_became_active":"2024-09-21T06:35:05.273865+0000","last_became_peered":"2024-09-21T06:35:05.273865+0000","last_unstale":"2024-09-21T06:36:05.771556+0000","last_undegraded":"2024-09-21T06:36:05.771556+0000","last_fullsized":"2024-09-21T06:36:05.771556+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:43.339504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.246814+0000","last_change":"2024-09-21T06:34:32.645630+0000","last_active":"2024-09-21T06:36:34.246814+0000","last_peered":"2024-09-21T06:36:34.246814+0000","last_clean":"2024-09-21T06:36:34.246814+0000","last_became_active":"2024-09-21T06:34:32.645237+0000","last_became_peered":"2024-09-21T06:34:32.645237+0000","last_unstale":"2024-09-21T06:36:34.246814+0000","last_undegraded":"2024-09-21T06:36:34.246814+0000","last_fullsized":"2024-09-21T06:36:34.246814+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:25.064318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.633983+0000","last_change":"2024-09-21T06:36:05.633484+0000","last_active":"2024-09-21T06:36:05.633983+0000","last_peered":"2024-09-21T06:36:05.633983+0000","last_clean":"2024-09-21T06:36:05.633983+0000","last_became_active":"2024-09-21T06:36:05.627637+0000","last_became_peered":"2024-09-21T06:36:05.627637+0000","last_unstale":"2024-09-21T06:36:05.633983+0000","last_undegraded":"2024-09-21T06:36:05.633983+0000","last_fullsized":"2024-09-21T06:36:05.633983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:50:00.626538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771936+0000","last_change":"2024-09-21T06:34:39.714177+0000","last_active":"2024-09-21T06:36:05.771936+0000","last_peered":"2024-09-21T06:36:05.771936+0000","last_clean":"2024-09-21T06:36:05.771936+0000","last_became_active":"2024-09-21T06:34:39.713713+0000","last_became_peered":"2024-09-21T06:34:39.713713+0000","last_unstale":"2024-09-21T06:36:05.771936+0000","last_undegraded":"2024-09-21T06:36:05.771936+0000","last_fullsized":"2024-09-21T06:36:05.771936+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:07.042240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.234553+0000","last_change":"2024-09-21T06:34:32.636297+0000","last_active":"2024-09-21T06:36:34.234553+0000","last_peered":"2024-09-21T06:36:34.234553+0000","last_clean":"2024-09-21T06:36:34.234553+0000","last_became_active":"2024-09-21T06:34:32.635930+0000","last_became_peered":"2024-09-21T06:34:32.635930+0000","last_unstale":"2024-09-21T06:36:34.234553+0000","last_undegraded":"2024-09-21T06:36:34.234553+0000","last_fullsized":"2024-09-21T06:36:34.234553+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:43.091682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621975+0000","last_change":"2024-09-21T06:34:30.631405+0000","last_active":"2024-09-21T06:36:05.621975+0000","last_peered":"2024-09-21T06:36:05.621975+0000","last_clean":"2024-09-21T06:36:05.621975+0000","last_became_active":"2024-09-21T06:34:30.630512+0000","last_became_peered":"2024-09-21T06:34:30.630512+0000","last_unstale":"2024-09-21T06:36:05.621975+0000","last_undegraded":"2024-09-21T06:36:05.621975+0000","last_fullsized":"2024-09-21T06:36:05.621975+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:43.412695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"83'1","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771897+0000","last_change":"2024-09-21T06:34:39.710573+0000","last_active":"2024-09-21T06:36:05.771897+0000","last_peered":"2024-09-21T06:36:05.771897+0000","last_clean":"2024-09-21T06:36:05.771897+0000","last_became_active":"2024-09-21T06:34:39.708969+0000","last_became_peered":"2024-09-21T06:34:39.708969+0000","last_unstale":"2024-09-21T06:36:05.771897+0000","last_undegraded":"2024-09-21T06:36:05.771897+0000","last_fullsized":"2024-09-21T06:36:05.771897+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:34.847587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.234683+0000","last_change":"2024-09-21T06:34:32.641085+0000","last_active":"2024-09-21T06:36:34.234683+0000","last_peered":"2024-09-21T06:36:34.234683+0000","last_clean":"2024-09-21T06:36:34.234683+0000","last_became_active":"2024-09-21T06:34:32.640723+0000","last_became_peered":"2024-09-21T06:34:32.640723+0000","last_unstale":"2024-09-21T06:36:34.234683+0000","last_undegraded":"2024-09-21T06:36:34.234683+0000","last_fullsized":"2024-09-21T06:36:34.234683+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:21.410262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834947+0000","last_change":"2024-09-21T06:34:30.626719+0000","last_active":"2024-09-21T06:36:05.834947+0000","last_peered":"2024-09-21T06:36:05.834947+0000","last_clean":"2024-09-21T06:36:05.834947+0000","last_became_active":"2024-09-21T06:34:30.626312+0000","last_became_peered":"2024-09-21T06:34:30.626312+0000","last_unstale":"2024-09-21T06:36:05.834947+0000","last_undegraded":"2024-09-21T06:36:05.834947+0000","last_fullsized":"2024-09-21T06:36:05.834947+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:53.938914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621617+0000","last_change":"2024-09-21T06:34:39.711172+0000","last_active":"2024-09-21T06:36:05.621617+0000","last_peered":"2024-09-21T06:36:05.621617+0000","last_clean":"2024-09-21T06:36:05.621617+0000","last_became_active":"2024-09-21T06:34:39.708424+0000","last_became_peered":"2024-09-21T06:34:39.708424+0000","last_unstale":"2024-09-21T06:36:05.621617+0000","last_undegraded":"2024-09-21T06:36:05.621617+0000","last_fullsized":"2024-09-21T06:36:05.621617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:43.918355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.209527+0000","last_change":"2024-09-21T06:34:32.641463+0000","last_active":"2024-09-21T06:36:34.209527+0000","last_peered":"2024-09-21T06:36:34.209527+0000","last_clean":"2024-09-21T06:36:34.209527+0000","last_became_active":"2024-09-21T06:34:32.641086+0000","last_became_peered":"2024-09-21T06:34:32.641086+0000","last_unstale":"2024-09-21T06:36:34.209527+0000","last_undegraded":"2024-09-21T06:36:34.209527+0000","last_fullsized":"2024-09-21T06:36:34.209527+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:58.991374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835725+0000","last_change":"2024-09-21T06:34:30.946698+0000","last_active":"2024-09-21T06:36:05.835725+0000","last_peered":"2024-09-21T06:36:05.835725+0000","last_clean":"2024-09-21T06:36:05.835725+0000","last_became_active":"2024-09-21T06:34:30.946325+0000","last_became_peered":"2024-09-21T06:34:30.946325+0000","last_unstale":"2024-09-21T06:36:05.835725+0000","last_undegraded":"2024-09-21T06:36:05.835725+0000","last_fullsized":"2024-09-21T06:36:05.835725+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:22.639265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.772211+0000","last_change":"2024-09-21T06:34:39.712429+0000","last_active":"2024-09-21T06:36:05.772211+0000","last_peered":"2024-09-21T06:36:05.772211+0000","last_clean":"2024-09-21T06:36:05.772211+0000","last_became_active":"2024-09-21T06:34:39.711823+0000","last_became_peered":"2024-09-21T06:34:39.711823+0000","last_unstale":"2024-09-21T06:36:05.772211+0000","last_undegraded":"2024-09-21T06:36:05.772211+0000","last_fullsized":"2024-09-21T06:36:05.772211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:03:49.513991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.243704+0000","last_change":"2024-09-21T06:35:05.274326+0000","last_active":"2024-09-21T06:36:34.243704+0000","last_peered":"2024-09-21T06:36:34.243704+0000","last_clean":"2024-09-21T06:36:34.243704+0000","last_became_active":"2024-09-21T06:35:05.273884+0000","last_became_peered":"2024-09-21T06:35:05.273884+0000","last_unstale":"2024-09-21T06:36:34.243704+0000","last_undegraded":"2024-09-21T06:36:34.243704+0000","last_fullsized":"2024-09-21T06:36:34.243704+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:03.219765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.772393+0000","last_change":"2024-09-21T06:34:30.676794+0000","last_active":"2024-09-21T06:36:05.772393+0000","last_peered":"2024-09-21T06:36:05.772393+0000","last_clean":"2024-09-21T06:36:05.772393+0000","last_became_active":"2024-09-21T06:34:30.676400+0000","last_became_peered":"2024-09-21T06:34:30.676400+0000","last_unstale":"2024-09-21T06:36:05.772393+0000","last_undegraded":"2024-09-21T06:36:05.772393+0000","last_fullsized":"2024-09-21T06:36:05.772393+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:23.681363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622160+0000","last_change":"2024-09-21T06:34:39.708980+0000","last_active":"2024-09-21T06:36:05.622160+0000","last_peered":"2024-09-21T06:36:05.622160+0000","last_clean":"2024-09-21T06:36:05.622160+0000","last_became_active":"2024-09-21T06:34:39.707748+0000","last_became_peered":"2024-09-21T06:34:39.707748+0000","last_unstale":"2024-09-21T06:36:05.622160+0000","last_undegraded":"2024-09-21T06:36:05.622160+0000","last_fullsized":"2024-09-21T06:36:05.622160+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:16.812226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.240400+0000","last_change":"2024-09-21T06:34:32.638337+0000","last_active":"2024-09-21T06:36:34.240400+0000","last_peered":"2024-09-21T06:36:34.240400+0000","last_clean":"2024-09-21T06:36:34.240400+0000","last_became_active":"2024-09-21T06:34:32.638013+0000","last_became_peered":"2024-09-21T06:34:32.638013+0000","last_unstale":"2024-09-21T06:36:34.240400+0000","last_undegraded":"2024-09-21T06:36:34.240400+0000","last_fullsized":"2024-09-21T06:36:34.240400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:26.321190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'10","reported_seq":50,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.673339+0000","last_change":"2024-09-21T06:34:30.631086+0000","last_active":"2024-09-21T06:36:34.673339+0000","last_peered":"2024-09-21T06:36:34.673339+0000","last_clean":"2024-09-21T06:36:34.673339+0000","last_became_active":"2024-09-21T06:34:30.630706+0000","last_became_peered":"2024-09-21T06:34:30.630706+0000","last_unstale":"2024-09-21T06:36:34.673339+0000","last_undegraded":"2024-09-21T06:36:34.673339+0000","last_fullsized":"2024-09-21T06:36:34.673339+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:30.260359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834003+0000","last_change":"2024-09-21T06:34:39.709097+0000","last_active":"2024-09-21T06:36:05.834003+0000","last_peered":"2024-09-21T06:36:05.834003+0000","last_clean":"2024-09-21T06:36:05.834003+0000","last_became_active":"2024-09-21T06:34:39.708609+0000","last_became_peered":"2024-09-21T06:34:39.708609+0000","last_unstale":"2024-09-21T06:36:05.834003+0000","last_undegraded":"2024-09-21T06:36:05.834003+0000","last_fullsized":"2024-09-21T06:36:05.834003+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:48.842174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.229558+0000","last_change":"2024-09-21T06:34:32.636816+0000","last_active":"2024-09-21T06:36:34.229558+0000","last_peered":"2024-09-21T06:36:34.229558+0000","last_clean":"2024-09-21T06:36:34.229558+0000","last_became_active":"2024-09-21T06:34:32.636429+0000","last_became_peered":"2024-09-21T06:34:32.636429+0000","last_unstale":"2024-09-21T06:36:34.229558+0000","last_undegraded":"2024-09-21T06:36:34.229558+0000","last_fullsized":"2024-09-21T06:36:34.229558+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:49.010491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.625629+0000","last_change":"2024-09-21T06:34:30.947481+0000","last_active":"2024-09-21T06:36:05.625629+0000","last_peered":"2024-09-21T06:36:05.625629+0000","last_clean":"2024-09-21T06:36:05.625629+0000","last_became_active":"2024-09-21T06:34:30.947040+0000","last_became_peered":"2024-09-21T06:34:30.947040+0000","last_unstale":"2024-09-21T06:36:05.625629+0000","last_undegraded":"2024-09-21T06:36:05.625629+0000","last_fullsized":"2024-09-21T06:36:05.625629+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:52.813476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624972+0000","last_change":"2024-09-21T06:34:39.708674+0000","last_active":"2024-09-21T06:36:05.624972+0000","last_peered":"2024-09-21T06:36:05.624972+0000","last_clean":"2024-09-21T06:36:05.624972+0000","last_became_active":"2024-09-21T06:34:39.708176+0000","last_became_peered":"2024-09-21T06:34:39.708176+0000","last_unstale":"2024-09-21T06:36:05.624972+0000","last_undegraded":"2024-09-21T06:36:05.624972+0000","last_fullsized":"2024-09-21T06:36:05.624972+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:57.070792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"85'4","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.239723+0000","last_change":"2024-09-21T06:34:32.636381+0000","last_active":"2024-09-21T06:36:34.239723+0000","last_peered":"2024-09-21T06:36:34.239723+0000","last_clean":"2024-09-21T06:36:34.239723+0000","last_became_active":"2024-09-21T06:34:32.635937+0000","last_became_peered":"2024-09-21T06:34:32.635937+0000","last_unstale":"2024-09-21T06:36:34.239723+0000","last_undegraded":"2024-09-21T06:36:34.239723+0000","last_fullsized":"2024-09-21T06:36:34.239723+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:52.870676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621524+0000","last_change":"2024-09-21T06:34:30.947160+0000","last_active":"2024-09-21T06:36:05.621524+0000","last_peered":"2024-09-21T06:36:05.621524+0000","last_clean":"2024-09-21T06:36:05.621524+0000","last_became_active":"2024-09-21T06:34:30.946736+0000","last_became_peered":"2024-09-21T06:34:30.946736+0000","last_unstale":"2024-09-21T06:36:05.621524+0000","last_undegraded":"2024-09-21T06:36:05.621524+0000","last_fullsized":"2024-09-21T06:36:05.621524+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:47.425481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.625820+0000","last_change":"2024-09-21T06:35:05.006032+0000","last_active":"2024-09-21T06:36:05.625820+0000","last_peered":"2024-09-21T06:36:05.625820+0000","last_clean":"2024-09-21T06:36:05.625820+0000","last_became_active":"2024-09-21T06:35:05.005429+0000","last_became_peered":"2024-09-21T06:35:05.005429+0000","last_unstale":"2024-09-21T06:36:05.625820+0000","last_undegraded":"2024-09-21T06:36:05.625820+0000","last_fullsized":"2024-09-21T06:36:05.625820+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:04.138097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.238876+0000","last_change":"2024-09-21T06:34:32.648256+0000","last_active":"2024-09-21T06:36:34.238876+0000","last_peered":"2024-09-21T06:36:34.238876+0000","last_clean":"2024-09-21T06:36:34.238876+0000","last_became_active":"2024-09-21T06:34:32.647881+0000","last_became_peered":"2024-09-21T06:34:32.647881+0000","last_unstale":"2024-09-21T06:36:34.238876+0000","last_undegraded":"2024-09-21T06:36:34.238876+0000","last_fullsized":"2024-09-21T06:36:34.238876+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:14.590834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622945+0000","last_change":"2024-09-21T06:34:30.629918+0000","last_active":"2024-09-21T06:36:05.622945+0000","last_peered":"2024-09-21T06:36:05.622945+0000","last_clean":"2024-09-21T06:36:05.622945+0000","last_became_active":"2024-09-21T06:34:30.628785+0000","last_became_peered":"2024-09-21T06:34:30.628785+0000","last_unstale":"2024-09-21T06:36:05.622945+0000","last_undegraded":"2024-09-21T06:36:05.622945+0000","last_fullsized":"2024-09-21T06:36:05.622945+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:05.333406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834425+0000","last_change":"2024-09-21T06:34:39.710073+0000","last_active":"2024-09-21T06:36:05.834425+0000","last_peered":"2024-09-21T06:36:05.834425+0000","last_clean":"2024-09-21T06:36:05.834425+0000","last_became_active":"2024-09-21T06:34:39.708986+0000","last_became_peered":"2024-09-21T06:34:39.708986+0000","last_unstale":"2024-09-21T06:36:05.834425+0000","last_undegraded":"2024-09-21T06:36:05.834425+0000","last_fullsized":"2024-09-21T06:36:05.834425+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:46.379902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.234328+0000","last_change":"2024-09-21T06:34:32.644224+0000","last_active":"2024-09-21T06:36:34.234328+0000","last_peered":"2024-09-21T06:36:34.234328+0000","last_clean":"2024-09-21T06:36:34.234328+0000","last_became_active":"2024-09-21T06:34:32.643850+0000","last_became_peered":"2024-09-21T06:34:32.643850+0000","last_unstale":"2024-09-21T06:36:34.234328+0000","last_undegraded":"2024-09-21T06:36:34.234328+0000","last_fullsized":"2024-09-21T06:36:34.234328+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:24:33.188537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.833952+0000","last_change":"2024-09-21T06:34:30.946334+0000","last_active":"2024-09-21T06:36:05.833952+0000","last_peered":"2024-09-21T06:36:05.833952+0000","last_clean":"2024-09-21T06:36:05.833952+0000","last_became_active":"2024-09-21T06:34:30.945908+0000","last_became_peered":"2024-09-21T06:34:30.945908+0000","last_unstale":"2024-09-21T06:36:05.833952+0000","last_undegraded":"2024-09-21T06:36:05.833952+0000","last_fullsized":"2024-09-21T06:36:05.833952+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:46.746527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834586+0000","last_change":"2024-09-21T06:34:39.707909+0000","last_active":"2024-09-21T06:36:05.834586+0000","last_peered":"2024-09-21T06:36:05.834586+0000","last_clean":"2024-09-21T06:36:05.834586+0000","last_became_active":"2024-09-21T06:34:39.707492+0000","last_became_peered":"2024-09-21T06:34:39.707492+0000","last_unstale":"2024-09-21T06:36:05.834586+0000","last_undegraded":"2024-09-21T06:36:05.834586+0000","last_fullsized":"2024-09-21T06:36:05.834586+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:56.201212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.227669+0000","last_change":"2024-09-21T06:34:32.635870+0000","last_active":"2024-09-21T06:36:34.227669+0000","last_peered":"2024-09-21T06:36:34.227669+0000","last_clean":"2024-09-21T06:36:34.227669+0000","last_became_active":"2024-09-21T06:34:32.635252+0000","last_became_peered":"2024-09-21T06:34:32.635252+0000","last_unstale":"2024-09-21T06:36:34.227669+0000","last_undegraded":"2024-09-21T06:36:34.227669+0000","last_fullsized":"2024-09-21T06:36:34.227669+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:10.613033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622347+0000","last_change":"2024-09-21T06:34:30.634360+0000","last_active":"2024-09-21T06:36:05.622347+0000","last_peered":"2024-09-21T06:36:05.622347+0000","last_clean":"2024-09-21T06:36:05.622347+0000","last_became_active":"2024-09-21T06:34:30.633845+0000","last_became_peered":"2024-09-21T06:34:30.633845+0000","last_unstale":"2024-09-21T06:36:05.622347+0000","last_undegraded":"2024-09-21T06:36:05.622347+0000","last_fullsized":"2024-09-21T06:36:05.622347+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:34:53.388670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834837+0000","last_change":"2024-09-21T06:34:39.719620+0000","last_active":"2024-09-21T06:36:05.834837+0000","last_peered":"2024-09-21T06:36:05.834837+0000","last_clean":"2024-09-21T06:36:05.834837+0000","last_became_active":"2024-09-21T06:34:39.718786+0000","last_became_peered":"2024-09-21T06:34:39.718786+0000","last_unstale":"2024-09-21T06:36:05.834837+0000","last_undegraded":"2024-09-21T06:36:05.834837+0000","last_fullsized":"2024-09-21T06:36:05.834837+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:43.524932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.236491+0000","last_change":"2024-09-21T06:34:32.634970+0000","last_active":"2024-09-21T06:36:34.236491+0000","last_peered":"2024-09-21T06:36:34.236491+0000","last_clean":"2024-09-21T06:36:34.236491+0000","last_became_active":"2024-09-21T06:34:32.634611+0000","last_became_peered":"2024-09-21T06:34:32.634611+0000","last_unstale":"2024-09-21T06:36:34.236491+0000","last_undegraded":"2024-09-21T06:36:34.236491+0000","last_fullsized":"2024-09-21T06:36:34.236491+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:37.855849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.772176+0000","last_change":"2024-09-21T06:34:30.628123+0000","last_active":"2024-09-21T06:36:05.772176+0000","last_peered":"2024-09-21T06:36:05.772176+0000","last_clean":"2024-09-21T06:36:05.772176+0000","last_became_active":"2024-09-21T06:34:30.627498+0000","last_became_peered":"2024-09-21T06:34:30.627498+0000","last_unstale":"2024-09-21T06:36:05.772176+0000","last_undegraded":"2024-09-21T06:36:05.772176+0000","last_fullsized":"2024-09-21T06:36:05.772176+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:42.576441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":115,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622503+0000","last_change":"2024-09-21T06:33:16.856623+0000","last_active":"2024-09-21T06:36:05.622503+0000","last_peered":"2024-09-21T06:36:05.622503+0000","last_clean":"2024-09-21T06:36:05.622503+0000","last_became_active":"2024-09-21T06:33:16.850787+0000","last_became_peered":"2024-09-21T06:33:16.850787+0000","last_unstale":"2024-09-21T06:36:05.622503+0000","last_undegraded":"2024-09-21T06:36:05.622503+0000","last_fullsized":"2024-09-21T06:36:05.622503+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:50.283252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:50.283252+0000","last_clean_scrub_stamp":"2024-09-21T06:29:50.283252+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:29:20.711208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622549+0000","last_change":"2024-09-21T06:34:39.706472+0000","last_active":"2024-09-21T06:36:05.622549+0000","last_peered":"2024-09-21T06:36:05.622549+0000","last_clean":"2024-09-21T06:36:05.622549+0000","last_became_active":"2024-09-21T06:34:39.705724+0000","last_became_peered":"2024-09-21T06:34:39.705724+0000","last_unstale":"2024-09-21T06:36:05.622549+0000","last_undegraded":"2024-09-21T06:36:05.622549+0000","last_fullsized":"2024-09-21T06:36:05.622549+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:43.238704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.246993+0000","last_change":"2024-09-21T06:34:32.645944+0000","last_active":"2024-09-21T06:36:34.246993+0000","last_peered":"2024-09-21T06:36:34.246993+0000","last_clean":"2024-09-21T06:36:34.246993+0000","last_became_active":"2024-09-21T06:34:32.645134+0000","last_became_peered":"2024-09-21T06:34:32.645134+0000","last_unstale":"2024-09-21T06:36:34.246993+0000","last_undegraded":"2024-09-21T06:36:34.246993+0000","last_fullsized":"2024-09-21T06:36:34.246993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:45.254164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622645+0000","last_change":"2024-09-21T06:34:30.946296+0000","last_active":"2024-09-21T06:36:05.622645+0000","last_peered":"2024-09-21T06:36:05.622645+0000","last_clean":"2024-09-21T06:36:05.622645+0000","last_became_active":"2024-09-21T06:34:30.945777+0000","last_became_peered":"2024-09-21T06:34:30.945777+0000","last_unstale":"2024-09-21T06:36:05.622645+0000","last_undegraded":"2024-09-21T06:36:05.622645+0000","last_fullsized":"2024-09-21T06:36:05.622645+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:32:25.988512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835130+0000","last_change":"2024-09-21T06:34:39.705876+0000","last_active":"2024-09-21T06:36:05.835130+0000","last_peered":"2024-09-21T06:36:05.835130+0000","last_clean":"2024-09-21T06:36:05.835130+0000","last_became_active":"2024-09-21T06:34:39.705132+0000","last_became_peered":"2024-09-21T06:34:39.705132+0000","last_unstale":"2024-09-21T06:36:05.835130+0000","last_undegraded":"2024-09-21T06:36:05.835130+0000","last_fullsized":"2024-09-21T06:36:05.835130+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:18.180244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"85'6","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.242114+0000","last_change":"2024-09-21T06:34:32.638242+0000","last_active":"2024-09-21T06:36:34.242114+0000","last_peered":"2024-09-21T06:36:34.242114+0000","last_clean":"2024-09-21T06:36:34.242114+0000","last_became_active":"2024-09-21T06:34:32.637816+0000","last_became_peered":"2024-09-21T06:34:32.637816+0000","last_unstale":"2024-09-21T06:36:34.242114+0000","last_undegraded":"2024-09-21T06:36:34.242114+0000","last_fullsized":"2024-09-21T06:36:34.242114+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:40.926147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835006+0000","last_change":"2024-09-21T06:34:30.632173+0000","last_active":"2024-09-21T06:36:05.835006+0000","last_peered":"2024-09-21T06:36:05.835006+0000","last_clean":"2024-09-21T06:36:05.835006+0000","last_became_active":"2024-09-21T06:34:30.631638+0000","last_became_peered":"2024-09-21T06:34:30.631638+0000","last_unstale":"2024-09-21T06:36:05.835006+0000","last_undegraded":"2024-09-21T06:36:05.835006+0000","last_fullsized":"2024-09-21T06:36:05.835006+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:29.763272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.628935+0000","last_change":"2024-09-21T06:34:39.714620+0000","last_active":"2024-09-21T06:36:05.628935+0000","last_peered":"2024-09-21T06:36:05.628935+0000","last_clean":"2024-09-21T06:36:05.628935+0000","last_became_active":"2024-09-21T06:34:39.714199+0000","last_became_peered":"2024-09-21T06:34:39.714199+0000","last_unstale":"2024-09-21T06:36:05.628935+0000","last_undegraded":"2024-09-21T06:36:05.628935+0000","last_fullsized":"2024-09-21T06:36:05.628935+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:37.041249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.236401+0000","last_change":"2024-09-21T06:34:32.645001+0000","last_active":"2024-09-21T06:36:34.236401+0000","last_peered":"2024-09-21T06:36:34.236401+0000","last_clean":"2024-09-21T06:36:34.236401+0000","last_became_active":"2024-09-21T06:34:32.644584+0000","last_became_peered":"2024-09-21T06:34:32.644584+0000","last_unstale":"2024-09-21T06:36:34.236401+0000","last_undegraded":"2024-09-21T06:36:34.236401+0000","last_fullsized":"2024-09-21T06:36:34.236401+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:47.098850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835010+0000","last_change":"2024-09-21T06:34:30.632593+0000","last_active":"2024-09-21T06:36:05.835010+0000","last_peered":"2024-09-21T06:36:05.835010+0000","last_clean":"2024-09-21T06:36:05.835010+0000","last_became_active":"2024-09-21T06:34:30.632170+0000","last_became_peered":"2024-09-21T06:34:30.632170+0000","last_unstale":"2024-09-21T06:36:05.835010+0000","last_undegraded":"2024-09-21T06:36:05.835010+0000","last_fullsized":"2024-09-21T06:36:05.835010+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:56.170655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834120+0000","last_change":"2024-09-21T06:34:39.710172+0000","last_active":"2024-09-21T06:36:05.834120+0000","last_peered":"2024-09-21T06:36:05.834120+0000","last_clean":"2024-09-21T06:36:05.834120+0000","last_became_active":"2024-09-21T06:34:39.708876+0000","last_became_peered":"2024-09-21T06:34:39.708876+0000","last_unstale":"2024-09-21T06:36:05.834120+0000","last_undegraded":"2024-09-21T06:36:05.834120+0000","last_fullsized":"2024-09-21T06:36:05.834120+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:32.371873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.226698+0000","last_change":"2024-09-21T06:34:32.651056+0000","last_active":"2024-09-21T06:36:34.226698+0000","last_peered":"2024-09-21T06:36:34.226698+0000","last_clean":"2024-09-21T06:36:34.226698+0000","last_became_active":"2024-09-21T06:34:32.650676+0000","last_became_peered":"2024-09-21T06:34:32.650676+0000","last_unstale":"2024-09-21T06:36:34.226698+0000","last_undegraded":"2024-09-21T06:36:34.226698+0000","last_fullsized":"2024-09-21T06:36:34.226698+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:34.599654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.625707+0000","last_change":"2024-09-21T06:34:30.632307+0000","last_active":"2024-09-21T06:36:05.625707+0000","last_peered":"2024-09-21T06:36:05.625707+0000","last_clean":"2024-09-21T06:36:05.625707+0000","last_became_active":"2024-09-21T06:34:30.631920+0000","last_became_peered":"2024-09-21T06:34:30.631920+0000","last_unstale":"2024-09-21T06:36:05.625707+0000","last_undegraded":"2024-09-21T06:36:05.625707+0000","last_fullsized":"2024-09-21T06:36:05.625707+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:46.940898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834319+0000","last_change":"2024-09-21T06:34:39.715056+0000","last_active":"2024-09-21T06:36:05.834319+0000","last_peered":"2024-09-21T06:36:05.834319+0000","last_clean":"2024-09-21T06:36:05.834319+0000","last_became_active":"2024-09-21T06:34:39.714563+0000","last_became_peered":"2024-09-21T06:34:39.714563+0000","last_unstale":"2024-09-21T06:36:05.834319+0000","last_undegraded":"2024-09-21T06:36:05.834319+0000","last_fullsized":"2024-09-21T06:36:05.834319+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:48.567057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.246554+0000","last_change":"2024-09-21T06:34:32.646934+0000","last_active":"2024-09-21T06:36:34.246554+0000","last_peered":"2024-09-21T06:36:34.246554+0000","last_clean":"2024-09-21T06:36:34.246554+0000","last_became_active":"2024-09-21T06:34:32.646559+0000","last_became_peered":"2024-09-21T06:34:32.646559+0000","last_unstale":"2024-09-21T06:36:34.246554+0000","last_undegraded":"2024-09-21T06:36:34.246554+0000","last_fullsized":"2024-09-21T06:36:34.246554+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:37.802590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":65,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.843557+0000","last_change":"2024-09-21T06:36:05.843018+0000","last_active":"2024-09-21T06:36:05.843557+0000","last_peered":"2024-09-21T06:36:05.843557+0000","last_clean":"2024-09-21T06:36:05.843557+0000","last_became_active":"2024-09-21T06:36:05.836284+0000","last_became_peered":"2024-09-21T06:36:05.836284+0000","last_unstale":"2024-09-21T06:36:05.843557+0000","last_undegraded":"2024-09-21T06:36:05.843557+0000","last_fullsized":"2024-09-21T06:36:05.843557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:51.539804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834952+0000","last_change":"2024-09-21T06:34:39.713398+0000","last_active":"2024-09-21T06:36:05.834952+0000","last_peered":"2024-09-21T06:36:05.834952+0000","last_clean":"2024-09-21T06:36:05.834952+0000","last_became_active":"2024-09-21T06:34:39.712954+0000","last_became_peered":"2024-09-21T06:34:39.712954+0000","last_unstale":"2024-09-21T06:36:05.834952+0000","last_undegraded":"2024-09-21T06:36:05.834952+0000","last_fullsized":"2024-09-21T06:36:05.834952+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:58.881014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.236031+0000","last_change":"2024-09-21T06:34:32.640879+0000","last_active":"2024-09-21T06:36:34.236031+0000","last_peered":"2024-09-21T06:36:34.236031+0000","last_clean":"2024-09-21T06:36:34.236031+0000","last_became_active":"2024-09-21T06:34:32.640526+0000","last_became_peered":"2024-09-21T06:34:32.640526+0000","last_unstale":"2024-09-21T06:36:34.236031+0000","last_undegraded":"2024-09-21T06:36:34.236031+0000","last_fullsized":"2024-09-21T06:36:34.236031+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:12.327195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771987+0000","last_change":"2024-09-21T06:34:30.631449+0000","last_active":"2024-09-21T06:36:05.771987+0000","last_peered":"2024-09-21T06:36:05.771987+0000","last_clean":"2024-09-21T06:36:05.771987+0000","last_became_active":"2024-09-21T06:34:30.631081+0000","last_became_peered":"2024-09-21T06:34:30.631081+0000","last_unstale":"2024-09-21T06:36:05.771987+0000","last_undegraded":"2024-09-21T06:36:05.771987+0000","last_fullsized":"2024-09-21T06:36:05.771987+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:09.202696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621660+0000","last_change":"2024-09-21T06:34:39.707379+0000","last_active":"2024-09-21T06:36:05.621660+0000","last_peered":"2024-09-21T06:36:05.621660+0000","last_clean":"2024-09-21T06:36:05.621660+0000","last_became_active":"2024-09-21T06:34:39.706979+0000","last_became_peered":"2024-09-21T06:34:39.706979+0000","last_unstale":"2024-09-21T06:36:05.621660+0000","last_undegraded":"2024-09-21T06:36:05.621660+0000","last_fullsized":"2024-09-21T06:36:05.621660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:24.394233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.243091+0000","last_change":"2024-09-21T06:34:32.649458+0000","last_active":"2024-09-21T06:36:34.243091+0000","last_peered":"2024-09-21T06:36:34.243091+0000","last_clean":"2024-09-21T06:36:34.243091+0000","last_became_active":"2024-09-21T06:34:32.649051+0000","last_became_peered":"2024-09-21T06:34:32.649051+0000","last_unstale":"2024-09-21T06:36:34.243091+0000","last_undegraded":"2024-09-21T06:36:34.243091+0000","last_fullsized":"2024-09-21T06:36:34.243091+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:12.631446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834455+0000","last_change":"2024-09-21T06:34:30.634592+0000","last_active":"2024-09-21T06:36:05.834455+0000","last_peered":"2024-09-21T06:36:05.834455+0000","last_clean":"2024-09-21T06:36:05.834455+0000","last_became_active":"2024-09-21T06:34:30.634251+0000","last_became_peered":"2024-09-21T06:34:30.634251+0000","last_unstale":"2024-09-21T06:36:05.834455+0000","last_undegraded":"2024-09-21T06:36:05.834455+0000","last_fullsized":"2024-09-21T06:36:05.834455+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:32.817467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624233+0000","last_change":"2024-09-21T06:34:39.719416+0000","last_active":"2024-09-21T06:36:05.624233+0000","last_peered":"2024-09-21T06:36:05.624233+0000","last_clean":"2024-09-21T06:36:05.624233+0000","last_became_active":"2024-09-21T06:34:39.718910+0000","last_became_peered":"2024-09-21T06:34:39.718910+0000","last_unstale":"2024-09-21T06:36:05.624233+0000","last_undegraded":"2024-09-21T06:36:05.624233+0000","last_fullsized":"2024-09-21T06:36:05.624233+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:41.413250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.235711+0000","last_change":"2024-09-21T06:34:32.640727+0000","last_active":"2024-09-21T06:36:34.235711+0000","last_peered":"2024-09-21T06:36:34.235711+0000","last_clean":"2024-09-21T06:36:34.235711+0000","last_became_active":"2024-09-21T06:34:32.640296+0000","last_became_peered":"2024-09-21T06:34:32.640296+0000","last_unstale":"2024-09-21T06:36:34.235711+0000","last_undegraded":"2024-09-21T06:36:34.235711+0000","last_fullsized":"2024-09-21T06:36:34.235711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:35.124059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622197+0000","last_change":"2024-09-21T06:34:30.633332+0000","last_active":"2024-09-21T06:36:05.622197+0000","last_peered":"2024-09-21T06:36:05.622197+0000","last_clean":"2024-09-21T06:36:05.622197+0000","last_became_active":"2024-09-21T06:34:30.632641+0000","last_became_peered":"2024-09-21T06:34:30.632641+0000","last_unstale":"2024-09-21T06:36:05.622197+0000","last_undegraded":"2024-09-21T06:36:05.622197+0000","last_fullsized":"2024-09-21T06:36:05.622197+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:30:23.930313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834659+0000","last_change":"2024-09-21T06:34:39.712504+0000","last_active":"2024-09-21T06:36:05.834659+0000","last_peered":"2024-09-21T06:36:05.834659+0000","last_clean":"2024-09-21T06:36:05.834659+0000","last_became_active":"2024-09-21T06:34:39.711638+0000","last_became_peered":"2024-09-21T06:34:39.711638+0000","last_unstale":"2024-09-21T06:36:05.834659+0000","last_undegraded":"2024-09-21T06:36:05.834659+0000","last_fullsized":"2024-09-21T06:36:05.834659+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:26:09.003367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.237201+0000","last_change":"2024-09-21T06:34:32.639582+0000","last_active":"2024-09-21T06:36:34.237201+0000","last_peered":"2024-09-21T06:36:34.237201+0000","last_clean":"2024-09-21T06:36:34.237201+0000","last_became_active":"2024-09-21T06:34:32.639023+0000","last_became_peered":"2024-09-21T06:34:32.639023+0000","last_unstale":"2024-09-21T06:36:34.237201+0000","last_undegraded":"2024-09-21T06:36:34.237201+0000","last_fullsized":"2024-09-21T06:36:34.237201+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:26.773389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"85'33","reported_seq":287,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:50.204582+0000","last_change":"2024-09-21T06:34:30.626280+0000","last_active":"2024-09-21T06:36:50.204582+0000","last_peered":"2024-09-21T06:36:50.204582+0000","last_clean":"2024-09-21T06:36:50.204582+0000","last_became_active":"2024-09-21T06:34:30.625662+0000","last_became_peered":"2024-09-21T06:34:30.625662+0000","last_unstale":"2024-09-21T06:36:50.204582+0000","last_undegraded":"2024-09-21T06:36:50.204582+0000","last_fullsized":"2024-09-21T06:36:50.204582+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:10.541881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69034,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":214,"num_read_kb":175,"num_write":33,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621183+0000","last_change":"2024-09-21T06:34:39.708999+0000","last_active":"2024-09-21T06:36:05.621183+0000","last_peered":"2024-09-21T06:36:05.621183+0000","last_clean":"2024-09-21T06:36:05.621183+0000","last_became_active":"2024-09-21T06:34:39.708580+0000","last_became_peered":"2024-09-21T06:34:39.708580+0000","last_unstale":"2024-09-21T06:36:05.621183+0000","last_undegraded":"2024-09-21T06:36:05.621183+0000","last_fullsized":"2024-09-21T06:36:05.621183+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:07.648358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.247017+0000","last_change":"2024-09-21T06:34:32.651157+0000","last_active":"2024-09-21T06:36:34.247017+0000","last_peered":"2024-09-21T06:36:34.247017+0000","last_clean":"2024-09-21T06:36:34.247017+0000","last_became_active":"2024-09-21T06:34:32.650804+0000","last_became_peered":"2024-09-21T06:34:32.650804+0000","last_unstale":"2024-09-21T06:36:34.247017+0000","last_undegraded":"2024-09-21T06:36:34.247017+0000","last_fullsized":"2024-09-21T06:36:34.247017+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:49.758496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834880+0000","last_change":"2024-09-21T06:34:30.947142+0000","last_active":"2024-09-21T06:36:05.834880+0000","last_peered":"2024-09-21T06:36:05.834880+0000","last_clean":"2024-09-21T06:36:05.834880+0000","last_became_active":"2024-09-21T06:34:30.946748+0000","last_became_peered":"2024-09-21T06:34:30.946748+0000","last_unstale":"2024-09-21T06:36:05.834880+0000","last_undegraded":"2024-09-21T06:36:05.834880+0000","last_fullsized":"2024-09-21T06:36:05.834880+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:39.222286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771885+0000","last_change":"2024-09-21T06:34:39.709266+0000","last_active":"2024-09-21T06:36:05.771885+0000","last_peered":"2024-09-21T06:36:05.771885+0000","last_clean":"2024-09-21T06:36:05.771885+0000","last_became_active":"2024-09-21T06:34:39.708753+0000","last_became_peered":"2024-09-21T06:34:39.708753+0000","last_unstale":"2024-09-21T06:36:05.771885+0000","last_undegraded":"2024-09-21T06:36:05.771885+0000","last_fullsized":"2024-09-21T06:36:05.771885+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:48.711681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.241271+0000","last_change":"2024-09-21T06:34:32.637976+0000","last_active":"2024-09-21T06:36:34.241271+0000","last_peered":"2024-09-21T06:36:34.241271+0000","last_clean":"2024-09-21T06:36:34.241271+0000","last_became_active":"2024-09-21T06:34:32.637267+0000","last_became_peered":"2024-09-21T06:34:32.637267+0000","last_unstale":"2024-09-21T06:36:34.241271+0000","last_undegraded":"2024-09-21T06:36:34.241271+0000","last_fullsized":"2024-09-21T06:36:34.241271+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:31.340583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.836482+0000","last_change":"2024-09-21T06:36:05.836382+0000","last_active":"2024-09-21T06:36:05.836482+0000","last_peered":"2024-09-21T06:36:05.836482+0000","last_clean":"2024-09-21T06:36:05.836482+0000","last_became_active":"2024-09-21T06:36:05.835918+0000","last_became_peered":"2024-09-21T06:36:05.835918+0000","last_unstale":"2024-09-21T06:36:05.836482+0000","last_undegraded":"2024-09-21T06:36:05.836482+0000","last_fullsized":"2024-09-21T06:36:05.836482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:47.346400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621023+0000","last_change":"2024-09-21T06:34:39.713831+0000","last_active":"2024-09-21T06:36:05.621023+0000","last_peered":"2024-09-21T06:36:05.621023+0000","last_clean":"2024-09-21T06:36:05.621023+0000","last_became_active":"2024-09-21T06:34:39.713403+0000","last_became_peered":"2024-09-21T06:34:39.713403+0000","last_unstale":"2024-09-21T06:36:05.621023+0000","last_undegraded":"2024-09-21T06:36:05.621023+0000","last_fullsized":"2024-09-21T06:36:05.621023+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:22.201063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.246305+0000","last_change":"2024-09-21T06:34:32.646924+0000","last_active":"2024-09-21T06:36:34.246305+0000","last_peered":"2024-09-21T06:36:34.246305+0000","last_clean":"2024-09-21T06:36:34.246305+0000","last_became_active":"2024-09-21T06:34:32.646531+0000","last_became_peered":"2024-09-21T06:34:32.646531+0000","last_unstale":"2024-09-21T06:36:34.246305+0000","last_undegraded":"2024-09-21T06:36:34.246305+0000","last_fullsized":"2024-09-21T06:36:34.246305+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:48.550193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835325+0000","last_change":"2024-09-21T06:34:30.631901+0000","last_active":"2024-09-21T06:36:05.835325+0000","last_peered":"2024-09-21T06:36:05.835325+0000","last_clean":"2024-09-21T06:36:05.835325+0000","last_became_active":"2024-09-21T06:34:30.631450+0000","last_became_peered":"2024-09-21T06:34:30.631450+0000","last_unstale":"2024-09-21T06:36:05.835325+0000","last_undegraded":"2024-09-21T06:36:05.835325+0000","last_fullsized":"2024-09-21T06:36:05.835325+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:20.709456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834441+0000","last_change":"2024-09-21T06:34:39.711951+0000","last_active":"2024-09-21T06:36:05.834441+0000","last_peered":"2024-09-21T06:36:05.834441+0000","last_clean":"2024-09-21T06:36:05.834441+0000","last_became_active":"2024-09-21T06:34:39.710783+0000","last_became_peered":"2024-09-21T06:34:39.710783+0000","last_unstale":"2024-09-21T06:36:05.834441+0000","last_undegraded":"2024-09-21T06:36:05.834441+0000","last_fullsized":"2024-09-21T06:36:05.834441+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:27:58.501155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.629078+0000","last_change":"2024-09-21T06:34:39.709639+0000","last_active":"2024-09-21T06:36:05.629078+0000","last_peered":"2024-09-21T06:36:05.629078+0000","last_clean":"2024-09-21T06:36:05.629078+0000","last_became_active":"2024-09-21T06:34:39.709241+0000","last_became_peered":"2024-09-21T06:34:39.709241+0000","last_unstale":"2024-09-21T06:36:05.629078+0000","last_undegraded":"2024-09-21T06:36:05.629078+0000","last_fullsized":"2024-09-21T06:36:05.629078+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.357399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.240180+0000","last_change":"2024-09-21T06:34:32.635365+0000","last_active":"2024-09-21T06:36:34.240180+0000","last_peered":"2024-09-21T06:36:34.240180+0000","last_clean":"2024-09-21T06:36:34.240180+0000","last_became_active":"2024-09-21T06:34:32.635012+0000","last_became_peered":"2024-09-21T06:34:32.635012+0000","last_unstale":"2024-09-21T06:36:34.240180+0000","last_undegraded":"2024-09-21T06:36:34.240180+0000","last_fullsized":"2024-09-21T06:36:34.240180+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:54.671074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834581+0000","last_change":"2024-09-21T06:34:30.633347+0000","last_active":"2024-09-21T06:36:05.834581+0000","last_peered":"2024-09-21T06:36:05.834581+0000","last_clean":"2024-09-21T06:36:05.834581+0000","last_became_active":"2024-09-21T06:34:30.632513+0000","last_became_peered":"2024-09-21T06:34:30.632513+0000","last_unstale":"2024-09-21T06:36:05.834581+0000","last_undegraded":"2024-09-21T06:36:05.834581+0000","last_fullsized":"2024-09-21T06:36:05.834581+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:12.439109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624854+0000","last_change":"2024-09-21T06:34:39.710204+0000","last_active":"2024-09-21T06:36:05.624854+0000","last_peered":"2024-09-21T06:36:05.624854+0000","last_clean":"2024-09-21T06:36:05.624854+0000","last_became_active":"2024-09-21T06:34:39.709744+0000","last_became_peered":"2024-09-21T06:34:39.709744+0000","last_unstale":"2024-09-21T06:36:05.624854+0000","last_undegraded":"2024-09-21T06:36:05.624854+0000","last_fullsized":"2024-09-21T06:36:05.624854+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:45.484088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.229155+0000","last_change":"2024-09-21T06:34:32.634170+0000","last_active":"2024-09-21T06:36:34.229155+0000","last_peered":"2024-09-21T06:36:34.229155+0000","last_clean":"2024-09-21T06:36:34.229155+0000","last_became_active":"2024-09-21T06:34:32.633831+0000","last_became_peered":"2024-09-21T06:34:32.633831+0000","last_unstale":"2024-09-21T06:36:34.229155+0000","last_undegraded":"2024-09-21T06:36:34.229155+0000","last_fullsized":"2024-09-21T06:36:34.229155+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:36.738279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.833746+0000","last_change":"2024-09-21T06:34:30.629422+0000","last_active":"2024-09-21T06:36:05.833746+0000","last_peered":"2024-09-21T06:36:05.833746+0000","last_clean":"2024-09-21T06:36:05.833746+0000","last_became_active":"2024-09-21T06:34:30.629025+0000","last_became_peered":"2024-09-21T06:34:30.629025+0000","last_unstale":"2024-09-21T06:36:05.833746+0000","last_undegraded":"2024-09-21T06:36:05.833746+0000","last_fullsized":"2024-09-21T06:36:05.833746+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:05.068816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834561+0000","last_change":"2024-09-21T06:34:30.634562+0000","last_active":"2024-09-21T06:36:05.834561+0000","last_peered":"2024-09-21T06:36:05.834561+0000","last_clean":"2024-09-21T06:36:05.834561+0000","last_became_active":"2024-09-21T06:34:30.634170+0000","last_became_peered":"2024-09-21T06:34:30.634170+0000","last_unstale":"2024-09-21T06:36:05.834561+0000","last_undegraded":"2024-09-21T06:36:05.834561+0000","last_fullsized":"2024-09-21T06:36:05.834561+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:15.243103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.234496+0000","last_change":"2024-09-21T06:34:32.645718+0000","last_active":"2024-09-21T06:36:34.234496+0000","last_peered":"2024-09-21T06:36:34.234496+0000","last_clean":"2024-09-21T06:36:34.234496+0000","last_became_active":"2024-09-21T06:34:32.645072+0000","last_became_peered":"2024-09-21T06:34:32.645072+0000","last_unstale":"2024-09-21T06:36:34.234496+0000","last_undegraded":"2024-09-21T06:36:34.234496+0000","last_fullsized":"2024-09-21T06:36:34.234496+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:54.109246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622307+0000","last_change":"2024-09-21T06:34:39.716947+0000","last_active":"2024-09-21T06:36:05.622307+0000","last_peered":"2024-09-21T06:36:05.622307+0000","last_clean":"2024-09-21T06:36:05.622307+0000","last_became_active":"2024-09-21T06:34:39.716540+0000","last_became_peered":"2024-09-21T06:34:39.716540+0000","last_unstale":"2024-09-21T06:36:05.622307+0000","last_undegraded":"2024-09-21T06:36:05.622307+0000","last_fullsized":"2024-09-21T06:36:05.622307+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:12.748444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834526+0000","last_change":"2024-09-21T06:34:30.636389+0000","last_active":"2024-09-21T06:36:05.834526+0000","last_peered":"2024-09-21T06:36:05.834526+0000","last_clean":"2024-09-21T06:36:05.834526+0000","last_became_active":"2024-09-21T06:34:30.636008+0000","last_became_peered":"2024-09-21T06:34:30.636008+0000","last_unstale":"2024-09-21T06:36:05.834526+0000","last_undegraded":"2024-09-21T06:36:05.834526+0000","last_fullsized":"2024-09-21T06:36:05.834526+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:45.476267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.219675+0000","last_change":"2024-09-21T06:34:32.635644+0000","last_active":"2024-09-21T06:36:34.219675+0000","last_peered":"2024-09-21T06:36:34.219675+0000","last_clean":"2024-09-21T06:36:34.219675+0000","last_became_active":"2024-09-21T06:34:32.635253+0000","last_became_peered":"2024-09-21T06:34:32.635253+0000","last_unstale":"2024-09-21T06:36:34.219675+0000","last_undegraded":"2024-09-21T06:36:34.219675+0000","last_fullsized":"2024-09-21T06:36:34.219675+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:34.713656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622327+0000","last_change":"2024-09-21T06:34:39.710382+0000","last_active":"2024-09-21T06:36:05.622327+0000","last_peered":"2024-09-21T06:36:05.622327+0000","last_clean":"2024-09-21T06:36:05.622327+0000","last_became_active":"2024-09-21T06:34:39.709884+0000","last_became_peered":"2024-09-21T06:34:39.709884+0000","last_unstale":"2024-09-21T06:36:05.622327+0000","last_undegraded":"2024-09-21T06:36:05.622327+0000","last_fullsized":"2024-09-21T06:36:05.622327+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:01.602366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.234789+0000","last_change":"2024-09-21T06:34:32.640025+0000","last_active":"2024-09-21T06:36:34.234789+0000","last_peered":"2024-09-21T06:36:34.234789+0000","last_clean":"2024-09-21T06:36:34.234789+0000","last_became_active":"2024-09-21T06:34:32.639648+0000","last_became_peered":"2024-09-21T06:34:32.639648+0000","last_unstale":"2024-09-21T06:36:34.234789+0000","last_undegraded":"2024-09-21T06:36:34.234789+0000","last_fullsized":"2024-09-21T06:36:34.234789+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:05.833065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":54,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.230047+0000","last_change":"2024-09-21T06:35:05.115344+0000","last_active":"2024-09-21T06:36:34.230047+0000","last_peered":"2024-09-21T06:36:34.230047+0000","last_clean":"2024-09-21T06:36:34.230047+0000","last_became_active":"2024-09-21T06:35:05.114678+0000","last_became_peered":"2024-09-21T06:35:05.114678+0000","last_unstale":"2024-09-21T06:36:34.230047+0000","last_undegraded":"2024-09-21T06:36:34.230047+0000","last_fullsized":"2024-09-21T06:36:34.230047+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:24.593145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":50,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.229653+0000","last_change":"2024-09-21T06:35:05.115301+0000","last_active":"2024-09-21T06:36:34.229653+0000","last_peered":"2024-09-21T06:36:34.229653+0000","last_clean":"2024-09-21T06:36:34.229653+0000","last_became_active":"2024-09-21T06:35:05.114706+0000","last_became_peered":"2024-09-21T06:35:05.114706+0000","last_unstale":"2024-09-21T06:36:34.229653+0000","last_undegraded":"2024-09-21T06:36:34.229653+0000","last_fullsized":"2024-09-21T06:36:34.229653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:09.513594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.241136+0000","last_change":"2024-09-21T06:34:32.634679+0000","last_active":"2024-09-21T06:36:34.241136+0000","last_peered":"2024-09-21T06:36:34.241136+0000","last_clean":"2024-09-21T06:36:34.241136+0000","last_became_active":"2024-09-21T06:34:32.634285+0000","last_became_peered":"2024-09-21T06:34:32.634285+0000","last_unstale":"2024-09-21T06:36:34.241136+0000","last_undegraded":"2024-09-21T06:36:34.241136+0000","last_fullsized":"2024-09-21T06:36:34.241136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:15.185453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.238589+0000","last_change":"2024-09-21T06:34:32.639037+0000","last_active":"2024-09-21T06:36:34.238589+0000","last_peered":"2024-09-21T06:36:34.238589+0000","last_clean":"2024-09-21T06:36:34.238589+0000","last_became_active":"2024-09-21T06:34:32.638689+0000","last_became_peered":"2024-09-21T06:34:32.638689+0000","last_unstale":"2024-09-21T06:36:34.238589+0000","last_undegraded":"2024-09-21T06:36:34.238589+0000","last_fullsized":"2024-09-21T06:36:34.238589+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:13.609191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.213709+0000","last_change":"2024-09-21T06:35:05.115051+0000","last_active":"2024-09-21T06:36:34.213709+0000","last_peered":"2024-09-21T06:36:34.213709+0000","last_clean":"2024-09-21T06:36:34.213709+0000","last_became_active":"2024-09-21T06:35:05.114602+0000","last_became_peered":"2024-09-21T06:35:05.114602+0000","last_unstale":"2024-09-21T06:36:34.213709+0000","last_undegraded":"2024-09-21T06:36:34.213709+0000","last_fullsized":"2024-09-21T06:36:34.213709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:48.778188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":50,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.233404+0000","last_change":"2024-09-21T06:34:32.637237+0000","last_active":"2024-09-21T06:36:34.233404+0000","last_peered":"2024-09-21T06:36:34.233404+0000","last_clean":"2024-09-21T06:36:34.233404+0000","last_became_active":"2024-09-21T06:34:32.634105+0000","last_became_peered":"2024-09-21T06:34:32.634105+0000","last_unstale":"2024-09-21T06:36:34.233404+0000","last_undegraded":"2024-09-21T06:36:34.233404+0000","last_fullsized":"2024-09-21T06:36:34.233404+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:52.530461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.238241+0000","last_change":"2024-09-21T06:34:32.624441+0000","last_active":"2024-09-21T06:36:34.238241+0000","last_peered":"2024-09-21T06:36:34.238241+0000","last_clean":"2024-09-21T06:36:34.238241+0000","last_became_active":"2024-09-21T06:34:32.623974+0000","last_became_peered":"2024-09-21T06:34:32.623974+0000","last_unstale":"2024-09-21T06:36:34.238241+0000","last_undegraded":"2024-09-21T06:36:34.238241+0000","last_fullsized":"2024-09-21T06:36:34.238241+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:12.378113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.228142+0000","last_change":"2024-09-21T06:34:32.633879+0000","last_active":"2024-09-21T06:36:34.228142+0000","last_peered":"2024-09-21T06:36:34.228142+0000","last_clean":"2024-09-21T06:36:34.228142+0000","last_became_active":"2024-09-21T06:34:32.631791+0000","last_became_peered":"2024-09-21T06:34:32.631791+0000","last_unstale":"2024-09-21T06:36:34.228142+0000","last_undegraded":"2024-09-21T06:36:34.228142+0000","last_fullsized":"2024-09-21T06:36:34.228142+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:46.246892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.238453+0000","last_change":"2024-09-21T06:34:32.639974+0000","last_active":"2024-09-21T06:36:34.238453+0000","last_peered":"2024-09-21T06:36:34.238453+0000","last_clean":"2024-09-21T06:36:34.238453+0000","last_became_active":"2024-09-21T06:34:32.639631+0000","last_became_peered":"2024-09-21T06:34:32.639631+0000","last_unstale":"2024-09-21T06:36:34.238453+0000","last_undegraded":"2024-09-21T06:36:34.238453+0000","last_fullsized":"2024-09-21T06:36:34.238453+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:50.069378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.243809+0000","last_change":"2024-09-21T06:34:32.645403+0000","last_active":"2024-09-21T06:36:34.243809+0000","last_peered":"2024-09-21T06:36:34.243809+0000","last_clean":"2024-09-21T06:36:34.243809+0000","last_became_active":"2024-09-21T06:34:32.644986+0000","last_became_peered":"2024-09-21T06:34:32.644986+0000","last_unstale":"2024-09-21T06:36:34.243809+0000","last_undegraded":"2024-09-21T06:36:34.243809+0000","last_fullsized":"2024-09-21T06:36:34.243809+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:43.640107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.237746+0000","last_change":"2024-09-21T06:34:32.639933+0000","last_active":"2024-09-21T06:36:34.237746+0000","last_peered":"2024-09-21T06:36:34.237746+0000","last_clean":"2024-09-21T06:36:34.237746+0000","last_became_active":"2024-09-21T06:34:32.639438+0000","last_became_peered":"2024-09-21T06:34:32.639438+0000","last_unstale":"2024-09-21T06:36:34.237746+0000","last_undegraded":"2024-09-21T06:36:34.237746+0000","last_fullsized":"2024-09-21T06:36:34.237746+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:18.860219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.226962+0000","last_change":"2024-09-21T06:34:32.648580+0000","last_active":"2024-09-21T06:36:34.226962+0000","last_peered":"2024-09-21T06:36:34.226962+0000","last_clean":"2024-09-21T06:36:34.226962+0000","last_became_active":"2024-09-21T06:34:32.648016+0000","last_became_peered":"2024-09-21T06:34:32.648016+0000","last_unstale":"2024-09-21T06:36:34.226962+0000","last_undegraded":"2024-09-21T06:36:34.226962+0000","last_fullsized":"2024-09-21T06:36:34.226962+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:52.858069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.240213+0000","last_change":"2024-09-21T06:34:32.633376+0000","last_active":"2024-09-21T06:36:34.240213+0000","last_peered":"2024-09-21T06:36:34.240213+0000","last_clean":"2024-09-21T06:36:34.240213+0000","last_became_active":"2024-09-21T06:34:32.631052+0000","last_became_peered":"2024-09-21T06:34:32.631052+0000","last_unstale":"2024-09-21T06:36:34.240213+0000","last_undegraded":"2024-09-21T06:36:34.240213+0000","last_fullsized":"2024-09-21T06:36:34.240213+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:46.229599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.244444+0000","last_change":"2024-09-21T06:34:32.636966+0000","last_active":"2024-09-21T06:36:34.244444+0000","last_peered":"2024-09-21T06:36:34.244444+0000","last_clean":"2024-09-21T06:36:34.244444+0000","last_became_active":"2024-09-21T06:34:32.636565+0000","last_became_peered":"2024-09-21T06:34:32.636565+0000","last_unstale":"2024-09-21T06:36:34.244444+0000","last_undegraded":"2024-09-21T06:36:34.244444+0000","last_fullsized":"2024-09-21T06:36:34.244444+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:36.233082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.246729+0000","last_change":"2024-09-21T06:34:32.646895+0000","last_active":"2024-09-21T06:36:34.246729+0000","last_peered":"2024-09-21T06:36:34.246729+0000","last_clean":"2024-09-21T06:36:34.246729+0000","last_became_active":"2024-09-21T06:34:32.646505+0000","last_became_peered":"2024-09-21T06:34:32.646505+0000","last_unstale":"2024-09-21T06:36:34.246729+0000","last_undegraded":"2024-09-21T06:36:34.246729+0000","last_fullsized":"2024-09-21T06:36:34.246729+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:43.441337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.246723+0000","last_change":"2024-09-21T06:34:32.645402+0000","last_active":"2024-09-21T06:36:34.246723+0000","last_peered":"2024-09-21T06:36:34.246723+0000","last_clean":"2024-09-21T06:36:34.246723+0000","last_became_active":"2024-09-21T06:34:32.645013+0000","last_became_peered":"2024-09-21T06:34:32.645013+0000","last_unstale":"2024-09-21T06:36:34.246723+0000","last_undegraded":"2024-09-21T06:36:34.246723+0000","last_fullsized":"2024-09-21T06:36:34.246723+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:33.475847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.240830+0000","last_change":"2024-09-21T06:34:32.643234+0000","last_active":"2024-09-21T06:36:34.240830+0000","last_peered":"2024-09-21T06:36:34.240830+0000","last_clean":"2024-09-21T06:36:34.240830+0000","last_became_active":"2024-09-21T06:34:32.642614+0000","last_became_peered":"2024-09-21T06:34:32.642614+0000","last_unstale":"2024-09-21T06:36:34.240830+0000","last_undegraded":"2024-09-21T06:36:34.240830+0000","last_fullsized":"2024-09-21T06:36:34.240830+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:50.903500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.242716+0000","last_change":"2024-09-21T06:34:32.641887+0000","last_active":"2024-09-21T06:36:34.242716+0000","last_peered":"2024-09-21T06:36:34.242716+0000","last_clean":"2024-09-21T06:36:34.242716+0000","last_became_active":"2024-09-21T06:34:32.641490+0000","last_became_peered":"2024-09-21T06:34:32.641490+0000","last_unstale":"2024-09-21T06:36:34.242716+0000","last_undegraded":"2024-09-21T06:36:34.242716+0000","last_fullsized":"2024-09-21T06:36:34.242716+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:34.550970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.234750+0000","last_change":"2024-09-21T06:34:32.641865+0000","last_active":"2024-09-21T06:36:34.234750+0000","last_peered":"2024-09-21T06:36:34.234750+0000","last_clean":"2024-09-21T06:36:34.234750+0000","last_became_active":"2024-09-21T06:34:32.641252+0000","last_became_peered":"2024-09-21T06:34:32.641252+0000","last_unstale":"2024-09-21T06:36:34.234750+0000","last_undegraded":"2024-09-21T06:36:34.234750+0000","last_fullsized":"2024-09-21T06:36:34.234750+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:44.459202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.236389+0000","last_change":"2024-09-21T06:34:32.641259+0000","last_active":"2024-09-21T06:36:34.236389+0000","last_peered":"2024-09-21T06:36:34.236389+0000","last_clean":"2024-09-21T06:36:34.236389+0000","last_became_active":"2024-09-21T06:34:32.640907+0000","last_became_peered":"2024-09-21T06:34:32.640907+0000","last_unstale":"2024-09-21T06:36:34.236389+0000","last_undegraded":"2024-09-21T06:36:34.236389+0000","last_fullsized":"2024-09-21T06:36:34.236389+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:59.966857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.246472+0000","last_change":"2024-09-21T06:34:32.646095+0000","last_active":"2024-09-21T06:36:34.246472+0000","last_peered":"2024-09-21T06:36:34.246472+0000","last_clean":"2024-09-21T06:36:34.246472+0000","last_became_active":"2024-09-21T06:34:32.645604+0000","last_became_peered":"2024-09-21T06:34:32.645604+0000","last_unstale":"2024-09-21T06:36:34.246472+0000","last_undegraded":"2024-09-21T06:36:34.246472+0000","last_fullsized":"2024-09-21T06:36:34.246472+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:30.872428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":50,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.235849+0000","last_change":"2024-09-21T06:34:32.649303+0000","last_active":"2024-09-21T06:36:34.235849+0000","last_peered":"2024-09-21T06:36:34.235849+0000","last_clean":"2024-09-21T06:36:34.235849+0000","last_became_active":"2024-09-21T06:34:32.648965+0000","last_became_peered":"2024-09-21T06:34:32.648965+0000","last_unstale":"2024-09-21T06:36:34.235849+0000","last_undegraded":"2024-09-21T06:36:34.235849+0000","last_fullsized":"2024-09-21T06:36:34.235849+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:51.335949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.247103+0000","last_change":"2024-09-21T06:34:32.637261+0000","last_active":"2024-09-21T06:36:34.247103+0000","last_peered":"2024-09-21T06:36:34.247103+0000","last_clean":"2024-09-21T06:36:34.247103+0000","last_became_active":"2024-09-21T06:34:32.636861+0000","last_became_peered":"2024-09-21T06:34:32.636861+0000","last_unstale":"2024-09-21T06:36:34.247103+0000","last_undegraded":"2024-09-21T06:36:34.247103+0000","last_fullsized":"2024-09-21T06:36:34.247103+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:55.271027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.237027+0000","last_change":"2024-09-21T06:34:32.635156+0000","last_active":"2024-09-21T06:36:34.237027+0000","last_peered":"2024-09-21T06:36:34.237027+0000","last_clean":"2024-09-21T06:36:34.237027+0000","last_became_active":"2024-09-21T06:34:32.634802+0000","last_became_peered":"2024-09-21T06:34:32.634802+0000","last_unstale":"2024-09-21T06:36:34.237027+0000","last_undegraded":"2024-09-21T06:36:34.237027+0000","last_fullsized":"2024-09-21T06:36:34.237027+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:16.139829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.228891+0000","last_change":"2024-09-21T06:34:32.637509+0000","last_active":"2024-09-21T06:36:34.228891+0000","last_peered":"2024-09-21T06:36:34.228891+0000","last_clean":"2024-09-21T06:36:34.228891+0000","last_became_active":"2024-09-21T06:34:32.637136+0000","last_became_peered":"2024-09-21T06:34:32.637136+0000","last_unstale":"2024-09-21T06:36:34.228891+0000","last_undegraded":"2024-09-21T06:36:34.228891+0000","last_fullsized":"2024-09-21T06:36:34.228891+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:48.088988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.233952+0000","last_change":"2024-09-21T06:35:05.114819+0000","last_active":"2024-09-21T06:36:34.233952+0000","last_peered":"2024-09-21T06:36:34.233952+0000","last_clean":"2024-09-21T06:36:34.233952+0000","last_became_active":"2024-09-21T06:35:05.114424+0000","last_became_peered":"2024-09-21T06:35:05.114424+0000","last_unstale":"2024-09-21T06:36:34.233952+0000","last_undegraded":"2024-09-21T06:36:34.233952+0000","last_fullsized":"2024-09-21T06:36:34.233952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:59.323643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.225869+0000","last_change":"2024-09-21T06:34:32.636103+0000","last_active":"2024-09-21T06:36:34.225869+0000","last_peered":"2024-09-21T06:36:34.225869+0000","last_clean":"2024-09-21T06:36:34.225869+0000","last_became_active":"2024-09-21T06:34:32.635728+0000","last_became_peered":"2024-09-21T06:34:32.635728+0000","last_unstale":"2024-09-21T06:36:34.225869+0000","last_undegraded":"2024-09-21T06:36:34.225869+0000","last_fullsized":"2024-09-21T06:36:34.225869+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:16.731402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.235424+0000","last_change":"2024-09-21T06:34:32.635654+0000","last_active":"2024-09-21T06:36:34.235424+0000","last_peered":"2024-09-21T06:36:34.235424+0000","last_clean":"2024-09-21T06:36:34.235424+0000","last_became_active":"2024-09-21T06:34:32.635265+0000","last_became_peered":"2024-09-21T06:34:32.635265+0000","last_unstale":"2024-09-21T06:36:34.235424+0000","last_undegraded":"2024-09-21T06:36:34.235424+0000","last_fullsized":"2024-09-21T06:36:34.235424+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:28.169625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.246376+0000","last_change":"2024-09-21T06:34:32.644640+0000","last_active":"2024-09-21T06:36:34.246376+0000","last_peered":"2024-09-21T06:36:34.246376+0000","last_clean":"2024-09-21T06:36:34.246376+0000","last_became_active":"2024-09-21T06:34:32.644334+0000","last_became_peered":"2024-09-21T06:34:32.644334+0000","last_unstale":"2024-09-21T06:36:34.246376+0000","last_undegraded":"2024-09-21T06:36:34.246376+0000","last_fullsized":"2024-09-21T06:36:34.246376+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:25.572178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.242912+0000","last_change":"2024-09-21T06:34:32.643705+0000","last_active":"2024-09-21T06:36:34.242912+0000","last_peered":"2024-09-21T06:36:34.242912+0000","last_clean":"2024-09-21T06:36:34.242912+0000","last_became_active":"2024-09-21T06:34:32.642705+0000","last_became_peered":"2024-09-21T06:34:32.642705+0000","last_unstale":"2024-09-21T06:36:34.242912+0000","last_undegraded":"2024-09-21T06:36:34.242912+0000","last_fullsized":"2024-09-21T06:36:34.242912+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:22.542559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"85'8","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.246617+0000","last_change":"2024-09-21T06:34:32.646539+0000","last_active":"2024-09-21T06:36:34.246617+0000","last_peered":"2024-09-21T06:36:34.246617+0000","last_clean":"2024-09-21T06:36:34.246617+0000","last_became_active":"2024-09-21T06:34:32.645992+0000","last_became_peered":"2024-09-21T06:34:32.645992+0000","last_unstale":"2024-09-21T06:36:34.246617+0000","last_undegraded":"2024-09-21T06:36:34.246617+0000","last_fullsized":"2024-09-21T06:36:34.246617+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:38.856001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69832,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":218,"num_read_kb":179,"num_write":90,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":209989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282016,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743466,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":485699,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939694,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":1141199,"data_compressed":65460,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103220,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":490377,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299449,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":486303,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495677,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":987136,"data_stored":1080745,"data_compressed":65460,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659203,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":481603,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855432,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":1076249,"data_compressed":65460,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018958,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":485721,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215186,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":933888,"data_stored":481707,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":550671,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":550637,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":929792,"data_stored":481693,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":69068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:36:53.161 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T06:36:53.420 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:36:53.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:53 smithi203 ceph-mon[33735]: pgmap v369: 129 pgs: 129 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:36:53.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:53 smithi203 ceph-mon[33735]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:36:53.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:53 smithi067 ceph-mon[31232]: pgmap v369: 129 pgs: 129 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:36:53.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:53 smithi067 ceph-mon[31232]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:36:53.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:53 smithi100 ceph-mon[33401]: pgmap v369: 129 pgs: 129 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:36:53.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:53 smithi100 ceph-mon[33401]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:36:53.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.906+0000 7f8657b2c640 1 -- 172.21.15.67:0/3020355800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86501094b0 msgr2=0x7f86501098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:53.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.906+0000 7f8657b2c640 1 --2- 172.21.15.67:0/3020355800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86501094b0 0x7f86501098c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8640009900 tx=0x7f864002a410 comp rx=0 tx=0).stop 2024-09-21T06:36:53.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.907+0000 7f8657b2c640 1 -- 172.21.15.67:0/3020355800 shutdown_connections 2024-09-21T06:36:53.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.907+0000 7f8657b2c640 1 --2- 172.21.15.67:0/3020355800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8650109e00 0x7f8650110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:53.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.907+0000 7f8657b2c640 1 --2- 172.21.15.67:0/3020355800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86501094b0 0x7f86501098c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:53.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.907+0000 7f8657b2c640 1 --2- 172.21.15.67:0/3020355800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8650103bc0 0x7f8650103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:53.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.907+0000 7f8657b2c640 1 -- 172.21.15.67:0/3020355800 >> 172.21.15.67:0/3020355800 conn(0x7f865007b910 msgr2=0x7f865007bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:53.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.907+0000 7f8657b2c640 1 -- 172.21.15.67:0/3020355800 shutdown_connections 2024-09-21T06:36:53.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.907+0000 7f8657b2c640 1 -- 172.21.15.67:0/3020355800 wait complete. 2024-09-21T06:36:53.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.907+0000 7f8657b2c640 1 Processor -- start 2024-09-21T06:36:53.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.907+0000 7f8657b2c640 1 -- start start 2024-09-21T06:36:53.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.908+0000 7f8657b2c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8650103bc0 0x7f865019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:53.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.908+0000 7f8657b2c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86501094b0 0x7f865019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:53.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.908+0000 7f8657b2c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8650109e00 0x7f865019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:53.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.908+0000 7f8657b2c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8650113990 con 0x7f8650109e00 2024-09-21T06:36:53.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.908+0000 7f8657b2c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8650113810 con 0x7f8650103bc0 2024-09-21T06:36:53.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.908+0000 7f8657b2c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8650113b10 con 0x7f86501094b0 2024-09-21T06:36:53.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.908+0000 7f86560c9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8650109e00 0x7f865019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:53.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.908+0000 7f86560c9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8650109e00 0x7f865019fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54672/0 (socket says 172.21.15.67:54672) 2024-09-21T06:36:53.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.908+0000 7f86560c9640 1 -- 172.21.15.67:0/2439157825 learned_addr learned my addr 172.21.15.67:0/2439157825 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:53.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.908+0000 7f86550c7640 1 --2- 172.21.15.67:0/2439157825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86501094b0 0x7f865019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:53.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.908+0000 7f86558c8640 1 --2- 172.21.15.67:0/2439157825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8650103bc0 0x7f865019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:53.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.908+0000 7f86560c9640 1 -- 172.21.15.67:0/2439157825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86501094b0 msgr2=0x7f865019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:53.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.908+0000 7f86560c9640 1 --2- 172.21.15.67:0/2439157825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86501094b0 0x7f865019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:53.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.908+0000 7f86560c9640 1 -- 172.21.15.67:0/2439157825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8650103bc0 msgr2=0x7f865019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:53.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.908+0000 7f86560c9640 1 --2- 172.21.15.67:0/2439157825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8650103bc0 0x7f865019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:53.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.908+0000 7f86560c9640 1 -- 172.21.15.67:0/2439157825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86501a03d0 con 0x7f8650109e00 2024-09-21T06:36:53.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.908+0000 7f86558c8640 1 --2- 172.21.15.67:0/2439157825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8650103bc0 0x7f865019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:36:53.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.908+0000 7f86560c9640 1 --2- 172.21.15.67:0/2439157825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8650109e00 0x7f865019fca0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8648009fd0 tx=0x7f864800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:53.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.908+0000 7f8646ffd640 1 -- 172.21.15.67:0/2439157825 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8648017070 con 0x7f8650109e00 2024-09-21T06:36:53.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.908+0000 7f8646ffd640 1 -- 172.21.15.67:0/2439157825 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8648003e00 con 0x7f8650109e00 2024-09-21T06:36:53.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.909+0000 7f8646ffd640 1 -- 172.21.15.67:0/2439157825 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8648010460 con 0x7f8650109e00 2024-09-21T06:36:53.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.909+0000 7f8657b2c640 1 -- 172.21.15.67:0/2439157825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86501a06c0 con 0x7f8650109e00 2024-09-21T06:36:53.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.909+0000 7f8657b2c640 1 -- 172.21.15.67:0/2439157825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86501a6e20 con 0x7f8650109e00 2024-09-21T06:36:53.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.910+0000 7f8657b2c640 1 -- 172.21.15.67:0/2439157825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8650103fd0 con 0x7f8650109e00 2024-09-21T06:36:53.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.911+0000 7f8646ffd640 1 -- 172.21.15.67:0/2439157825 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f8648003970 con 0x7f8650109e00 2024-09-21T06:36:53.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.912+0000 7f8646ffd640 1 --2- 172.21.15.67:0/2439157825 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f863c0b6590 0x7f863c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:53.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.912+0000 7f86558c8640 1 --2- 172.21.15.67:0/2439157825 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f863c0b6590 0x7f863c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:53.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.913+0000 7f8646ffd640 1 -- 172.21.15.67:0/2439157825 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f86480e78b0 con 0x7f8650109e00 2024-09-21T06:36:53.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.913+0000 7f86558c8640 1 --2- 172.21.15.67:0/2439157825 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f863c0b6590 0x7f863c0b8aa0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f864c006fd0 tx=0x7f864c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:53.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:53.917+0000 7f8646ffd640 1 -- 172.21.15.67:0/2439157825 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f86480add70 con 0x7f8650109e00 2024-09-21T06:36:54.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:54.103+0000 7f8657b2c640 1 -- 172.21.15.67:0/2439157825 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8650063800 con 0x7f863c0b6590 2024-09-21T06:36:54.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:54.109+0000 7f8646ffd640 1 -- 172.21.15.67:0/2439157825 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340645 (secure 0 0 0) 0x7f8650063800 con 0x7f863c0b6590 2024-09-21T06:36:54.115 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:36:54.116 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-09-21T06:36:54.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:54.113+0000 7f8657b2c640 1 -- 172.21.15.67:0/2439157825 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f863c0b6590 msgr2=0x7f863c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:54.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:54.113+0000 7f8657b2c640 1 --2- 172.21.15.67:0/2439157825 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f863c0b6590 0x7f863c0b8aa0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f864c006fd0 tx=0x7f864c008040 comp rx=0 tx=0).stop 2024-09-21T06:36:54.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:54.113+0000 7f8657b2c640 1 -- 172.21.15.67:0/2439157825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8650109e00 msgr2=0x7f865019fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:54.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:54.113+0000 7f8657b2c640 1 --2- 172.21.15.67:0/2439157825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8650109e00 0x7f865019fca0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8648009fd0 tx=0x7f864800b260 comp rx=0 tx=0).stop 2024-09-21T06:36:54.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:54.113+0000 7f8657b2c640 1 -- 172.21.15.67:0/2439157825 shutdown_connections 2024-09-21T06:36:54.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:54.113+0000 7f8657b2c640 1 --2- 172.21.15.67:0/2439157825 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f863c0b6590 0x7f863c0b8aa0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:54.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:54.113+0000 7f8657b2c640 1 --2- 172.21.15.67:0/2439157825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8650109e00 0x7f865019fca0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:54.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:54.113+0000 7f8657b2c640 1 --2- 172.21.15.67:0/2439157825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86501094b0 0x7f865019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:54.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:54.113+0000 7f8657b2c640 1 --2- 172.21.15.67:0/2439157825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8650103bc0 0x7f865019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:54.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:54.113+0000 7f8657b2c640 1 -- 172.21.15.67:0/2439157825 >> 172.21.15.67:0/2439157825 conn(0x7f865007b910 msgr2=0x7f8650107660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:54.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:54.113+0000 7f8657b2c640 1 -- 172.21.15.67:0/2439157825 shutdown_connections 2024-09-21T06:36:54.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:54.113+0000 7f8657b2c640 1 -- 172.21.15.67:0/2439157825 wait complete. 2024-09-21T06:36:54.861 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":370,"stamp":"2024-09-21T06:36:53.882568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660434,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":294,"num_read_kb":243,"num_write":215,"num_write_kb":2251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":669,"ondisk_log_size":669,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368880,"kb_used_data":11312,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572944,"statfs":{"total":1151940427776,"available":1151562694656,"internally_reserved":0,"allocated":11583488,"data_stored":7792604,"data_compressed":462771,"data_compressed_allocated":626688,"data_compressed_original":2752512,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003853"},"pg_stats":[{"pgid":"2.1f","version":"75'4","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621812+0000","last_change":"2024-09-21T06:34:30.633366+0000","last_active":"2024-09-21T06:36:05.621812+0000","last_peered":"2024-09-21T06:36:05.621812+0000","last_clean":"2024-09-21T06:36:05.621812+0000","last_became_active":"2024-09-21T06:34:30.632929+0000","last_became_peered":"2024-09-21T06:34:30.632929+0000","last_unstale":"2024-09-21T06:36:05.621812+0000","last_undegraded":"2024-09-21T06:36:05.621812+0000","last_fullsized":"2024-09-21T06:36:05.621812+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:47.722461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.238429+0000","last_change":"2024-09-21T06:34:32.632989+0000","last_active":"2024-09-21T06:36:34.238429+0000","last_peered":"2024-09-21T06:36:34.238429+0000","last_clean":"2024-09-21T06:36:34.238429+0000","last_became_active":"2024-09-21T06:34:32.632024+0000","last_became_peered":"2024-09-21T06:34:32.632024+0000","last_unstale":"2024-09-21T06:36:34.238429+0000","last_undegraded":"2024-09-21T06:36:34.238429+0000","last_fullsized":"2024-09-21T06:36:34.238429+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:13.669613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621767+0000","last_change":"2024-09-21T06:35:05.274434+0000","last_active":"2024-09-21T06:36:05.621767+0000","last_peered":"2024-09-21T06:36:05.621767+0000","last_clean":"2024-09-21T06:36:05.621767+0000","last_became_active":"2024-09-21T06:35:05.273833+0000","last_became_peered":"2024-09-21T06:35:05.273833+0000","last_unstale":"2024-09-21T06:36:05.621767+0000","last_undegraded":"2024-09-21T06:36:05.621767+0000","last_fullsized":"2024-09-21T06:36:05.621767+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:48.847861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835213+0000","last_change":"2024-09-21T06:34:30.946309+0000","last_active":"2024-09-21T06:36:05.835213+0000","last_peered":"2024-09-21T06:36:05.835213+0000","last_clean":"2024-09-21T06:36:05.835213+0000","last_became_active":"2024-09-21T06:34:30.945913+0000","last_became_peered":"2024-09-21T06:34:30.945913+0000","last_unstale":"2024-09-21T06:36:05.835213+0000","last_undegraded":"2024-09-21T06:36:05.835213+0000","last_fullsized":"2024-09-21T06:36:05.835213+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:21.676524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.246270+0000","last_change":"2024-09-21T06:34:32.637636+0000","last_active":"2024-09-21T06:36:34.246270+0000","last_peered":"2024-09-21T06:36:34.246270+0000","last_clean":"2024-09-21T06:36:34.246270+0000","last_became_active":"2024-09-21T06:34:32.636534+0000","last_became_peered":"2024-09-21T06:34:32.636534+0000","last_unstale":"2024-09-21T06:36:34.246270+0000","last_undegraded":"2024-09-21T06:36:34.246270+0000","last_fullsized":"2024-09-21T06:36:34.246270+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:05.802118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771498+0000","last_change":"2024-09-21T06:35:05.274419+0000","last_active":"2024-09-21T06:36:05.771498+0000","last_peered":"2024-09-21T06:36:05.771498+0000","last_clean":"2024-09-21T06:36:05.771498+0000","last_became_active":"2024-09-21T06:35:05.273865+0000","last_became_peered":"2024-09-21T06:35:05.273865+0000","last_unstale":"2024-09-21T06:36:05.771498+0000","last_undegraded":"2024-09-21T06:36:05.771498+0000","last_fullsized":"2024-09-21T06:36:05.771498+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:04.490355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.631709+0000","last_change":"2024-09-21T06:36:05.631098+0000","last_active":"2024-09-21T06:36:05.631709+0000","last_peered":"2024-09-21T06:36:05.631709+0000","last_clean":"2024-09-21T06:36:05.631709+0000","last_became_active":"2024-09-21T06:36:05.627521+0000","last_became_peered":"2024-09-21T06:36:05.627521+0000","last_unstale":"2024-09-21T06:36:05.631709+0000","last_undegraded":"2024-09-21T06:36:05.631709+0000","last_fullsized":"2024-09-21T06:36:05.631709+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:02.467758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.247327+0000","last_change":"2024-09-21T06:34:32.650303+0000","last_active":"2024-09-21T06:36:34.247327+0000","last_peered":"2024-09-21T06:36:34.247327+0000","last_clean":"2024-09-21T06:36:34.247327+0000","last_became_active":"2024-09-21T06:34:32.647189+0000","last_became_peered":"2024-09-21T06:34:32.647189+0000","last_unstale":"2024-09-21T06:36:34.247327+0000","last_undegraded":"2024-09-21T06:36:34.247327+0000","last_fullsized":"2024-09-21T06:36:34.247327+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:14.889384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771714+0000","last_change":"2024-09-21T06:34:39.713139+0000","last_active":"2024-09-21T06:36:05.771714+0000","last_peered":"2024-09-21T06:36:05.771714+0000","last_clean":"2024-09-21T06:36:05.771714+0000","last_became_active":"2024-09-21T06:34:39.712379+0000","last_became_peered":"2024-09-21T06:34:39.712379+0000","last_unstale":"2024-09-21T06:36:05.771714+0000","last_undegraded":"2024-09-21T06:36:05.771714+0000","last_fullsized":"2024-09-21T06:36:05.771714+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:53.100529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.625272+0000","last_change":"2024-09-21T06:34:30.633850+0000","last_active":"2024-09-21T06:36:05.625272+0000","last_peered":"2024-09-21T06:36:05.625272+0000","last_clean":"2024-09-21T06:36:05.625272+0000","last_became_active":"2024-09-21T06:34:30.633468+0000","last_became_peered":"2024-09-21T06:34:30.633468+0000","last_unstale":"2024-09-21T06:36:05.625272+0000","last_undegraded":"2024-09-21T06:36:05.625272+0000","last_fullsized":"2024-09-21T06:36:05.625272+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:59.095803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":56,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.247919+0000","last_change":"2024-09-21T06:35:05.005240+0000","last_active":"2024-09-21T06:36:34.247919+0000","last_peered":"2024-09-21T06:36:34.247919+0000","last_clean":"2024-09-21T06:36:34.247919+0000","last_became_active":"2024-09-21T06:35:05.004849+0000","last_became_peered":"2024-09-21T06:35:05.004849+0000","last_unstale":"2024-09-21T06:36:34.247919+0000","last_undegraded":"2024-09-21T06:36:34.247919+0000","last_fullsized":"2024-09-21T06:36:34.247919+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:28.849604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771556+0000","last_change":"2024-09-21T06:35:05.274437+0000","last_active":"2024-09-21T06:36:05.771556+0000","last_peered":"2024-09-21T06:36:05.771556+0000","last_clean":"2024-09-21T06:36:05.771556+0000","last_became_active":"2024-09-21T06:35:05.273865+0000","last_became_peered":"2024-09-21T06:35:05.273865+0000","last_unstale":"2024-09-21T06:36:05.771556+0000","last_undegraded":"2024-09-21T06:36:05.771556+0000","last_fullsized":"2024-09-21T06:36:05.771556+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:43.339504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.246814+0000","last_change":"2024-09-21T06:34:32.645630+0000","last_active":"2024-09-21T06:36:34.246814+0000","last_peered":"2024-09-21T06:36:34.246814+0000","last_clean":"2024-09-21T06:36:34.246814+0000","last_became_active":"2024-09-21T06:34:32.645237+0000","last_became_peered":"2024-09-21T06:34:32.645237+0000","last_unstale":"2024-09-21T06:36:34.246814+0000","last_undegraded":"2024-09-21T06:36:34.246814+0000","last_fullsized":"2024-09-21T06:36:34.246814+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:25.064318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.633983+0000","last_change":"2024-09-21T06:36:05.633484+0000","last_active":"2024-09-21T06:36:05.633983+0000","last_peered":"2024-09-21T06:36:05.633983+0000","last_clean":"2024-09-21T06:36:05.633983+0000","last_became_active":"2024-09-21T06:36:05.627637+0000","last_became_peered":"2024-09-21T06:36:05.627637+0000","last_unstale":"2024-09-21T06:36:05.633983+0000","last_undegraded":"2024-09-21T06:36:05.633983+0000","last_fullsized":"2024-09-21T06:36:05.633983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:50:00.626538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771936+0000","last_change":"2024-09-21T06:34:39.714177+0000","last_active":"2024-09-21T06:36:05.771936+0000","last_peered":"2024-09-21T06:36:05.771936+0000","last_clean":"2024-09-21T06:36:05.771936+0000","last_became_active":"2024-09-21T06:34:39.713713+0000","last_became_peered":"2024-09-21T06:34:39.713713+0000","last_unstale":"2024-09-21T06:36:05.771936+0000","last_undegraded":"2024-09-21T06:36:05.771936+0000","last_fullsized":"2024-09-21T06:36:05.771936+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:07.042240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.234553+0000","last_change":"2024-09-21T06:34:32.636297+0000","last_active":"2024-09-21T06:36:34.234553+0000","last_peered":"2024-09-21T06:36:34.234553+0000","last_clean":"2024-09-21T06:36:34.234553+0000","last_became_active":"2024-09-21T06:34:32.635930+0000","last_became_peered":"2024-09-21T06:34:32.635930+0000","last_unstale":"2024-09-21T06:36:34.234553+0000","last_undegraded":"2024-09-21T06:36:34.234553+0000","last_fullsized":"2024-09-21T06:36:34.234553+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:43.091682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621975+0000","last_change":"2024-09-21T06:34:30.631405+0000","last_active":"2024-09-21T06:36:05.621975+0000","last_peered":"2024-09-21T06:36:05.621975+0000","last_clean":"2024-09-21T06:36:05.621975+0000","last_became_active":"2024-09-21T06:34:30.630512+0000","last_became_peered":"2024-09-21T06:34:30.630512+0000","last_unstale":"2024-09-21T06:36:05.621975+0000","last_undegraded":"2024-09-21T06:36:05.621975+0000","last_fullsized":"2024-09-21T06:36:05.621975+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:43.412695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"83'1","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771897+0000","last_change":"2024-09-21T06:34:39.710573+0000","last_active":"2024-09-21T06:36:05.771897+0000","last_peered":"2024-09-21T06:36:05.771897+0000","last_clean":"2024-09-21T06:36:05.771897+0000","last_became_active":"2024-09-21T06:34:39.708969+0000","last_became_peered":"2024-09-21T06:34:39.708969+0000","last_unstale":"2024-09-21T06:36:05.771897+0000","last_undegraded":"2024-09-21T06:36:05.771897+0000","last_fullsized":"2024-09-21T06:36:05.771897+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:34.847587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.234683+0000","last_change":"2024-09-21T06:34:32.641085+0000","last_active":"2024-09-21T06:36:34.234683+0000","last_peered":"2024-09-21T06:36:34.234683+0000","last_clean":"2024-09-21T06:36:34.234683+0000","last_became_active":"2024-09-21T06:34:32.640723+0000","last_became_peered":"2024-09-21T06:34:32.640723+0000","last_unstale":"2024-09-21T06:36:34.234683+0000","last_undegraded":"2024-09-21T06:36:34.234683+0000","last_fullsized":"2024-09-21T06:36:34.234683+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:21.410262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834947+0000","last_change":"2024-09-21T06:34:30.626719+0000","last_active":"2024-09-21T06:36:05.834947+0000","last_peered":"2024-09-21T06:36:05.834947+0000","last_clean":"2024-09-21T06:36:05.834947+0000","last_became_active":"2024-09-21T06:34:30.626312+0000","last_became_peered":"2024-09-21T06:34:30.626312+0000","last_unstale":"2024-09-21T06:36:05.834947+0000","last_undegraded":"2024-09-21T06:36:05.834947+0000","last_fullsized":"2024-09-21T06:36:05.834947+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:53.938914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621617+0000","last_change":"2024-09-21T06:34:39.711172+0000","last_active":"2024-09-21T06:36:05.621617+0000","last_peered":"2024-09-21T06:36:05.621617+0000","last_clean":"2024-09-21T06:36:05.621617+0000","last_became_active":"2024-09-21T06:34:39.708424+0000","last_became_peered":"2024-09-21T06:34:39.708424+0000","last_unstale":"2024-09-21T06:36:05.621617+0000","last_undegraded":"2024-09-21T06:36:05.621617+0000","last_fullsized":"2024-09-21T06:36:05.621617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:43.918355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.209527+0000","last_change":"2024-09-21T06:34:32.641463+0000","last_active":"2024-09-21T06:36:34.209527+0000","last_peered":"2024-09-21T06:36:34.209527+0000","last_clean":"2024-09-21T06:36:34.209527+0000","last_became_active":"2024-09-21T06:34:32.641086+0000","last_became_peered":"2024-09-21T06:34:32.641086+0000","last_unstale":"2024-09-21T06:36:34.209527+0000","last_undegraded":"2024-09-21T06:36:34.209527+0000","last_fullsized":"2024-09-21T06:36:34.209527+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:58.991374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835725+0000","last_change":"2024-09-21T06:34:30.946698+0000","last_active":"2024-09-21T06:36:05.835725+0000","last_peered":"2024-09-21T06:36:05.835725+0000","last_clean":"2024-09-21T06:36:05.835725+0000","last_became_active":"2024-09-21T06:34:30.946325+0000","last_became_peered":"2024-09-21T06:34:30.946325+0000","last_unstale":"2024-09-21T06:36:05.835725+0000","last_undegraded":"2024-09-21T06:36:05.835725+0000","last_fullsized":"2024-09-21T06:36:05.835725+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:22.639265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.772211+0000","last_change":"2024-09-21T06:34:39.712429+0000","last_active":"2024-09-21T06:36:05.772211+0000","last_peered":"2024-09-21T06:36:05.772211+0000","last_clean":"2024-09-21T06:36:05.772211+0000","last_became_active":"2024-09-21T06:34:39.711823+0000","last_became_peered":"2024-09-21T06:34:39.711823+0000","last_unstale":"2024-09-21T06:36:05.772211+0000","last_undegraded":"2024-09-21T06:36:05.772211+0000","last_fullsized":"2024-09-21T06:36:05.772211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:03:49.513991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.243704+0000","last_change":"2024-09-21T06:35:05.274326+0000","last_active":"2024-09-21T06:36:34.243704+0000","last_peered":"2024-09-21T06:36:34.243704+0000","last_clean":"2024-09-21T06:36:34.243704+0000","last_became_active":"2024-09-21T06:35:05.273884+0000","last_became_peered":"2024-09-21T06:35:05.273884+0000","last_unstale":"2024-09-21T06:36:34.243704+0000","last_undegraded":"2024-09-21T06:36:34.243704+0000","last_fullsized":"2024-09-21T06:36:34.243704+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:03.219765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.772393+0000","last_change":"2024-09-21T06:34:30.676794+0000","last_active":"2024-09-21T06:36:05.772393+0000","last_peered":"2024-09-21T06:36:05.772393+0000","last_clean":"2024-09-21T06:36:05.772393+0000","last_became_active":"2024-09-21T06:34:30.676400+0000","last_became_peered":"2024-09-21T06:34:30.676400+0000","last_unstale":"2024-09-21T06:36:05.772393+0000","last_undegraded":"2024-09-21T06:36:05.772393+0000","last_fullsized":"2024-09-21T06:36:05.772393+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:23.681363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622160+0000","last_change":"2024-09-21T06:34:39.708980+0000","last_active":"2024-09-21T06:36:05.622160+0000","last_peered":"2024-09-21T06:36:05.622160+0000","last_clean":"2024-09-21T06:36:05.622160+0000","last_became_active":"2024-09-21T06:34:39.707748+0000","last_became_peered":"2024-09-21T06:34:39.707748+0000","last_unstale":"2024-09-21T06:36:05.622160+0000","last_undegraded":"2024-09-21T06:36:05.622160+0000","last_fullsized":"2024-09-21T06:36:05.622160+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:16.812226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.240400+0000","last_change":"2024-09-21T06:34:32.638337+0000","last_active":"2024-09-21T06:36:34.240400+0000","last_peered":"2024-09-21T06:36:34.240400+0000","last_clean":"2024-09-21T06:36:34.240400+0000","last_became_active":"2024-09-21T06:34:32.638013+0000","last_became_peered":"2024-09-21T06:34:32.638013+0000","last_unstale":"2024-09-21T06:36:34.240400+0000","last_undegraded":"2024-09-21T06:36:34.240400+0000","last_fullsized":"2024-09-21T06:36:34.240400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:26.321190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'10","reported_seq":50,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.673339+0000","last_change":"2024-09-21T06:34:30.631086+0000","last_active":"2024-09-21T06:36:34.673339+0000","last_peered":"2024-09-21T06:36:34.673339+0000","last_clean":"2024-09-21T06:36:34.673339+0000","last_became_active":"2024-09-21T06:34:30.630706+0000","last_became_peered":"2024-09-21T06:34:30.630706+0000","last_unstale":"2024-09-21T06:36:34.673339+0000","last_undegraded":"2024-09-21T06:36:34.673339+0000","last_fullsized":"2024-09-21T06:36:34.673339+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:30.260359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834003+0000","last_change":"2024-09-21T06:34:39.709097+0000","last_active":"2024-09-21T06:36:05.834003+0000","last_peered":"2024-09-21T06:36:05.834003+0000","last_clean":"2024-09-21T06:36:05.834003+0000","last_became_active":"2024-09-21T06:34:39.708609+0000","last_became_peered":"2024-09-21T06:34:39.708609+0000","last_unstale":"2024-09-21T06:36:05.834003+0000","last_undegraded":"2024-09-21T06:36:05.834003+0000","last_fullsized":"2024-09-21T06:36:05.834003+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:48.842174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.229558+0000","last_change":"2024-09-21T06:34:32.636816+0000","last_active":"2024-09-21T06:36:34.229558+0000","last_peered":"2024-09-21T06:36:34.229558+0000","last_clean":"2024-09-21T06:36:34.229558+0000","last_became_active":"2024-09-21T06:34:32.636429+0000","last_became_peered":"2024-09-21T06:34:32.636429+0000","last_unstale":"2024-09-21T06:36:34.229558+0000","last_undegraded":"2024-09-21T06:36:34.229558+0000","last_fullsized":"2024-09-21T06:36:34.229558+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:49.010491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.625629+0000","last_change":"2024-09-21T06:34:30.947481+0000","last_active":"2024-09-21T06:36:05.625629+0000","last_peered":"2024-09-21T06:36:05.625629+0000","last_clean":"2024-09-21T06:36:05.625629+0000","last_became_active":"2024-09-21T06:34:30.947040+0000","last_became_peered":"2024-09-21T06:34:30.947040+0000","last_unstale":"2024-09-21T06:36:05.625629+0000","last_undegraded":"2024-09-21T06:36:05.625629+0000","last_fullsized":"2024-09-21T06:36:05.625629+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:52.813476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624972+0000","last_change":"2024-09-21T06:34:39.708674+0000","last_active":"2024-09-21T06:36:05.624972+0000","last_peered":"2024-09-21T06:36:05.624972+0000","last_clean":"2024-09-21T06:36:05.624972+0000","last_became_active":"2024-09-21T06:34:39.708176+0000","last_became_peered":"2024-09-21T06:34:39.708176+0000","last_unstale":"2024-09-21T06:36:05.624972+0000","last_undegraded":"2024-09-21T06:36:05.624972+0000","last_fullsized":"2024-09-21T06:36:05.624972+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:57.070792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"85'4","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.239723+0000","last_change":"2024-09-21T06:34:32.636381+0000","last_active":"2024-09-21T06:36:34.239723+0000","last_peered":"2024-09-21T06:36:34.239723+0000","last_clean":"2024-09-21T06:36:34.239723+0000","last_became_active":"2024-09-21T06:34:32.635937+0000","last_became_peered":"2024-09-21T06:34:32.635937+0000","last_unstale":"2024-09-21T06:36:34.239723+0000","last_undegraded":"2024-09-21T06:36:34.239723+0000","last_fullsized":"2024-09-21T06:36:34.239723+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:52.870676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621524+0000","last_change":"2024-09-21T06:34:30.947160+0000","last_active":"2024-09-21T06:36:05.621524+0000","last_peered":"2024-09-21T06:36:05.621524+0000","last_clean":"2024-09-21T06:36:05.621524+0000","last_became_active":"2024-09-21T06:34:30.946736+0000","last_became_peered":"2024-09-21T06:34:30.946736+0000","last_unstale":"2024-09-21T06:36:05.621524+0000","last_undegraded":"2024-09-21T06:36:05.621524+0000","last_fullsized":"2024-09-21T06:36:05.621524+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:47.425481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.625820+0000","last_change":"2024-09-21T06:35:05.006032+0000","last_active":"2024-09-21T06:36:05.625820+0000","last_peered":"2024-09-21T06:36:05.625820+0000","last_clean":"2024-09-21T06:36:05.625820+0000","last_became_active":"2024-09-21T06:35:05.005429+0000","last_became_peered":"2024-09-21T06:35:05.005429+0000","last_unstale":"2024-09-21T06:36:05.625820+0000","last_undegraded":"2024-09-21T06:36:05.625820+0000","last_fullsized":"2024-09-21T06:36:05.625820+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:04.138097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.238876+0000","last_change":"2024-09-21T06:34:32.648256+0000","last_active":"2024-09-21T06:36:34.238876+0000","last_peered":"2024-09-21T06:36:34.238876+0000","last_clean":"2024-09-21T06:36:34.238876+0000","last_became_active":"2024-09-21T06:34:32.647881+0000","last_became_peered":"2024-09-21T06:34:32.647881+0000","last_unstale":"2024-09-21T06:36:34.238876+0000","last_undegraded":"2024-09-21T06:36:34.238876+0000","last_fullsized":"2024-09-21T06:36:34.238876+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:14.590834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622945+0000","last_change":"2024-09-21T06:34:30.629918+0000","last_active":"2024-09-21T06:36:05.622945+0000","last_peered":"2024-09-21T06:36:05.622945+0000","last_clean":"2024-09-21T06:36:05.622945+0000","last_became_active":"2024-09-21T06:34:30.628785+0000","last_became_peered":"2024-09-21T06:34:30.628785+0000","last_unstale":"2024-09-21T06:36:05.622945+0000","last_undegraded":"2024-09-21T06:36:05.622945+0000","last_fullsized":"2024-09-21T06:36:05.622945+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:05.333406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834425+0000","last_change":"2024-09-21T06:34:39.710073+0000","last_active":"2024-09-21T06:36:05.834425+0000","last_peered":"2024-09-21T06:36:05.834425+0000","last_clean":"2024-09-21T06:36:05.834425+0000","last_became_active":"2024-09-21T06:34:39.708986+0000","last_became_peered":"2024-09-21T06:34:39.708986+0000","last_unstale":"2024-09-21T06:36:05.834425+0000","last_undegraded":"2024-09-21T06:36:05.834425+0000","last_fullsized":"2024-09-21T06:36:05.834425+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:46.379902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.234328+0000","last_change":"2024-09-21T06:34:32.644224+0000","last_active":"2024-09-21T06:36:34.234328+0000","last_peered":"2024-09-21T06:36:34.234328+0000","last_clean":"2024-09-21T06:36:34.234328+0000","last_became_active":"2024-09-21T06:34:32.643850+0000","last_became_peered":"2024-09-21T06:34:32.643850+0000","last_unstale":"2024-09-21T06:36:34.234328+0000","last_undegraded":"2024-09-21T06:36:34.234328+0000","last_fullsized":"2024-09-21T06:36:34.234328+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:24:33.188537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.833952+0000","last_change":"2024-09-21T06:34:30.946334+0000","last_active":"2024-09-21T06:36:05.833952+0000","last_peered":"2024-09-21T06:36:05.833952+0000","last_clean":"2024-09-21T06:36:05.833952+0000","last_became_active":"2024-09-21T06:34:30.945908+0000","last_became_peered":"2024-09-21T06:34:30.945908+0000","last_unstale":"2024-09-21T06:36:05.833952+0000","last_undegraded":"2024-09-21T06:36:05.833952+0000","last_fullsized":"2024-09-21T06:36:05.833952+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:46.746527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834642+0000","last_change":"2024-09-21T06:34:39.707909+0000","last_active":"2024-09-21T06:36:05.834642+0000","last_peered":"2024-09-21T06:36:05.834642+0000","last_clean":"2024-09-21T06:36:05.834642+0000","last_became_active":"2024-09-21T06:34:39.707492+0000","last_became_peered":"2024-09-21T06:34:39.707492+0000","last_unstale":"2024-09-21T06:36:05.834642+0000","last_undegraded":"2024-09-21T06:36:05.834642+0000","last_fullsized":"2024-09-21T06:36:05.834642+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:56.201212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.227669+0000","last_change":"2024-09-21T06:34:32.635870+0000","last_active":"2024-09-21T06:36:34.227669+0000","last_peered":"2024-09-21T06:36:34.227669+0000","last_clean":"2024-09-21T06:36:34.227669+0000","last_became_active":"2024-09-21T06:34:32.635252+0000","last_became_peered":"2024-09-21T06:34:32.635252+0000","last_unstale":"2024-09-21T06:36:34.227669+0000","last_undegraded":"2024-09-21T06:36:34.227669+0000","last_fullsized":"2024-09-21T06:36:34.227669+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:10.613033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622347+0000","last_change":"2024-09-21T06:34:30.634360+0000","last_active":"2024-09-21T06:36:05.622347+0000","last_peered":"2024-09-21T06:36:05.622347+0000","last_clean":"2024-09-21T06:36:05.622347+0000","last_became_active":"2024-09-21T06:34:30.633845+0000","last_became_peered":"2024-09-21T06:34:30.633845+0000","last_unstale":"2024-09-21T06:36:05.622347+0000","last_undegraded":"2024-09-21T06:36:05.622347+0000","last_fullsized":"2024-09-21T06:36:05.622347+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:34:53.388670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834895+0000","last_change":"2024-09-21T06:34:39.719620+0000","last_active":"2024-09-21T06:36:05.834895+0000","last_peered":"2024-09-21T06:36:05.834895+0000","last_clean":"2024-09-21T06:36:05.834895+0000","last_became_active":"2024-09-21T06:34:39.718786+0000","last_became_peered":"2024-09-21T06:34:39.718786+0000","last_unstale":"2024-09-21T06:36:05.834895+0000","last_undegraded":"2024-09-21T06:36:05.834895+0000","last_fullsized":"2024-09-21T06:36:05.834895+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:43.524932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.236491+0000","last_change":"2024-09-21T06:34:32.634970+0000","last_active":"2024-09-21T06:36:34.236491+0000","last_peered":"2024-09-21T06:36:34.236491+0000","last_clean":"2024-09-21T06:36:34.236491+0000","last_became_active":"2024-09-21T06:34:32.634611+0000","last_became_peered":"2024-09-21T06:34:32.634611+0000","last_unstale":"2024-09-21T06:36:34.236491+0000","last_undegraded":"2024-09-21T06:36:34.236491+0000","last_fullsized":"2024-09-21T06:36:34.236491+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:37.855849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.772176+0000","last_change":"2024-09-21T06:34:30.628123+0000","last_active":"2024-09-21T06:36:05.772176+0000","last_peered":"2024-09-21T06:36:05.772176+0000","last_clean":"2024-09-21T06:36:05.772176+0000","last_became_active":"2024-09-21T06:34:30.627498+0000","last_became_peered":"2024-09-21T06:34:30.627498+0000","last_unstale":"2024-09-21T06:36:05.772176+0000","last_undegraded":"2024-09-21T06:36:05.772176+0000","last_fullsized":"2024-09-21T06:36:05.772176+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:42.576441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":115,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622503+0000","last_change":"2024-09-21T06:33:16.856623+0000","last_active":"2024-09-21T06:36:05.622503+0000","last_peered":"2024-09-21T06:36:05.622503+0000","last_clean":"2024-09-21T06:36:05.622503+0000","last_became_active":"2024-09-21T06:33:16.850787+0000","last_became_peered":"2024-09-21T06:33:16.850787+0000","last_unstale":"2024-09-21T06:36:05.622503+0000","last_undegraded":"2024-09-21T06:36:05.622503+0000","last_fullsized":"2024-09-21T06:36:05.622503+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:50.283252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:50.283252+0000","last_clean_scrub_stamp":"2024-09-21T06:29:50.283252+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:29:20.711208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622549+0000","last_change":"2024-09-21T06:34:39.706472+0000","last_active":"2024-09-21T06:36:05.622549+0000","last_peered":"2024-09-21T06:36:05.622549+0000","last_clean":"2024-09-21T06:36:05.622549+0000","last_became_active":"2024-09-21T06:34:39.705724+0000","last_became_peered":"2024-09-21T06:34:39.705724+0000","last_unstale":"2024-09-21T06:36:05.622549+0000","last_undegraded":"2024-09-21T06:36:05.622549+0000","last_fullsized":"2024-09-21T06:36:05.622549+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:43.238704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.246993+0000","last_change":"2024-09-21T06:34:32.645944+0000","last_active":"2024-09-21T06:36:34.246993+0000","last_peered":"2024-09-21T06:36:34.246993+0000","last_clean":"2024-09-21T06:36:34.246993+0000","last_became_active":"2024-09-21T06:34:32.645134+0000","last_became_peered":"2024-09-21T06:34:32.645134+0000","last_unstale":"2024-09-21T06:36:34.246993+0000","last_undegraded":"2024-09-21T06:36:34.246993+0000","last_fullsized":"2024-09-21T06:36:34.246993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:45.254164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622645+0000","last_change":"2024-09-21T06:34:30.946296+0000","last_active":"2024-09-21T06:36:05.622645+0000","last_peered":"2024-09-21T06:36:05.622645+0000","last_clean":"2024-09-21T06:36:05.622645+0000","last_became_active":"2024-09-21T06:34:30.945777+0000","last_became_peered":"2024-09-21T06:34:30.945777+0000","last_unstale":"2024-09-21T06:36:05.622645+0000","last_undegraded":"2024-09-21T06:36:05.622645+0000","last_fullsized":"2024-09-21T06:36:05.622645+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:32:25.988512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835130+0000","last_change":"2024-09-21T06:34:39.705876+0000","last_active":"2024-09-21T06:36:05.835130+0000","last_peered":"2024-09-21T06:36:05.835130+0000","last_clean":"2024-09-21T06:36:05.835130+0000","last_became_active":"2024-09-21T06:34:39.705132+0000","last_became_peered":"2024-09-21T06:34:39.705132+0000","last_unstale":"2024-09-21T06:36:05.835130+0000","last_undegraded":"2024-09-21T06:36:05.835130+0000","last_fullsized":"2024-09-21T06:36:05.835130+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:18.180244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"85'6","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.242114+0000","last_change":"2024-09-21T06:34:32.638242+0000","last_active":"2024-09-21T06:36:34.242114+0000","last_peered":"2024-09-21T06:36:34.242114+0000","last_clean":"2024-09-21T06:36:34.242114+0000","last_became_active":"2024-09-21T06:34:32.637816+0000","last_became_peered":"2024-09-21T06:34:32.637816+0000","last_unstale":"2024-09-21T06:36:34.242114+0000","last_undegraded":"2024-09-21T06:36:34.242114+0000","last_fullsized":"2024-09-21T06:36:34.242114+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:40.926147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835006+0000","last_change":"2024-09-21T06:34:30.632173+0000","last_active":"2024-09-21T06:36:05.835006+0000","last_peered":"2024-09-21T06:36:05.835006+0000","last_clean":"2024-09-21T06:36:05.835006+0000","last_became_active":"2024-09-21T06:34:30.631638+0000","last_became_peered":"2024-09-21T06:34:30.631638+0000","last_unstale":"2024-09-21T06:36:05.835006+0000","last_undegraded":"2024-09-21T06:36:05.835006+0000","last_fullsized":"2024-09-21T06:36:05.835006+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:29.763272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.628935+0000","last_change":"2024-09-21T06:34:39.714620+0000","last_active":"2024-09-21T06:36:05.628935+0000","last_peered":"2024-09-21T06:36:05.628935+0000","last_clean":"2024-09-21T06:36:05.628935+0000","last_became_active":"2024-09-21T06:34:39.714199+0000","last_became_peered":"2024-09-21T06:34:39.714199+0000","last_unstale":"2024-09-21T06:36:05.628935+0000","last_undegraded":"2024-09-21T06:36:05.628935+0000","last_fullsized":"2024-09-21T06:36:05.628935+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:37.041249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.236401+0000","last_change":"2024-09-21T06:34:32.645001+0000","last_active":"2024-09-21T06:36:34.236401+0000","last_peered":"2024-09-21T06:36:34.236401+0000","last_clean":"2024-09-21T06:36:34.236401+0000","last_became_active":"2024-09-21T06:34:32.644584+0000","last_became_peered":"2024-09-21T06:34:32.644584+0000","last_unstale":"2024-09-21T06:36:34.236401+0000","last_undegraded":"2024-09-21T06:36:34.236401+0000","last_fullsized":"2024-09-21T06:36:34.236401+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:47.098850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835010+0000","last_change":"2024-09-21T06:34:30.632593+0000","last_active":"2024-09-21T06:36:05.835010+0000","last_peered":"2024-09-21T06:36:05.835010+0000","last_clean":"2024-09-21T06:36:05.835010+0000","last_became_active":"2024-09-21T06:34:30.632170+0000","last_became_peered":"2024-09-21T06:34:30.632170+0000","last_unstale":"2024-09-21T06:36:05.835010+0000","last_undegraded":"2024-09-21T06:36:05.835010+0000","last_fullsized":"2024-09-21T06:36:05.835010+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:56.170655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834214+0000","last_change":"2024-09-21T06:34:39.710172+0000","last_active":"2024-09-21T06:36:05.834214+0000","last_peered":"2024-09-21T06:36:05.834214+0000","last_clean":"2024-09-21T06:36:05.834214+0000","last_became_active":"2024-09-21T06:34:39.708876+0000","last_became_peered":"2024-09-21T06:34:39.708876+0000","last_unstale":"2024-09-21T06:36:05.834214+0000","last_undegraded":"2024-09-21T06:36:05.834214+0000","last_fullsized":"2024-09-21T06:36:05.834214+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:32.371873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.226698+0000","last_change":"2024-09-21T06:34:32.651056+0000","last_active":"2024-09-21T06:36:34.226698+0000","last_peered":"2024-09-21T06:36:34.226698+0000","last_clean":"2024-09-21T06:36:34.226698+0000","last_became_active":"2024-09-21T06:34:32.650676+0000","last_became_peered":"2024-09-21T06:34:32.650676+0000","last_unstale":"2024-09-21T06:36:34.226698+0000","last_undegraded":"2024-09-21T06:36:34.226698+0000","last_fullsized":"2024-09-21T06:36:34.226698+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:34.599654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.625707+0000","last_change":"2024-09-21T06:34:30.632307+0000","last_active":"2024-09-21T06:36:05.625707+0000","last_peered":"2024-09-21T06:36:05.625707+0000","last_clean":"2024-09-21T06:36:05.625707+0000","last_became_active":"2024-09-21T06:34:30.631920+0000","last_became_peered":"2024-09-21T06:34:30.631920+0000","last_unstale":"2024-09-21T06:36:05.625707+0000","last_undegraded":"2024-09-21T06:36:05.625707+0000","last_fullsized":"2024-09-21T06:36:05.625707+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:46.940898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834440+0000","last_change":"2024-09-21T06:34:39.715056+0000","last_active":"2024-09-21T06:36:05.834440+0000","last_peered":"2024-09-21T06:36:05.834440+0000","last_clean":"2024-09-21T06:36:05.834440+0000","last_became_active":"2024-09-21T06:34:39.714563+0000","last_became_peered":"2024-09-21T06:34:39.714563+0000","last_unstale":"2024-09-21T06:36:05.834440+0000","last_undegraded":"2024-09-21T06:36:05.834440+0000","last_fullsized":"2024-09-21T06:36:05.834440+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:48.567057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.246651+0000","last_change":"2024-09-21T06:34:32.646934+0000","last_active":"2024-09-21T06:36:34.246651+0000","last_peered":"2024-09-21T06:36:34.246651+0000","last_clean":"2024-09-21T06:36:34.246651+0000","last_became_active":"2024-09-21T06:34:32.646559+0000","last_became_peered":"2024-09-21T06:34:32.646559+0000","last_unstale":"2024-09-21T06:36:34.246651+0000","last_undegraded":"2024-09-21T06:36:34.246651+0000","last_fullsized":"2024-09-21T06:36:34.246651+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:37.802590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":65,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.843557+0000","last_change":"2024-09-21T06:36:05.843018+0000","last_active":"2024-09-21T06:36:05.843557+0000","last_peered":"2024-09-21T06:36:05.843557+0000","last_clean":"2024-09-21T06:36:05.843557+0000","last_became_active":"2024-09-21T06:36:05.836284+0000","last_became_peered":"2024-09-21T06:36:05.836284+0000","last_unstale":"2024-09-21T06:36:05.843557+0000","last_undegraded":"2024-09-21T06:36:05.843557+0000","last_fullsized":"2024-09-21T06:36:05.843557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:51.539804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834952+0000","last_change":"2024-09-21T06:34:39.713398+0000","last_active":"2024-09-21T06:36:05.834952+0000","last_peered":"2024-09-21T06:36:05.834952+0000","last_clean":"2024-09-21T06:36:05.834952+0000","last_became_active":"2024-09-21T06:34:39.712954+0000","last_became_peered":"2024-09-21T06:34:39.712954+0000","last_unstale":"2024-09-21T06:36:05.834952+0000","last_undegraded":"2024-09-21T06:36:05.834952+0000","last_fullsized":"2024-09-21T06:36:05.834952+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:58.881014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.236221+0000","last_change":"2024-09-21T06:34:32.640879+0000","last_active":"2024-09-21T06:36:34.236221+0000","last_peered":"2024-09-21T06:36:34.236221+0000","last_clean":"2024-09-21T06:36:34.236221+0000","last_became_active":"2024-09-21T06:34:32.640526+0000","last_became_peered":"2024-09-21T06:34:32.640526+0000","last_unstale":"2024-09-21T06:36:34.236221+0000","last_undegraded":"2024-09-21T06:36:34.236221+0000","last_fullsized":"2024-09-21T06:36:34.236221+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:12.327195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771987+0000","last_change":"2024-09-21T06:34:30.631449+0000","last_active":"2024-09-21T06:36:05.771987+0000","last_peered":"2024-09-21T06:36:05.771987+0000","last_clean":"2024-09-21T06:36:05.771987+0000","last_became_active":"2024-09-21T06:34:30.631081+0000","last_became_peered":"2024-09-21T06:34:30.631081+0000","last_unstale":"2024-09-21T06:36:05.771987+0000","last_undegraded":"2024-09-21T06:36:05.771987+0000","last_fullsized":"2024-09-21T06:36:05.771987+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:09.202696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621660+0000","last_change":"2024-09-21T06:34:39.707379+0000","last_active":"2024-09-21T06:36:05.621660+0000","last_peered":"2024-09-21T06:36:05.621660+0000","last_clean":"2024-09-21T06:36:05.621660+0000","last_became_active":"2024-09-21T06:34:39.706979+0000","last_became_peered":"2024-09-21T06:34:39.706979+0000","last_unstale":"2024-09-21T06:36:05.621660+0000","last_undegraded":"2024-09-21T06:36:05.621660+0000","last_fullsized":"2024-09-21T06:36:05.621660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:24.394233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":50,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.243242+0000","last_change":"2024-09-21T06:34:32.649458+0000","last_active":"2024-09-21T06:36:34.243242+0000","last_peered":"2024-09-21T06:36:34.243242+0000","last_clean":"2024-09-21T06:36:34.243242+0000","last_became_active":"2024-09-21T06:34:32.649051+0000","last_became_peered":"2024-09-21T06:34:32.649051+0000","last_unstale":"2024-09-21T06:36:34.243242+0000","last_undegraded":"2024-09-21T06:36:34.243242+0000","last_fullsized":"2024-09-21T06:36:34.243242+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:12.631446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834534+0000","last_change":"2024-09-21T06:34:30.634592+0000","last_active":"2024-09-21T06:36:05.834534+0000","last_peered":"2024-09-21T06:36:05.834534+0000","last_clean":"2024-09-21T06:36:05.834534+0000","last_became_active":"2024-09-21T06:34:30.634251+0000","last_became_peered":"2024-09-21T06:34:30.634251+0000","last_unstale":"2024-09-21T06:36:05.834534+0000","last_undegraded":"2024-09-21T06:36:05.834534+0000","last_fullsized":"2024-09-21T06:36:05.834534+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:32.817467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624233+0000","last_change":"2024-09-21T06:34:39.719416+0000","last_active":"2024-09-21T06:36:05.624233+0000","last_peered":"2024-09-21T06:36:05.624233+0000","last_clean":"2024-09-21T06:36:05.624233+0000","last_became_active":"2024-09-21T06:34:39.718910+0000","last_became_peered":"2024-09-21T06:34:39.718910+0000","last_unstale":"2024-09-21T06:36:05.624233+0000","last_undegraded":"2024-09-21T06:36:05.624233+0000","last_fullsized":"2024-09-21T06:36:05.624233+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:41.413250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.235830+0000","last_change":"2024-09-21T06:34:32.640727+0000","last_active":"2024-09-21T06:36:34.235830+0000","last_peered":"2024-09-21T06:36:34.235830+0000","last_clean":"2024-09-21T06:36:34.235830+0000","last_became_active":"2024-09-21T06:34:32.640296+0000","last_became_peered":"2024-09-21T06:34:32.640296+0000","last_unstale":"2024-09-21T06:36:34.235830+0000","last_undegraded":"2024-09-21T06:36:34.235830+0000","last_fullsized":"2024-09-21T06:36:34.235830+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:35.124059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622197+0000","last_change":"2024-09-21T06:34:30.633332+0000","last_active":"2024-09-21T06:36:05.622197+0000","last_peered":"2024-09-21T06:36:05.622197+0000","last_clean":"2024-09-21T06:36:05.622197+0000","last_became_active":"2024-09-21T06:34:30.632641+0000","last_became_peered":"2024-09-21T06:34:30.632641+0000","last_unstale":"2024-09-21T06:36:05.622197+0000","last_undegraded":"2024-09-21T06:36:05.622197+0000","last_fullsized":"2024-09-21T06:36:05.622197+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:30:23.930313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834659+0000","last_change":"2024-09-21T06:34:39.712504+0000","last_active":"2024-09-21T06:36:05.834659+0000","last_peered":"2024-09-21T06:36:05.834659+0000","last_clean":"2024-09-21T06:36:05.834659+0000","last_became_active":"2024-09-21T06:34:39.711638+0000","last_became_peered":"2024-09-21T06:34:39.711638+0000","last_unstale":"2024-09-21T06:36:05.834659+0000","last_undegraded":"2024-09-21T06:36:05.834659+0000","last_fullsized":"2024-09-21T06:36:05.834659+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:26:09.003367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.237201+0000","last_change":"2024-09-21T06:34:32.639582+0000","last_active":"2024-09-21T06:36:34.237201+0000","last_peered":"2024-09-21T06:36:34.237201+0000","last_clean":"2024-09-21T06:36:34.237201+0000","last_became_active":"2024-09-21T06:34:32.639023+0000","last_became_peered":"2024-09-21T06:34:32.639023+0000","last_unstale":"2024-09-21T06:36:34.237201+0000","last_undegraded":"2024-09-21T06:36:34.237201+0000","last_fullsized":"2024-09-21T06:36:34.237201+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:26.773389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"85'33","reported_seq":287,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:50.204582+0000","last_change":"2024-09-21T06:34:30.626280+0000","last_active":"2024-09-21T06:36:50.204582+0000","last_peered":"2024-09-21T06:36:50.204582+0000","last_clean":"2024-09-21T06:36:50.204582+0000","last_became_active":"2024-09-21T06:34:30.625662+0000","last_became_peered":"2024-09-21T06:34:30.625662+0000","last_unstale":"2024-09-21T06:36:50.204582+0000","last_undegraded":"2024-09-21T06:36:50.204582+0000","last_fullsized":"2024-09-21T06:36:50.204582+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:10.541881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69034,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":214,"num_read_kb":175,"num_write":33,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621183+0000","last_change":"2024-09-21T06:34:39.708999+0000","last_active":"2024-09-21T06:36:05.621183+0000","last_peered":"2024-09-21T06:36:05.621183+0000","last_clean":"2024-09-21T06:36:05.621183+0000","last_became_active":"2024-09-21T06:34:39.708580+0000","last_became_peered":"2024-09-21T06:34:39.708580+0000","last_unstale":"2024-09-21T06:36:05.621183+0000","last_undegraded":"2024-09-21T06:36:05.621183+0000","last_fullsized":"2024-09-21T06:36:05.621183+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:07.648358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.247017+0000","last_change":"2024-09-21T06:34:32.651157+0000","last_active":"2024-09-21T06:36:34.247017+0000","last_peered":"2024-09-21T06:36:34.247017+0000","last_clean":"2024-09-21T06:36:34.247017+0000","last_became_active":"2024-09-21T06:34:32.650804+0000","last_became_peered":"2024-09-21T06:34:32.650804+0000","last_unstale":"2024-09-21T06:36:34.247017+0000","last_undegraded":"2024-09-21T06:36:34.247017+0000","last_fullsized":"2024-09-21T06:36:34.247017+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:49.758496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834880+0000","last_change":"2024-09-21T06:34:30.947142+0000","last_active":"2024-09-21T06:36:05.834880+0000","last_peered":"2024-09-21T06:36:05.834880+0000","last_clean":"2024-09-21T06:36:05.834880+0000","last_became_active":"2024-09-21T06:34:30.946748+0000","last_became_peered":"2024-09-21T06:34:30.946748+0000","last_unstale":"2024-09-21T06:36:05.834880+0000","last_undegraded":"2024-09-21T06:36:05.834880+0000","last_fullsized":"2024-09-21T06:36:05.834880+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:39.222286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771885+0000","last_change":"2024-09-21T06:34:39.709266+0000","last_active":"2024-09-21T06:36:05.771885+0000","last_peered":"2024-09-21T06:36:05.771885+0000","last_clean":"2024-09-21T06:36:05.771885+0000","last_became_active":"2024-09-21T06:34:39.708753+0000","last_became_peered":"2024-09-21T06:34:39.708753+0000","last_unstale":"2024-09-21T06:36:05.771885+0000","last_undegraded":"2024-09-21T06:36:05.771885+0000","last_fullsized":"2024-09-21T06:36:05.771885+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:48.711681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.241271+0000","last_change":"2024-09-21T06:34:32.637976+0000","last_active":"2024-09-21T06:36:34.241271+0000","last_peered":"2024-09-21T06:36:34.241271+0000","last_clean":"2024-09-21T06:36:34.241271+0000","last_became_active":"2024-09-21T06:34:32.637267+0000","last_became_peered":"2024-09-21T06:34:32.637267+0000","last_unstale":"2024-09-21T06:36:34.241271+0000","last_undegraded":"2024-09-21T06:36:34.241271+0000","last_fullsized":"2024-09-21T06:36:34.241271+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:31.340583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.836482+0000","last_change":"2024-09-21T06:36:05.836382+0000","last_active":"2024-09-21T06:36:05.836482+0000","last_peered":"2024-09-21T06:36:05.836482+0000","last_clean":"2024-09-21T06:36:05.836482+0000","last_became_active":"2024-09-21T06:36:05.835918+0000","last_became_peered":"2024-09-21T06:36:05.835918+0000","last_unstale":"2024-09-21T06:36:05.836482+0000","last_undegraded":"2024-09-21T06:36:05.836482+0000","last_fullsized":"2024-09-21T06:36:05.836482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:47.346400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621023+0000","last_change":"2024-09-21T06:34:39.713831+0000","last_active":"2024-09-21T06:36:05.621023+0000","last_peered":"2024-09-21T06:36:05.621023+0000","last_clean":"2024-09-21T06:36:05.621023+0000","last_became_active":"2024-09-21T06:34:39.713403+0000","last_became_peered":"2024-09-21T06:34:39.713403+0000","last_unstale":"2024-09-21T06:36:05.621023+0000","last_undegraded":"2024-09-21T06:36:05.621023+0000","last_fullsized":"2024-09-21T06:36:05.621023+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:22.201063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.246305+0000","last_change":"2024-09-21T06:34:32.646924+0000","last_active":"2024-09-21T06:36:34.246305+0000","last_peered":"2024-09-21T06:36:34.246305+0000","last_clean":"2024-09-21T06:36:34.246305+0000","last_became_active":"2024-09-21T06:34:32.646531+0000","last_became_peered":"2024-09-21T06:34:32.646531+0000","last_unstale":"2024-09-21T06:36:34.246305+0000","last_undegraded":"2024-09-21T06:36:34.246305+0000","last_fullsized":"2024-09-21T06:36:34.246305+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:48.550193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835325+0000","last_change":"2024-09-21T06:34:30.631901+0000","last_active":"2024-09-21T06:36:05.835325+0000","last_peered":"2024-09-21T06:36:05.835325+0000","last_clean":"2024-09-21T06:36:05.835325+0000","last_became_active":"2024-09-21T06:34:30.631450+0000","last_became_peered":"2024-09-21T06:34:30.631450+0000","last_unstale":"2024-09-21T06:36:05.835325+0000","last_undegraded":"2024-09-21T06:36:05.835325+0000","last_fullsized":"2024-09-21T06:36:05.835325+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:20.709456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834441+0000","last_change":"2024-09-21T06:34:39.711951+0000","last_active":"2024-09-21T06:36:05.834441+0000","last_peered":"2024-09-21T06:36:05.834441+0000","last_clean":"2024-09-21T06:36:05.834441+0000","last_became_active":"2024-09-21T06:34:39.710783+0000","last_became_peered":"2024-09-21T06:34:39.710783+0000","last_unstale":"2024-09-21T06:36:05.834441+0000","last_undegraded":"2024-09-21T06:36:05.834441+0000","last_fullsized":"2024-09-21T06:36:05.834441+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:27:58.501155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.629078+0000","last_change":"2024-09-21T06:34:39.709639+0000","last_active":"2024-09-21T06:36:05.629078+0000","last_peered":"2024-09-21T06:36:05.629078+0000","last_clean":"2024-09-21T06:36:05.629078+0000","last_became_active":"2024-09-21T06:34:39.709241+0000","last_became_peered":"2024-09-21T06:34:39.709241+0000","last_unstale":"2024-09-21T06:36:05.629078+0000","last_undegraded":"2024-09-21T06:36:05.629078+0000","last_fullsized":"2024-09-21T06:36:05.629078+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.357399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.240180+0000","last_change":"2024-09-21T06:34:32.635365+0000","last_active":"2024-09-21T06:36:34.240180+0000","last_peered":"2024-09-21T06:36:34.240180+0000","last_clean":"2024-09-21T06:36:34.240180+0000","last_became_active":"2024-09-21T06:34:32.635012+0000","last_became_peered":"2024-09-21T06:34:32.635012+0000","last_unstale":"2024-09-21T06:36:34.240180+0000","last_undegraded":"2024-09-21T06:36:34.240180+0000","last_fullsized":"2024-09-21T06:36:34.240180+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:54.671074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834581+0000","last_change":"2024-09-21T06:34:30.633347+0000","last_active":"2024-09-21T06:36:05.834581+0000","last_peered":"2024-09-21T06:36:05.834581+0000","last_clean":"2024-09-21T06:36:05.834581+0000","last_became_active":"2024-09-21T06:34:30.632513+0000","last_became_peered":"2024-09-21T06:34:30.632513+0000","last_unstale":"2024-09-21T06:36:05.834581+0000","last_undegraded":"2024-09-21T06:36:05.834581+0000","last_fullsized":"2024-09-21T06:36:05.834581+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:12.439109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624854+0000","last_change":"2024-09-21T06:34:39.710204+0000","last_active":"2024-09-21T06:36:05.624854+0000","last_peered":"2024-09-21T06:36:05.624854+0000","last_clean":"2024-09-21T06:36:05.624854+0000","last_became_active":"2024-09-21T06:34:39.709744+0000","last_became_peered":"2024-09-21T06:34:39.709744+0000","last_unstale":"2024-09-21T06:36:05.624854+0000","last_undegraded":"2024-09-21T06:36:05.624854+0000","last_fullsized":"2024-09-21T06:36:05.624854+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:45.484088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.229155+0000","last_change":"2024-09-21T06:34:32.634170+0000","last_active":"2024-09-21T06:36:34.229155+0000","last_peered":"2024-09-21T06:36:34.229155+0000","last_clean":"2024-09-21T06:36:34.229155+0000","last_became_active":"2024-09-21T06:34:32.633831+0000","last_became_peered":"2024-09-21T06:34:32.633831+0000","last_unstale":"2024-09-21T06:36:34.229155+0000","last_undegraded":"2024-09-21T06:36:34.229155+0000","last_fullsized":"2024-09-21T06:36:34.229155+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:36.738279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.833746+0000","last_change":"2024-09-21T06:34:30.629422+0000","last_active":"2024-09-21T06:36:05.833746+0000","last_peered":"2024-09-21T06:36:05.833746+0000","last_clean":"2024-09-21T06:36:05.833746+0000","last_became_active":"2024-09-21T06:34:30.629025+0000","last_became_peered":"2024-09-21T06:34:30.629025+0000","last_unstale":"2024-09-21T06:36:05.833746+0000","last_undegraded":"2024-09-21T06:36:05.833746+0000","last_fullsized":"2024-09-21T06:36:05.833746+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:05.068816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834561+0000","last_change":"2024-09-21T06:34:30.634562+0000","last_active":"2024-09-21T06:36:05.834561+0000","last_peered":"2024-09-21T06:36:05.834561+0000","last_clean":"2024-09-21T06:36:05.834561+0000","last_became_active":"2024-09-21T06:34:30.634170+0000","last_became_peered":"2024-09-21T06:34:30.634170+0000","last_unstale":"2024-09-21T06:36:05.834561+0000","last_undegraded":"2024-09-21T06:36:05.834561+0000","last_fullsized":"2024-09-21T06:36:05.834561+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:15.243103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.234496+0000","last_change":"2024-09-21T06:34:32.645718+0000","last_active":"2024-09-21T06:36:34.234496+0000","last_peered":"2024-09-21T06:36:34.234496+0000","last_clean":"2024-09-21T06:36:34.234496+0000","last_became_active":"2024-09-21T06:34:32.645072+0000","last_became_peered":"2024-09-21T06:34:32.645072+0000","last_unstale":"2024-09-21T06:36:34.234496+0000","last_undegraded":"2024-09-21T06:36:34.234496+0000","last_fullsized":"2024-09-21T06:36:34.234496+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:54.109246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622307+0000","last_change":"2024-09-21T06:34:39.716947+0000","last_active":"2024-09-21T06:36:05.622307+0000","last_peered":"2024-09-21T06:36:05.622307+0000","last_clean":"2024-09-21T06:36:05.622307+0000","last_became_active":"2024-09-21T06:34:39.716540+0000","last_became_peered":"2024-09-21T06:34:39.716540+0000","last_unstale":"2024-09-21T06:36:05.622307+0000","last_undegraded":"2024-09-21T06:36:05.622307+0000","last_fullsized":"2024-09-21T06:36:05.622307+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:12.748444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834526+0000","last_change":"2024-09-21T06:34:30.636389+0000","last_active":"2024-09-21T06:36:05.834526+0000","last_peered":"2024-09-21T06:36:05.834526+0000","last_clean":"2024-09-21T06:36:05.834526+0000","last_became_active":"2024-09-21T06:34:30.636008+0000","last_became_peered":"2024-09-21T06:34:30.636008+0000","last_unstale":"2024-09-21T06:36:05.834526+0000","last_undegraded":"2024-09-21T06:36:05.834526+0000","last_fullsized":"2024-09-21T06:36:05.834526+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:45.476267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.219675+0000","last_change":"2024-09-21T06:34:32.635644+0000","last_active":"2024-09-21T06:36:34.219675+0000","last_peered":"2024-09-21T06:36:34.219675+0000","last_clean":"2024-09-21T06:36:34.219675+0000","last_became_active":"2024-09-21T06:34:32.635253+0000","last_became_peered":"2024-09-21T06:34:32.635253+0000","last_unstale":"2024-09-21T06:36:34.219675+0000","last_undegraded":"2024-09-21T06:36:34.219675+0000","last_fullsized":"2024-09-21T06:36:34.219675+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:34.713656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622327+0000","last_change":"2024-09-21T06:34:39.710382+0000","last_active":"2024-09-21T06:36:05.622327+0000","last_peered":"2024-09-21T06:36:05.622327+0000","last_clean":"2024-09-21T06:36:05.622327+0000","last_became_active":"2024-09-21T06:34:39.709884+0000","last_became_peered":"2024-09-21T06:34:39.709884+0000","last_unstale":"2024-09-21T06:36:05.622327+0000","last_undegraded":"2024-09-21T06:36:05.622327+0000","last_fullsized":"2024-09-21T06:36:05.622327+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:01.602366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.234789+0000","last_change":"2024-09-21T06:34:32.640025+0000","last_active":"2024-09-21T06:36:34.234789+0000","last_peered":"2024-09-21T06:36:34.234789+0000","last_clean":"2024-09-21T06:36:34.234789+0000","last_became_active":"2024-09-21T06:34:32.639648+0000","last_became_peered":"2024-09-21T06:34:32.639648+0000","last_unstale":"2024-09-21T06:36:34.234789+0000","last_undegraded":"2024-09-21T06:36:34.234789+0000","last_fullsized":"2024-09-21T06:36:34.234789+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:05.833065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":54,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.230047+0000","last_change":"2024-09-21T06:35:05.115344+0000","last_active":"2024-09-21T06:36:34.230047+0000","last_peered":"2024-09-21T06:36:34.230047+0000","last_clean":"2024-09-21T06:36:34.230047+0000","last_became_active":"2024-09-21T06:35:05.114678+0000","last_became_peered":"2024-09-21T06:35:05.114678+0000","last_unstale":"2024-09-21T06:36:34.230047+0000","last_undegraded":"2024-09-21T06:36:34.230047+0000","last_fullsized":"2024-09-21T06:36:34.230047+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:24.593145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":50,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.229653+0000","last_change":"2024-09-21T06:35:05.115301+0000","last_active":"2024-09-21T06:36:34.229653+0000","last_peered":"2024-09-21T06:36:34.229653+0000","last_clean":"2024-09-21T06:36:34.229653+0000","last_became_active":"2024-09-21T06:35:05.114706+0000","last_became_peered":"2024-09-21T06:35:05.114706+0000","last_unstale":"2024-09-21T06:36:34.229653+0000","last_undegraded":"2024-09-21T06:36:34.229653+0000","last_fullsized":"2024-09-21T06:36:34.229653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:09.513594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.241136+0000","last_change":"2024-09-21T06:34:32.634679+0000","last_active":"2024-09-21T06:36:34.241136+0000","last_peered":"2024-09-21T06:36:34.241136+0000","last_clean":"2024-09-21T06:36:34.241136+0000","last_became_active":"2024-09-21T06:34:32.634285+0000","last_became_peered":"2024-09-21T06:34:32.634285+0000","last_unstale":"2024-09-21T06:36:34.241136+0000","last_undegraded":"2024-09-21T06:36:34.241136+0000","last_fullsized":"2024-09-21T06:36:34.241136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:15.185453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.238589+0000","last_change":"2024-09-21T06:34:32.639037+0000","last_active":"2024-09-21T06:36:34.238589+0000","last_peered":"2024-09-21T06:36:34.238589+0000","last_clean":"2024-09-21T06:36:34.238589+0000","last_became_active":"2024-09-21T06:34:32.638689+0000","last_became_peered":"2024-09-21T06:34:32.638689+0000","last_unstale":"2024-09-21T06:36:34.238589+0000","last_undegraded":"2024-09-21T06:36:34.238589+0000","last_fullsized":"2024-09-21T06:36:34.238589+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:13.609191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.213709+0000","last_change":"2024-09-21T06:35:05.115051+0000","last_active":"2024-09-21T06:36:34.213709+0000","last_peered":"2024-09-21T06:36:34.213709+0000","last_clean":"2024-09-21T06:36:34.213709+0000","last_became_active":"2024-09-21T06:35:05.114602+0000","last_became_peered":"2024-09-21T06:35:05.114602+0000","last_unstale":"2024-09-21T06:36:34.213709+0000","last_undegraded":"2024-09-21T06:36:34.213709+0000","last_fullsized":"2024-09-21T06:36:34.213709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:48.778188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":50,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.233404+0000","last_change":"2024-09-21T06:34:32.637237+0000","last_active":"2024-09-21T06:36:34.233404+0000","last_peered":"2024-09-21T06:36:34.233404+0000","last_clean":"2024-09-21T06:36:34.233404+0000","last_became_active":"2024-09-21T06:34:32.634105+0000","last_became_peered":"2024-09-21T06:34:32.634105+0000","last_unstale":"2024-09-21T06:36:34.233404+0000","last_undegraded":"2024-09-21T06:36:34.233404+0000","last_fullsized":"2024-09-21T06:36:34.233404+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:52.530461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.238241+0000","last_change":"2024-09-21T06:34:32.624441+0000","last_active":"2024-09-21T06:36:34.238241+0000","last_peered":"2024-09-21T06:36:34.238241+0000","last_clean":"2024-09-21T06:36:34.238241+0000","last_became_active":"2024-09-21T06:34:32.623974+0000","last_became_peered":"2024-09-21T06:34:32.623974+0000","last_unstale":"2024-09-21T06:36:34.238241+0000","last_undegraded":"2024-09-21T06:36:34.238241+0000","last_fullsized":"2024-09-21T06:36:34.238241+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:12.378113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.228142+0000","last_change":"2024-09-21T06:34:32.633879+0000","last_active":"2024-09-21T06:36:34.228142+0000","last_peered":"2024-09-21T06:36:34.228142+0000","last_clean":"2024-09-21T06:36:34.228142+0000","last_became_active":"2024-09-21T06:34:32.631791+0000","last_became_peered":"2024-09-21T06:34:32.631791+0000","last_unstale":"2024-09-21T06:36:34.228142+0000","last_undegraded":"2024-09-21T06:36:34.228142+0000","last_fullsized":"2024-09-21T06:36:34.228142+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:46.246892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.238453+0000","last_change":"2024-09-21T06:34:32.639974+0000","last_active":"2024-09-21T06:36:34.238453+0000","last_peered":"2024-09-21T06:36:34.238453+0000","last_clean":"2024-09-21T06:36:34.238453+0000","last_became_active":"2024-09-21T06:34:32.639631+0000","last_became_peered":"2024-09-21T06:34:32.639631+0000","last_unstale":"2024-09-21T06:36:34.238453+0000","last_undegraded":"2024-09-21T06:36:34.238453+0000","last_fullsized":"2024-09-21T06:36:34.238453+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:50.069378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.243809+0000","last_change":"2024-09-21T06:34:32.645403+0000","last_active":"2024-09-21T06:36:34.243809+0000","last_peered":"2024-09-21T06:36:34.243809+0000","last_clean":"2024-09-21T06:36:34.243809+0000","last_became_active":"2024-09-21T06:34:32.644986+0000","last_became_peered":"2024-09-21T06:34:32.644986+0000","last_unstale":"2024-09-21T06:36:34.243809+0000","last_undegraded":"2024-09-21T06:36:34.243809+0000","last_fullsized":"2024-09-21T06:36:34.243809+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:43.640107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.237746+0000","last_change":"2024-09-21T06:34:32.639933+0000","last_active":"2024-09-21T06:36:34.237746+0000","last_peered":"2024-09-21T06:36:34.237746+0000","last_clean":"2024-09-21T06:36:34.237746+0000","last_became_active":"2024-09-21T06:34:32.639438+0000","last_became_peered":"2024-09-21T06:34:32.639438+0000","last_unstale":"2024-09-21T06:36:34.237746+0000","last_undegraded":"2024-09-21T06:36:34.237746+0000","last_fullsized":"2024-09-21T06:36:34.237746+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:18.860219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.226962+0000","last_change":"2024-09-21T06:34:32.648580+0000","last_active":"2024-09-21T06:36:34.226962+0000","last_peered":"2024-09-21T06:36:34.226962+0000","last_clean":"2024-09-21T06:36:34.226962+0000","last_became_active":"2024-09-21T06:34:32.648016+0000","last_became_peered":"2024-09-21T06:34:32.648016+0000","last_unstale":"2024-09-21T06:36:34.226962+0000","last_undegraded":"2024-09-21T06:36:34.226962+0000","last_fullsized":"2024-09-21T06:36:34.226962+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:52.858069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.240213+0000","last_change":"2024-09-21T06:34:32.633376+0000","last_active":"2024-09-21T06:36:34.240213+0000","last_peered":"2024-09-21T06:36:34.240213+0000","last_clean":"2024-09-21T06:36:34.240213+0000","last_became_active":"2024-09-21T06:34:32.631052+0000","last_became_peered":"2024-09-21T06:34:32.631052+0000","last_unstale":"2024-09-21T06:36:34.240213+0000","last_undegraded":"2024-09-21T06:36:34.240213+0000","last_fullsized":"2024-09-21T06:36:34.240213+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:46.229599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.244444+0000","last_change":"2024-09-21T06:34:32.636966+0000","last_active":"2024-09-21T06:36:34.244444+0000","last_peered":"2024-09-21T06:36:34.244444+0000","last_clean":"2024-09-21T06:36:34.244444+0000","last_became_active":"2024-09-21T06:34:32.636565+0000","last_became_peered":"2024-09-21T06:34:32.636565+0000","last_unstale":"2024-09-21T06:36:34.244444+0000","last_undegraded":"2024-09-21T06:36:34.244444+0000","last_fullsized":"2024-09-21T06:36:34.244444+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:36.233082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.246729+0000","last_change":"2024-09-21T06:34:32.646895+0000","last_active":"2024-09-21T06:36:34.246729+0000","last_peered":"2024-09-21T06:36:34.246729+0000","last_clean":"2024-09-21T06:36:34.246729+0000","last_became_active":"2024-09-21T06:34:32.646505+0000","last_became_peered":"2024-09-21T06:34:32.646505+0000","last_unstale":"2024-09-21T06:36:34.246729+0000","last_undegraded":"2024-09-21T06:36:34.246729+0000","last_fullsized":"2024-09-21T06:36:34.246729+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:43.441337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.246723+0000","last_change":"2024-09-21T06:34:32.645402+0000","last_active":"2024-09-21T06:36:34.246723+0000","last_peered":"2024-09-21T06:36:34.246723+0000","last_clean":"2024-09-21T06:36:34.246723+0000","last_became_active":"2024-09-21T06:34:32.645013+0000","last_became_peered":"2024-09-21T06:34:32.645013+0000","last_unstale":"2024-09-21T06:36:34.246723+0000","last_undegraded":"2024-09-21T06:36:34.246723+0000","last_fullsized":"2024-09-21T06:36:34.246723+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:33.475847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.240830+0000","last_change":"2024-09-21T06:34:32.643234+0000","last_active":"2024-09-21T06:36:34.240830+0000","last_peered":"2024-09-21T06:36:34.240830+0000","last_clean":"2024-09-21T06:36:34.240830+0000","last_became_active":"2024-09-21T06:34:32.642614+0000","last_became_peered":"2024-09-21T06:34:32.642614+0000","last_unstale":"2024-09-21T06:36:34.240830+0000","last_undegraded":"2024-09-21T06:36:34.240830+0000","last_fullsized":"2024-09-21T06:36:34.240830+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:50.903500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.242716+0000","last_change":"2024-09-21T06:34:32.641887+0000","last_active":"2024-09-21T06:36:34.242716+0000","last_peered":"2024-09-21T06:36:34.242716+0000","last_clean":"2024-09-21T06:36:34.242716+0000","last_became_active":"2024-09-21T06:34:32.641490+0000","last_became_peered":"2024-09-21T06:34:32.641490+0000","last_unstale":"2024-09-21T06:36:34.242716+0000","last_undegraded":"2024-09-21T06:36:34.242716+0000","last_fullsized":"2024-09-21T06:36:34.242716+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:34.550970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.234750+0000","last_change":"2024-09-21T06:34:32.641865+0000","last_active":"2024-09-21T06:36:34.234750+0000","last_peered":"2024-09-21T06:36:34.234750+0000","last_clean":"2024-09-21T06:36:34.234750+0000","last_became_active":"2024-09-21T06:34:32.641252+0000","last_became_peered":"2024-09-21T06:34:32.641252+0000","last_unstale":"2024-09-21T06:36:34.234750+0000","last_undegraded":"2024-09-21T06:36:34.234750+0000","last_fullsized":"2024-09-21T06:36:34.234750+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:44.459202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.236389+0000","last_change":"2024-09-21T06:34:32.641259+0000","last_active":"2024-09-21T06:36:34.236389+0000","last_peered":"2024-09-21T06:36:34.236389+0000","last_clean":"2024-09-21T06:36:34.236389+0000","last_became_active":"2024-09-21T06:34:32.640907+0000","last_became_peered":"2024-09-21T06:34:32.640907+0000","last_unstale":"2024-09-21T06:36:34.236389+0000","last_undegraded":"2024-09-21T06:36:34.236389+0000","last_fullsized":"2024-09-21T06:36:34.236389+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:59.966857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.246472+0000","last_change":"2024-09-21T06:34:32.646095+0000","last_active":"2024-09-21T06:36:34.246472+0000","last_peered":"2024-09-21T06:36:34.246472+0000","last_clean":"2024-09-21T06:36:34.246472+0000","last_became_active":"2024-09-21T06:34:32.645604+0000","last_became_peered":"2024-09-21T06:34:32.645604+0000","last_unstale":"2024-09-21T06:36:34.246472+0000","last_undegraded":"2024-09-21T06:36:34.246472+0000","last_fullsized":"2024-09-21T06:36:34.246472+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:30.872428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":50,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.235849+0000","last_change":"2024-09-21T06:34:32.649303+0000","last_active":"2024-09-21T06:36:34.235849+0000","last_peered":"2024-09-21T06:36:34.235849+0000","last_clean":"2024-09-21T06:36:34.235849+0000","last_became_active":"2024-09-21T06:34:32.648965+0000","last_became_peered":"2024-09-21T06:34:32.648965+0000","last_unstale":"2024-09-21T06:36:34.235849+0000","last_undegraded":"2024-09-21T06:36:34.235849+0000","last_fullsized":"2024-09-21T06:36:34.235849+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:51.335949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.247103+0000","last_change":"2024-09-21T06:34:32.637261+0000","last_active":"2024-09-21T06:36:34.247103+0000","last_peered":"2024-09-21T06:36:34.247103+0000","last_clean":"2024-09-21T06:36:34.247103+0000","last_became_active":"2024-09-21T06:34:32.636861+0000","last_became_peered":"2024-09-21T06:34:32.636861+0000","last_unstale":"2024-09-21T06:36:34.247103+0000","last_undegraded":"2024-09-21T06:36:34.247103+0000","last_fullsized":"2024-09-21T06:36:34.247103+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:55.271027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.237027+0000","last_change":"2024-09-21T06:34:32.635156+0000","last_active":"2024-09-21T06:36:34.237027+0000","last_peered":"2024-09-21T06:36:34.237027+0000","last_clean":"2024-09-21T06:36:34.237027+0000","last_became_active":"2024-09-21T06:34:32.634802+0000","last_became_peered":"2024-09-21T06:34:32.634802+0000","last_unstale":"2024-09-21T06:36:34.237027+0000","last_undegraded":"2024-09-21T06:36:34.237027+0000","last_fullsized":"2024-09-21T06:36:34.237027+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:16.139829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.228891+0000","last_change":"2024-09-21T06:34:32.637509+0000","last_active":"2024-09-21T06:36:34.228891+0000","last_peered":"2024-09-21T06:36:34.228891+0000","last_clean":"2024-09-21T06:36:34.228891+0000","last_became_active":"2024-09-21T06:34:32.637136+0000","last_became_peered":"2024-09-21T06:34:32.637136+0000","last_unstale":"2024-09-21T06:36:34.228891+0000","last_undegraded":"2024-09-21T06:36:34.228891+0000","last_fullsized":"2024-09-21T06:36:34.228891+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:48.088988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.233952+0000","last_change":"2024-09-21T06:35:05.114819+0000","last_active":"2024-09-21T06:36:34.233952+0000","last_peered":"2024-09-21T06:36:34.233952+0000","last_clean":"2024-09-21T06:36:34.233952+0000","last_became_active":"2024-09-21T06:35:05.114424+0000","last_became_peered":"2024-09-21T06:35:05.114424+0000","last_unstale":"2024-09-21T06:36:34.233952+0000","last_undegraded":"2024-09-21T06:36:34.233952+0000","last_fullsized":"2024-09-21T06:36:34.233952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:59.323643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.225869+0000","last_change":"2024-09-21T06:34:32.636103+0000","last_active":"2024-09-21T06:36:34.225869+0000","last_peered":"2024-09-21T06:36:34.225869+0000","last_clean":"2024-09-21T06:36:34.225869+0000","last_became_active":"2024-09-21T06:34:32.635728+0000","last_became_peered":"2024-09-21T06:34:32.635728+0000","last_unstale":"2024-09-21T06:36:34.225869+0000","last_undegraded":"2024-09-21T06:36:34.225869+0000","last_fullsized":"2024-09-21T06:36:34.225869+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:16.731402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.235424+0000","last_change":"2024-09-21T06:34:32.635654+0000","last_active":"2024-09-21T06:36:34.235424+0000","last_peered":"2024-09-21T06:36:34.235424+0000","last_clean":"2024-09-21T06:36:34.235424+0000","last_became_active":"2024-09-21T06:34:32.635265+0000","last_became_peered":"2024-09-21T06:34:32.635265+0000","last_unstale":"2024-09-21T06:36:34.235424+0000","last_undegraded":"2024-09-21T06:36:34.235424+0000","last_fullsized":"2024-09-21T06:36:34.235424+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:28.169625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.246376+0000","last_change":"2024-09-21T06:34:32.644640+0000","last_active":"2024-09-21T06:36:34.246376+0000","last_peered":"2024-09-21T06:36:34.246376+0000","last_clean":"2024-09-21T06:36:34.246376+0000","last_became_active":"2024-09-21T06:34:32.644334+0000","last_became_peered":"2024-09-21T06:34:32.644334+0000","last_unstale":"2024-09-21T06:36:34.246376+0000","last_undegraded":"2024-09-21T06:36:34.246376+0000","last_fullsized":"2024-09-21T06:36:34.246376+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:25.572178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.242912+0000","last_change":"2024-09-21T06:34:32.643705+0000","last_active":"2024-09-21T06:36:34.242912+0000","last_peered":"2024-09-21T06:36:34.242912+0000","last_clean":"2024-09-21T06:36:34.242912+0000","last_became_active":"2024-09-21T06:34:32.642705+0000","last_became_peered":"2024-09-21T06:34:32.642705+0000","last_unstale":"2024-09-21T06:36:34.242912+0000","last_undegraded":"2024-09-21T06:36:34.242912+0000","last_fullsized":"2024-09-21T06:36:34.242912+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:22.542559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"85'8","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.246617+0000","last_change":"2024-09-21T06:34:32.646539+0000","last_active":"2024-09-21T06:36:34.246617+0000","last_peered":"2024-09-21T06:36:34.246617+0000","last_clean":"2024-09-21T06:36:34.246617+0000","last_became_active":"2024-09-21T06:34:32.645992+0000","last_became_peered":"2024-09-21T06:34:32.645992+0000","last_unstale":"2024-09-21T06:36:34.246617+0000","last_undegraded":"2024-09-21T06:36:34.246617+0000","last_fullsized":"2024-09-21T06:36:34.246617+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:38.856001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69832,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":218,"num_read_kb":179,"num_write":90,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":209989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282016,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743466,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":485699,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939694,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":1141199,"data_compressed":65460,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103220,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":490377,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299449,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":486303,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495678,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":987136,"data_stored":1080745,"data_compressed":65460,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659204,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":481603,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855433,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":1076249,"data_compressed":65460,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018958,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":485721,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215186,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":933888,"data_stored":481707,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":550671,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":550637,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":929792,"data_stored":481693,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":69068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:36:54.866 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T06:36:54.866 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T06:36:55.127 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:36:55.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:55 smithi067 ceph-mon[31232]: pgmap v370: 129 pgs: 129 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:36:55.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:55 smithi067 ceph-mon[31232]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:36:55.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.640+0000 7fef57e48640 1 -- 172.21.15.67:0/2068832909 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef50108cf0 msgr2=0x7fef5010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:55.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.640+0000 7fef57e48640 1 --2- 172.21.15.67:0/2068832909 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef50108cf0 0x7fef5010f520 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fef48009900 tx=0x7fef4802a3f0 comp rx=0 tx=0).stop 2024-09-21T06:36:55.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.640+0000 7fef57e48640 1 -- 172.21.15.67:0/2068832909 shutdown_connections 2024-09-21T06:36:55.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.640+0000 7fef57e48640 1 --2- 172.21.15.67:0/2068832909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef501083a0 0x7fef501087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:55.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.640+0000 7fef57e48640 1 --2- 172.21.15.67:0/2068832909 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef50108cf0 0x7fef5010f520 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:55.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.640+0000 7fef57e48640 1 --2- 172.21.15.67:0/2068832909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef50102a60 0x7fef50102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:55.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.640+0000 7fef57e48640 1 -- 172.21.15.67:0/2068832909 >> 172.21.15.67:0/2068832909 conn(0x7fef50079720 msgr2=0x7fef50079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:55.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.641+0000 7fef57e48640 1 -- 172.21.15.67:0/2068832909 shutdown_connections 2024-09-21T06:36:55.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.641+0000 7fef57e48640 1 -- 172.21.15.67:0/2068832909 wait complete. 2024-09-21T06:36:55.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.641+0000 7fef57e48640 1 Processor -- start 2024-09-21T06:36:55.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.641+0000 7fef57e48640 1 -- start start 2024-09-21T06:36:55.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.642+0000 7fef57e48640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef50102a60 0x7fef5019c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:55.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.642+0000 7fef57e48640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef501083a0 0x7fef5019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:55.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.642+0000 7fef57e48640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef50108cf0 0x7fef501a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:55.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.642+0000 7fef57e48640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fef50111720 con 0x7fef50108cf0 2024-09-21T06:36:55.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.642+0000 7fef57e48640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fef501115a0 con 0x7fef50102a60 2024-09-21T06:36:55.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.642+0000 7fef57e48640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fef501118a0 con 0x7fef501083a0 2024-09-21T06:36:55.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.642+0000 7fef563e5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef50108cf0 0x7fef501a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:55.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.642+0000 7fef563e5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef50108cf0 0x7fef501a1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54694/0 (socket says 172.21.15.67:54694) 2024-09-21T06:36:55.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.642+0000 7fef563e5640 1 -- 172.21.15.67:0/3185567811 learned_addr learned my addr 172.21.15.67:0/3185567811 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:55.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.642+0000 7fef553e3640 1 --2- 172.21.15.67:0/3185567811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef501083a0 0x7fef5019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:55.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.642+0000 7fef55be4640 1 --2- 172.21.15.67:0/3185567811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef50102a60 0x7fef5019c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:55.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.642+0000 7fef563e5640 1 -- 172.21.15.67:0/3185567811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef501083a0 msgr2=0x7fef5019cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:55.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.642+0000 7fef563e5640 1 --2- 172.21.15.67:0/3185567811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef501083a0 0x7fef5019cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:55.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.642+0000 7fef563e5640 1 -- 172.21.15.67:0/3185567811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef50102a60 msgr2=0x7fef5019c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:55.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.642+0000 7fef563e5640 1 --2- 172.21.15.67:0/3185567811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef50102a60 0x7fef5019c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:55.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.642+0000 7fef563e5640 1 -- 172.21.15.67:0/3185567811 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef501a2570 con 0x7fef50108cf0 2024-09-21T06:36:55.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.642+0000 7fef55be4640 1 --2- 172.21.15.67:0/3185567811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef50102a60 0x7fef5019c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:36:55.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.642+0000 7fef563e5640 1 --2- 172.21.15.67:0/3185567811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef50108cf0 0x7fef501a1e40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fef48005ab0 tx=0x7fef480040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:55.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.643+0000 7fef4effd640 1 -- 172.21.15.67:0/3185567811 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4803f030 con 0x7fef50108cf0 2024-09-21T06:36:55.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.643+0000 7fef4effd640 1 -- 172.21.15.67:0/3185567811 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef48003a80 con 0x7fef50108cf0 2024-09-21T06:36:55.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.643+0000 7fef57e48640 1 -- 172.21.15.67:0/3185567811 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef501a2800 con 0x7fef50108cf0 2024-09-21T06:36:55.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.643+0000 7fef4effd640 1 -- 172.21.15.67:0/3185567811 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48035070 con 0x7fef50108cf0 2024-09-21T06:36:55.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.643+0000 7fef57e48640 1 -- 172.21.15.67:0/3185567811 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef501a8fc0 con 0x7fef50108cf0 2024-09-21T06:36:55.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.644+0000 7fef57e48640 1 -- 172.21.15.67:0/3185567811 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef50103ce0 con 0x7fef50108cf0 2024-09-21T06:36:55.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.645+0000 7fef4effd640 1 -- 172.21.15.67:0/3185567811 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fef48042030 con 0x7fef50108cf0 2024-09-21T06:36:55.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.646+0000 7fef4effd640 1 --2- 172.21.15.67:0/3185567811 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fef380b6510 0x7fef380b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:55.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.646+0000 7fef55be4640 1 --2- 172.21.15.67:0/3185567811 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fef380b6510 0x7fef380b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:55.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.646+0000 7fef4effd640 1 -- 172.21.15.67:0/3185567811 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fef48108000 con 0x7fef50108cf0 2024-09-21T06:36:55.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.647+0000 7fef55be4640 1 --2- 172.21.15.67:0/3185567811 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fef380b6510 0x7fef380b8a20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fef40005c40 tx=0x7fef4000a410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:55.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.649+0000 7fef4effd640 1 -- 172.21.15.67:0/3185567811 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fef480ce4c0 con 0x7fef50108cf0 2024-09-21T06:36:55.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:55 smithi100 ceph-mon[33401]: pgmap v370: 129 pgs: 129 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:36:55.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:55 smithi100 ceph-mon[33401]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:36:55.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.842+0000 7fef57e48640 1 -- 172.21.15.67:0/3185567811 --> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fef50063800 con 0x7fef380b6510 2024-09-21T06:36:55.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.850+0000 7fef4effd640 1 -- 172.21.15.67:0/3185567811 <== mgr.14152 v2:172.21.15.67:6800/1134893458 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340645 (secure 0 0 0) 0x7fef50063800 con 0x7fef380b6510 2024-09-21T06:36:55.856 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:36:55.857 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-09-21T06:36:55.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.853+0000 7fef57e48640 1 -- 172.21.15.67:0/3185567811 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fef380b6510 msgr2=0x7fef380b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:55.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.853+0000 7fef57e48640 1 --2- 172.21.15.67:0/3185567811 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fef380b6510 0x7fef380b8a20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fef40005c40 tx=0x7fef4000a410 comp rx=0 tx=0).stop 2024-09-21T06:36:55.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.853+0000 7fef57e48640 1 -- 172.21.15.67:0/3185567811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef50108cf0 msgr2=0x7fef501a1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:55.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.853+0000 7fef57e48640 1 --2- 172.21.15.67:0/3185567811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef50108cf0 0x7fef501a1e40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fef48005ab0 tx=0x7fef480040e0 comp rx=0 tx=0).stop 2024-09-21T06:36:55.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.853+0000 7fef57e48640 1 -- 172.21.15.67:0/3185567811 shutdown_connections 2024-09-21T06:36:55.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.853+0000 7fef57e48640 1 --2- 172.21.15.67:0/3185567811 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fef380b6510 0x7fef380b8a20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:55.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.853+0000 7fef57e48640 1 --2- 172.21.15.67:0/3185567811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef50108cf0 0x7fef501a1e40 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:55.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.853+0000 7fef57e48640 1 --2- 172.21.15.67:0/3185567811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef501083a0 0x7fef5019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:55.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.853+0000 7fef57e48640 1 --2- 172.21.15.67:0/3185567811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef50102a60 0x7fef5019c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:55.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.853+0000 7fef57e48640 1 -- 172.21.15.67:0/3185567811 >> 172.21.15.67:0/3185567811 conn(0x7fef50079720 msgr2=0x7fef50104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:55.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.854+0000 7fef57e48640 1 -- 172.21.15.67:0/3185567811 shutdown_connections 2024-09-21T06:36:55.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:55.854+0000 7fef57e48640 1 -- 172.21.15.67:0/3185567811 wait complete. 2024-09-21T06:36:55.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:55 smithi203 ceph-mon[33735]: pgmap v370: 129 pgs: 129 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:36:55.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:55 smithi203 ceph-mon[33735]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:36:56.587 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":370,"stamp":"2024-09-21T06:36:53.882568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660434,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":294,"num_read_kb":243,"num_write":215,"num_write_kb":2251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":669,"ondisk_log_size":669,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368880,"kb_used_data":11312,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572944,"statfs":{"total":1151940427776,"available":1151562694656,"internally_reserved":0,"allocated":11583488,"data_stored":7792604,"data_compressed":462771,"data_compressed_allocated":626688,"data_compressed_original":2752512,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003853"},"pg_stats":[{"pgid":"2.1f","version":"75'4","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621812+0000","last_change":"2024-09-21T06:34:30.633366+0000","last_active":"2024-09-21T06:36:05.621812+0000","last_peered":"2024-09-21T06:36:05.621812+0000","last_clean":"2024-09-21T06:36:05.621812+0000","last_became_active":"2024-09-21T06:34:30.632929+0000","last_became_peered":"2024-09-21T06:34:30.632929+0000","last_unstale":"2024-09-21T06:36:05.621812+0000","last_undegraded":"2024-09-21T06:36:05.621812+0000","last_fullsized":"2024-09-21T06:36:05.621812+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:47.722461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.238429+0000","last_change":"2024-09-21T06:34:32.632989+0000","last_active":"2024-09-21T06:36:34.238429+0000","last_peered":"2024-09-21T06:36:34.238429+0000","last_clean":"2024-09-21T06:36:34.238429+0000","last_became_active":"2024-09-21T06:34:32.632024+0000","last_became_peered":"2024-09-21T06:34:32.632024+0000","last_unstale":"2024-09-21T06:36:34.238429+0000","last_undegraded":"2024-09-21T06:36:34.238429+0000","last_fullsized":"2024-09-21T06:36:34.238429+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:13.669613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621767+0000","last_change":"2024-09-21T06:35:05.274434+0000","last_active":"2024-09-21T06:36:05.621767+0000","last_peered":"2024-09-21T06:36:05.621767+0000","last_clean":"2024-09-21T06:36:05.621767+0000","last_became_active":"2024-09-21T06:35:05.273833+0000","last_became_peered":"2024-09-21T06:35:05.273833+0000","last_unstale":"2024-09-21T06:36:05.621767+0000","last_undegraded":"2024-09-21T06:36:05.621767+0000","last_fullsized":"2024-09-21T06:36:05.621767+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:48.847861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835213+0000","last_change":"2024-09-21T06:34:30.946309+0000","last_active":"2024-09-21T06:36:05.835213+0000","last_peered":"2024-09-21T06:36:05.835213+0000","last_clean":"2024-09-21T06:36:05.835213+0000","last_became_active":"2024-09-21T06:34:30.945913+0000","last_became_peered":"2024-09-21T06:34:30.945913+0000","last_unstale":"2024-09-21T06:36:05.835213+0000","last_undegraded":"2024-09-21T06:36:05.835213+0000","last_fullsized":"2024-09-21T06:36:05.835213+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:21.676524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.246270+0000","last_change":"2024-09-21T06:34:32.637636+0000","last_active":"2024-09-21T06:36:34.246270+0000","last_peered":"2024-09-21T06:36:34.246270+0000","last_clean":"2024-09-21T06:36:34.246270+0000","last_became_active":"2024-09-21T06:34:32.636534+0000","last_became_peered":"2024-09-21T06:34:32.636534+0000","last_unstale":"2024-09-21T06:36:34.246270+0000","last_undegraded":"2024-09-21T06:36:34.246270+0000","last_fullsized":"2024-09-21T06:36:34.246270+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:05.802118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771498+0000","last_change":"2024-09-21T06:35:05.274419+0000","last_active":"2024-09-21T06:36:05.771498+0000","last_peered":"2024-09-21T06:36:05.771498+0000","last_clean":"2024-09-21T06:36:05.771498+0000","last_became_active":"2024-09-21T06:35:05.273865+0000","last_became_peered":"2024-09-21T06:35:05.273865+0000","last_unstale":"2024-09-21T06:36:05.771498+0000","last_undegraded":"2024-09-21T06:36:05.771498+0000","last_fullsized":"2024-09-21T06:36:05.771498+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:04.490355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.631709+0000","last_change":"2024-09-21T06:36:05.631098+0000","last_active":"2024-09-21T06:36:05.631709+0000","last_peered":"2024-09-21T06:36:05.631709+0000","last_clean":"2024-09-21T06:36:05.631709+0000","last_became_active":"2024-09-21T06:36:05.627521+0000","last_became_peered":"2024-09-21T06:36:05.627521+0000","last_unstale":"2024-09-21T06:36:05.631709+0000","last_undegraded":"2024-09-21T06:36:05.631709+0000","last_fullsized":"2024-09-21T06:36:05.631709+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:02.467758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.247327+0000","last_change":"2024-09-21T06:34:32.650303+0000","last_active":"2024-09-21T06:36:34.247327+0000","last_peered":"2024-09-21T06:36:34.247327+0000","last_clean":"2024-09-21T06:36:34.247327+0000","last_became_active":"2024-09-21T06:34:32.647189+0000","last_became_peered":"2024-09-21T06:34:32.647189+0000","last_unstale":"2024-09-21T06:36:34.247327+0000","last_undegraded":"2024-09-21T06:36:34.247327+0000","last_fullsized":"2024-09-21T06:36:34.247327+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:14.889384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771714+0000","last_change":"2024-09-21T06:34:39.713139+0000","last_active":"2024-09-21T06:36:05.771714+0000","last_peered":"2024-09-21T06:36:05.771714+0000","last_clean":"2024-09-21T06:36:05.771714+0000","last_became_active":"2024-09-21T06:34:39.712379+0000","last_became_peered":"2024-09-21T06:34:39.712379+0000","last_unstale":"2024-09-21T06:36:05.771714+0000","last_undegraded":"2024-09-21T06:36:05.771714+0000","last_fullsized":"2024-09-21T06:36:05.771714+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:53.100529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.625272+0000","last_change":"2024-09-21T06:34:30.633850+0000","last_active":"2024-09-21T06:36:05.625272+0000","last_peered":"2024-09-21T06:36:05.625272+0000","last_clean":"2024-09-21T06:36:05.625272+0000","last_became_active":"2024-09-21T06:34:30.633468+0000","last_became_peered":"2024-09-21T06:34:30.633468+0000","last_unstale":"2024-09-21T06:36:05.625272+0000","last_undegraded":"2024-09-21T06:36:05.625272+0000","last_fullsized":"2024-09-21T06:36:05.625272+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:59.095803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":56,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.247919+0000","last_change":"2024-09-21T06:35:05.005240+0000","last_active":"2024-09-21T06:36:34.247919+0000","last_peered":"2024-09-21T06:36:34.247919+0000","last_clean":"2024-09-21T06:36:34.247919+0000","last_became_active":"2024-09-21T06:35:05.004849+0000","last_became_peered":"2024-09-21T06:35:05.004849+0000","last_unstale":"2024-09-21T06:36:34.247919+0000","last_undegraded":"2024-09-21T06:36:34.247919+0000","last_fullsized":"2024-09-21T06:36:34.247919+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:28.849604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771556+0000","last_change":"2024-09-21T06:35:05.274437+0000","last_active":"2024-09-21T06:36:05.771556+0000","last_peered":"2024-09-21T06:36:05.771556+0000","last_clean":"2024-09-21T06:36:05.771556+0000","last_became_active":"2024-09-21T06:35:05.273865+0000","last_became_peered":"2024-09-21T06:35:05.273865+0000","last_unstale":"2024-09-21T06:36:05.771556+0000","last_undegraded":"2024-09-21T06:36:05.771556+0000","last_fullsized":"2024-09-21T06:36:05.771556+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:43.339504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.246814+0000","last_change":"2024-09-21T06:34:32.645630+0000","last_active":"2024-09-21T06:36:34.246814+0000","last_peered":"2024-09-21T06:36:34.246814+0000","last_clean":"2024-09-21T06:36:34.246814+0000","last_became_active":"2024-09-21T06:34:32.645237+0000","last_became_peered":"2024-09-21T06:34:32.645237+0000","last_unstale":"2024-09-21T06:36:34.246814+0000","last_undegraded":"2024-09-21T06:36:34.246814+0000","last_fullsized":"2024-09-21T06:36:34.246814+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:25.064318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.633983+0000","last_change":"2024-09-21T06:36:05.633484+0000","last_active":"2024-09-21T06:36:05.633983+0000","last_peered":"2024-09-21T06:36:05.633983+0000","last_clean":"2024-09-21T06:36:05.633983+0000","last_became_active":"2024-09-21T06:36:05.627637+0000","last_became_peered":"2024-09-21T06:36:05.627637+0000","last_unstale":"2024-09-21T06:36:05.633983+0000","last_undegraded":"2024-09-21T06:36:05.633983+0000","last_fullsized":"2024-09-21T06:36:05.633983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:50:00.626538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771936+0000","last_change":"2024-09-21T06:34:39.714177+0000","last_active":"2024-09-21T06:36:05.771936+0000","last_peered":"2024-09-21T06:36:05.771936+0000","last_clean":"2024-09-21T06:36:05.771936+0000","last_became_active":"2024-09-21T06:34:39.713713+0000","last_became_peered":"2024-09-21T06:34:39.713713+0000","last_unstale":"2024-09-21T06:36:05.771936+0000","last_undegraded":"2024-09-21T06:36:05.771936+0000","last_fullsized":"2024-09-21T06:36:05.771936+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:07.042240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.234553+0000","last_change":"2024-09-21T06:34:32.636297+0000","last_active":"2024-09-21T06:36:34.234553+0000","last_peered":"2024-09-21T06:36:34.234553+0000","last_clean":"2024-09-21T06:36:34.234553+0000","last_became_active":"2024-09-21T06:34:32.635930+0000","last_became_peered":"2024-09-21T06:34:32.635930+0000","last_unstale":"2024-09-21T06:36:34.234553+0000","last_undegraded":"2024-09-21T06:36:34.234553+0000","last_fullsized":"2024-09-21T06:36:34.234553+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:43.091682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621975+0000","last_change":"2024-09-21T06:34:30.631405+0000","last_active":"2024-09-21T06:36:05.621975+0000","last_peered":"2024-09-21T06:36:05.621975+0000","last_clean":"2024-09-21T06:36:05.621975+0000","last_became_active":"2024-09-21T06:34:30.630512+0000","last_became_peered":"2024-09-21T06:34:30.630512+0000","last_unstale":"2024-09-21T06:36:05.621975+0000","last_undegraded":"2024-09-21T06:36:05.621975+0000","last_fullsized":"2024-09-21T06:36:05.621975+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:43.412695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"83'1","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771897+0000","last_change":"2024-09-21T06:34:39.710573+0000","last_active":"2024-09-21T06:36:05.771897+0000","last_peered":"2024-09-21T06:36:05.771897+0000","last_clean":"2024-09-21T06:36:05.771897+0000","last_became_active":"2024-09-21T06:34:39.708969+0000","last_became_peered":"2024-09-21T06:34:39.708969+0000","last_unstale":"2024-09-21T06:36:05.771897+0000","last_undegraded":"2024-09-21T06:36:05.771897+0000","last_fullsized":"2024-09-21T06:36:05.771897+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:34.847587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.234683+0000","last_change":"2024-09-21T06:34:32.641085+0000","last_active":"2024-09-21T06:36:34.234683+0000","last_peered":"2024-09-21T06:36:34.234683+0000","last_clean":"2024-09-21T06:36:34.234683+0000","last_became_active":"2024-09-21T06:34:32.640723+0000","last_became_peered":"2024-09-21T06:34:32.640723+0000","last_unstale":"2024-09-21T06:36:34.234683+0000","last_undegraded":"2024-09-21T06:36:34.234683+0000","last_fullsized":"2024-09-21T06:36:34.234683+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:21.410262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834947+0000","last_change":"2024-09-21T06:34:30.626719+0000","last_active":"2024-09-21T06:36:05.834947+0000","last_peered":"2024-09-21T06:36:05.834947+0000","last_clean":"2024-09-21T06:36:05.834947+0000","last_became_active":"2024-09-21T06:34:30.626312+0000","last_became_peered":"2024-09-21T06:34:30.626312+0000","last_unstale":"2024-09-21T06:36:05.834947+0000","last_undegraded":"2024-09-21T06:36:05.834947+0000","last_fullsized":"2024-09-21T06:36:05.834947+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:53.938914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621617+0000","last_change":"2024-09-21T06:34:39.711172+0000","last_active":"2024-09-21T06:36:05.621617+0000","last_peered":"2024-09-21T06:36:05.621617+0000","last_clean":"2024-09-21T06:36:05.621617+0000","last_became_active":"2024-09-21T06:34:39.708424+0000","last_became_peered":"2024-09-21T06:34:39.708424+0000","last_unstale":"2024-09-21T06:36:05.621617+0000","last_undegraded":"2024-09-21T06:36:05.621617+0000","last_fullsized":"2024-09-21T06:36:05.621617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:43.918355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.209527+0000","last_change":"2024-09-21T06:34:32.641463+0000","last_active":"2024-09-21T06:36:34.209527+0000","last_peered":"2024-09-21T06:36:34.209527+0000","last_clean":"2024-09-21T06:36:34.209527+0000","last_became_active":"2024-09-21T06:34:32.641086+0000","last_became_peered":"2024-09-21T06:34:32.641086+0000","last_unstale":"2024-09-21T06:36:34.209527+0000","last_undegraded":"2024-09-21T06:36:34.209527+0000","last_fullsized":"2024-09-21T06:36:34.209527+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:58.991374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835725+0000","last_change":"2024-09-21T06:34:30.946698+0000","last_active":"2024-09-21T06:36:05.835725+0000","last_peered":"2024-09-21T06:36:05.835725+0000","last_clean":"2024-09-21T06:36:05.835725+0000","last_became_active":"2024-09-21T06:34:30.946325+0000","last_became_peered":"2024-09-21T06:34:30.946325+0000","last_unstale":"2024-09-21T06:36:05.835725+0000","last_undegraded":"2024-09-21T06:36:05.835725+0000","last_fullsized":"2024-09-21T06:36:05.835725+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:22.639265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.772211+0000","last_change":"2024-09-21T06:34:39.712429+0000","last_active":"2024-09-21T06:36:05.772211+0000","last_peered":"2024-09-21T06:36:05.772211+0000","last_clean":"2024-09-21T06:36:05.772211+0000","last_became_active":"2024-09-21T06:34:39.711823+0000","last_became_peered":"2024-09-21T06:34:39.711823+0000","last_unstale":"2024-09-21T06:36:05.772211+0000","last_undegraded":"2024-09-21T06:36:05.772211+0000","last_fullsized":"2024-09-21T06:36:05.772211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:03:49.513991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.243704+0000","last_change":"2024-09-21T06:35:05.274326+0000","last_active":"2024-09-21T06:36:34.243704+0000","last_peered":"2024-09-21T06:36:34.243704+0000","last_clean":"2024-09-21T06:36:34.243704+0000","last_became_active":"2024-09-21T06:35:05.273884+0000","last_became_peered":"2024-09-21T06:35:05.273884+0000","last_unstale":"2024-09-21T06:36:34.243704+0000","last_undegraded":"2024-09-21T06:36:34.243704+0000","last_fullsized":"2024-09-21T06:36:34.243704+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:03.219765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.772393+0000","last_change":"2024-09-21T06:34:30.676794+0000","last_active":"2024-09-21T06:36:05.772393+0000","last_peered":"2024-09-21T06:36:05.772393+0000","last_clean":"2024-09-21T06:36:05.772393+0000","last_became_active":"2024-09-21T06:34:30.676400+0000","last_became_peered":"2024-09-21T06:34:30.676400+0000","last_unstale":"2024-09-21T06:36:05.772393+0000","last_undegraded":"2024-09-21T06:36:05.772393+0000","last_fullsized":"2024-09-21T06:36:05.772393+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:23.681363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622160+0000","last_change":"2024-09-21T06:34:39.708980+0000","last_active":"2024-09-21T06:36:05.622160+0000","last_peered":"2024-09-21T06:36:05.622160+0000","last_clean":"2024-09-21T06:36:05.622160+0000","last_became_active":"2024-09-21T06:34:39.707748+0000","last_became_peered":"2024-09-21T06:34:39.707748+0000","last_unstale":"2024-09-21T06:36:05.622160+0000","last_undegraded":"2024-09-21T06:36:05.622160+0000","last_fullsized":"2024-09-21T06:36:05.622160+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:16.812226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.240400+0000","last_change":"2024-09-21T06:34:32.638337+0000","last_active":"2024-09-21T06:36:34.240400+0000","last_peered":"2024-09-21T06:36:34.240400+0000","last_clean":"2024-09-21T06:36:34.240400+0000","last_became_active":"2024-09-21T06:34:32.638013+0000","last_became_peered":"2024-09-21T06:34:32.638013+0000","last_unstale":"2024-09-21T06:36:34.240400+0000","last_undegraded":"2024-09-21T06:36:34.240400+0000","last_fullsized":"2024-09-21T06:36:34.240400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:26.321190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'10","reported_seq":50,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.673339+0000","last_change":"2024-09-21T06:34:30.631086+0000","last_active":"2024-09-21T06:36:34.673339+0000","last_peered":"2024-09-21T06:36:34.673339+0000","last_clean":"2024-09-21T06:36:34.673339+0000","last_became_active":"2024-09-21T06:34:30.630706+0000","last_became_peered":"2024-09-21T06:34:30.630706+0000","last_unstale":"2024-09-21T06:36:34.673339+0000","last_undegraded":"2024-09-21T06:36:34.673339+0000","last_fullsized":"2024-09-21T06:36:34.673339+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:30.260359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834003+0000","last_change":"2024-09-21T06:34:39.709097+0000","last_active":"2024-09-21T06:36:05.834003+0000","last_peered":"2024-09-21T06:36:05.834003+0000","last_clean":"2024-09-21T06:36:05.834003+0000","last_became_active":"2024-09-21T06:34:39.708609+0000","last_became_peered":"2024-09-21T06:34:39.708609+0000","last_unstale":"2024-09-21T06:36:05.834003+0000","last_undegraded":"2024-09-21T06:36:05.834003+0000","last_fullsized":"2024-09-21T06:36:05.834003+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:48.842174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.229558+0000","last_change":"2024-09-21T06:34:32.636816+0000","last_active":"2024-09-21T06:36:34.229558+0000","last_peered":"2024-09-21T06:36:34.229558+0000","last_clean":"2024-09-21T06:36:34.229558+0000","last_became_active":"2024-09-21T06:34:32.636429+0000","last_became_peered":"2024-09-21T06:34:32.636429+0000","last_unstale":"2024-09-21T06:36:34.229558+0000","last_undegraded":"2024-09-21T06:36:34.229558+0000","last_fullsized":"2024-09-21T06:36:34.229558+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:49.010491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.625629+0000","last_change":"2024-09-21T06:34:30.947481+0000","last_active":"2024-09-21T06:36:05.625629+0000","last_peered":"2024-09-21T06:36:05.625629+0000","last_clean":"2024-09-21T06:36:05.625629+0000","last_became_active":"2024-09-21T06:34:30.947040+0000","last_became_peered":"2024-09-21T06:34:30.947040+0000","last_unstale":"2024-09-21T06:36:05.625629+0000","last_undegraded":"2024-09-21T06:36:05.625629+0000","last_fullsized":"2024-09-21T06:36:05.625629+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:52.813476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624972+0000","last_change":"2024-09-21T06:34:39.708674+0000","last_active":"2024-09-21T06:36:05.624972+0000","last_peered":"2024-09-21T06:36:05.624972+0000","last_clean":"2024-09-21T06:36:05.624972+0000","last_became_active":"2024-09-21T06:34:39.708176+0000","last_became_peered":"2024-09-21T06:34:39.708176+0000","last_unstale":"2024-09-21T06:36:05.624972+0000","last_undegraded":"2024-09-21T06:36:05.624972+0000","last_fullsized":"2024-09-21T06:36:05.624972+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:57.070792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"85'4","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.239723+0000","last_change":"2024-09-21T06:34:32.636381+0000","last_active":"2024-09-21T06:36:34.239723+0000","last_peered":"2024-09-21T06:36:34.239723+0000","last_clean":"2024-09-21T06:36:34.239723+0000","last_became_active":"2024-09-21T06:34:32.635937+0000","last_became_peered":"2024-09-21T06:34:32.635937+0000","last_unstale":"2024-09-21T06:36:34.239723+0000","last_undegraded":"2024-09-21T06:36:34.239723+0000","last_fullsized":"2024-09-21T06:36:34.239723+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:52.870676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621524+0000","last_change":"2024-09-21T06:34:30.947160+0000","last_active":"2024-09-21T06:36:05.621524+0000","last_peered":"2024-09-21T06:36:05.621524+0000","last_clean":"2024-09-21T06:36:05.621524+0000","last_became_active":"2024-09-21T06:34:30.946736+0000","last_became_peered":"2024-09-21T06:34:30.946736+0000","last_unstale":"2024-09-21T06:36:05.621524+0000","last_undegraded":"2024-09-21T06:36:05.621524+0000","last_fullsized":"2024-09-21T06:36:05.621524+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:47.425481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.625820+0000","last_change":"2024-09-21T06:35:05.006032+0000","last_active":"2024-09-21T06:36:05.625820+0000","last_peered":"2024-09-21T06:36:05.625820+0000","last_clean":"2024-09-21T06:36:05.625820+0000","last_became_active":"2024-09-21T06:35:05.005429+0000","last_became_peered":"2024-09-21T06:35:05.005429+0000","last_unstale":"2024-09-21T06:36:05.625820+0000","last_undegraded":"2024-09-21T06:36:05.625820+0000","last_fullsized":"2024-09-21T06:36:05.625820+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:04.138097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.238876+0000","last_change":"2024-09-21T06:34:32.648256+0000","last_active":"2024-09-21T06:36:34.238876+0000","last_peered":"2024-09-21T06:36:34.238876+0000","last_clean":"2024-09-21T06:36:34.238876+0000","last_became_active":"2024-09-21T06:34:32.647881+0000","last_became_peered":"2024-09-21T06:34:32.647881+0000","last_unstale":"2024-09-21T06:36:34.238876+0000","last_undegraded":"2024-09-21T06:36:34.238876+0000","last_fullsized":"2024-09-21T06:36:34.238876+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:14.590834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622945+0000","last_change":"2024-09-21T06:34:30.629918+0000","last_active":"2024-09-21T06:36:05.622945+0000","last_peered":"2024-09-21T06:36:05.622945+0000","last_clean":"2024-09-21T06:36:05.622945+0000","last_became_active":"2024-09-21T06:34:30.628785+0000","last_became_peered":"2024-09-21T06:34:30.628785+0000","last_unstale":"2024-09-21T06:36:05.622945+0000","last_undegraded":"2024-09-21T06:36:05.622945+0000","last_fullsized":"2024-09-21T06:36:05.622945+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:05.333406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834425+0000","last_change":"2024-09-21T06:34:39.710073+0000","last_active":"2024-09-21T06:36:05.834425+0000","last_peered":"2024-09-21T06:36:05.834425+0000","last_clean":"2024-09-21T06:36:05.834425+0000","last_became_active":"2024-09-21T06:34:39.708986+0000","last_became_peered":"2024-09-21T06:34:39.708986+0000","last_unstale":"2024-09-21T06:36:05.834425+0000","last_undegraded":"2024-09-21T06:36:05.834425+0000","last_fullsized":"2024-09-21T06:36:05.834425+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:46.379902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.234328+0000","last_change":"2024-09-21T06:34:32.644224+0000","last_active":"2024-09-21T06:36:34.234328+0000","last_peered":"2024-09-21T06:36:34.234328+0000","last_clean":"2024-09-21T06:36:34.234328+0000","last_became_active":"2024-09-21T06:34:32.643850+0000","last_became_peered":"2024-09-21T06:34:32.643850+0000","last_unstale":"2024-09-21T06:36:34.234328+0000","last_undegraded":"2024-09-21T06:36:34.234328+0000","last_fullsized":"2024-09-21T06:36:34.234328+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:24:33.188537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.833952+0000","last_change":"2024-09-21T06:34:30.946334+0000","last_active":"2024-09-21T06:36:05.833952+0000","last_peered":"2024-09-21T06:36:05.833952+0000","last_clean":"2024-09-21T06:36:05.833952+0000","last_became_active":"2024-09-21T06:34:30.945908+0000","last_became_peered":"2024-09-21T06:34:30.945908+0000","last_unstale":"2024-09-21T06:36:05.833952+0000","last_undegraded":"2024-09-21T06:36:05.833952+0000","last_fullsized":"2024-09-21T06:36:05.833952+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:46.746527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834642+0000","last_change":"2024-09-21T06:34:39.707909+0000","last_active":"2024-09-21T06:36:05.834642+0000","last_peered":"2024-09-21T06:36:05.834642+0000","last_clean":"2024-09-21T06:36:05.834642+0000","last_became_active":"2024-09-21T06:34:39.707492+0000","last_became_peered":"2024-09-21T06:34:39.707492+0000","last_unstale":"2024-09-21T06:36:05.834642+0000","last_undegraded":"2024-09-21T06:36:05.834642+0000","last_fullsized":"2024-09-21T06:36:05.834642+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:56.201212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.227669+0000","last_change":"2024-09-21T06:34:32.635870+0000","last_active":"2024-09-21T06:36:34.227669+0000","last_peered":"2024-09-21T06:36:34.227669+0000","last_clean":"2024-09-21T06:36:34.227669+0000","last_became_active":"2024-09-21T06:34:32.635252+0000","last_became_peered":"2024-09-21T06:34:32.635252+0000","last_unstale":"2024-09-21T06:36:34.227669+0000","last_undegraded":"2024-09-21T06:36:34.227669+0000","last_fullsized":"2024-09-21T06:36:34.227669+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:10.613033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622347+0000","last_change":"2024-09-21T06:34:30.634360+0000","last_active":"2024-09-21T06:36:05.622347+0000","last_peered":"2024-09-21T06:36:05.622347+0000","last_clean":"2024-09-21T06:36:05.622347+0000","last_became_active":"2024-09-21T06:34:30.633845+0000","last_became_peered":"2024-09-21T06:34:30.633845+0000","last_unstale":"2024-09-21T06:36:05.622347+0000","last_undegraded":"2024-09-21T06:36:05.622347+0000","last_fullsized":"2024-09-21T06:36:05.622347+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:34:53.388670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834895+0000","last_change":"2024-09-21T06:34:39.719620+0000","last_active":"2024-09-21T06:36:05.834895+0000","last_peered":"2024-09-21T06:36:05.834895+0000","last_clean":"2024-09-21T06:36:05.834895+0000","last_became_active":"2024-09-21T06:34:39.718786+0000","last_became_peered":"2024-09-21T06:34:39.718786+0000","last_unstale":"2024-09-21T06:36:05.834895+0000","last_undegraded":"2024-09-21T06:36:05.834895+0000","last_fullsized":"2024-09-21T06:36:05.834895+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:43.524932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.236491+0000","last_change":"2024-09-21T06:34:32.634970+0000","last_active":"2024-09-21T06:36:34.236491+0000","last_peered":"2024-09-21T06:36:34.236491+0000","last_clean":"2024-09-21T06:36:34.236491+0000","last_became_active":"2024-09-21T06:34:32.634611+0000","last_became_peered":"2024-09-21T06:34:32.634611+0000","last_unstale":"2024-09-21T06:36:34.236491+0000","last_undegraded":"2024-09-21T06:36:34.236491+0000","last_fullsized":"2024-09-21T06:36:34.236491+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:37.855849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.772176+0000","last_change":"2024-09-21T06:34:30.628123+0000","last_active":"2024-09-21T06:36:05.772176+0000","last_peered":"2024-09-21T06:36:05.772176+0000","last_clean":"2024-09-21T06:36:05.772176+0000","last_became_active":"2024-09-21T06:34:30.627498+0000","last_became_peered":"2024-09-21T06:34:30.627498+0000","last_unstale":"2024-09-21T06:36:05.772176+0000","last_undegraded":"2024-09-21T06:36:05.772176+0000","last_fullsized":"2024-09-21T06:36:05.772176+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:42.576441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":115,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622503+0000","last_change":"2024-09-21T06:33:16.856623+0000","last_active":"2024-09-21T06:36:05.622503+0000","last_peered":"2024-09-21T06:36:05.622503+0000","last_clean":"2024-09-21T06:36:05.622503+0000","last_became_active":"2024-09-21T06:33:16.850787+0000","last_became_peered":"2024-09-21T06:33:16.850787+0000","last_unstale":"2024-09-21T06:36:05.622503+0000","last_undegraded":"2024-09-21T06:36:05.622503+0000","last_fullsized":"2024-09-21T06:36:05.622503+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:29:50.283252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:29:50.283252+0000","last_clean_scrub_stamp":"2024-09-21T06:29:50.283252+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:29:20.711208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622549+0000","last_change":"2024-09-21T06:34:39.706472+0000","last_active":"2024-09-21T06:36:05.622549+0000","last_peered":"2024-09-21T06:36:05.622549+0000","last_clean":"2024-09-21T06:36:05.622549+0000","last_became_active":"2024-09-21T06:34:39.705724+0000","last_became_peered":"2024-09-21T06:34:39.705724+0000","last_unstale":"2024-09-21T06:36:05.622549+0000","last_undegraded":"2024-09-21T06:36:05.622549+0000","last_fullsized":"2024-09-21T06:36:05.622549+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:43.238704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.246993+0000","last_change":"2024-09-21T06:34:32.645944+0000","last_active":"2024-09-21T06:36:34.246993+0000","last_peered":"2024-09-21T06:36:34.246993+0000","last_clean":"2024-09-21T06:36:34.246993+0000","last_became_active":"2024-09-21T06:34:32.645134+0000","last_became_peered":"2024-09-21T06:34:32.645134+0000","last_unstale":"2024-09-21T06:36:34.246993+0000","last_undegraded":"2024-09-21T06:36:34.246993+0000","last_fullsized":"2024-09-21T06:36:34.246993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:45.254164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622645+0000","last_change":"2024-09-21T06:34:30.946296+0000","last_active":"2024-09-21T06:36:05.622645+0000","last_peered":"2024-09-21T06:36:05.622645+0000","last_clean":"2024-09-21T06:36:05.622645+0000","last_became_active":"2024-09-21T06:34:30.945777+0000","last_became_peered":"2024-09-21T06:34:30.945777+0000","last_unstale":"2024-09-21T06:36:05.622645+0000","last_undegraded":"2024-09-21T06:36:05.622645+0000","last_fullsized":"2024-09-21T06:36:05.622645+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:32:25.988512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835130+0000","last_change":"2024-09-21T06:34:39.705876+0000","last_active":"2024-09-21T06:36:05.835130+0000","last_peered":"2024-09-21T06:36:05.835130+0000","last_clean":"2024-09-21T06:36:05.835130+0000","last_became_active":"2024-09-21T06:34:39.705132+0000","last_became_peered":"2024-09-21T06:34:39.705132+0000","last_unstale":"2024-09-21T06:36:05.835130+0000","last_undegraded":"2024-09-21T06:36:05.835130+0000","last_fullsized":"2024-09-21T06:36:05.835130+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:18.180244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"85'6","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.242114+0000","last_change":"2024-09-21T06:34:32.638242+0000","last_active":"2024-09-21T06:36:34.242114+0000","last_peered":"2024-09-21T06:36:34.242114+0000","last_clean":"2024-09-21T06:36:34.242114+0000","last_became_active":"2024-09-21T06:34:32.637816+0000","last_became_peered":"2024-09-21T06:34:32.637816+0000","last_unstale":"2024-09-21T06:36:34.242114+0000","last_undegraded":"2024-09-21T06:36:34.242114+0000","last_fullsized":"2024-09-21T06:36:34.242114+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:40.926147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835006+0000","last_change":"2024-09-21T06:34:30.632173+0000","last_active":"2024-09-21T06:36:05.835006+0000","last_peered":"2024-09-21T06:36:05.835006+0000","last_clean":"2024-09-21T06:36:05.835006+0000","last_became_active":"2024-09-21T06:34:30.631638+0000","last_became_peered":"2024-09-21T06:34:30.631638+0000","last_unstale":"2024-09-21T06:36:05.835006+0000","last_undegraded":"2024-09-21T06:36:05.835006+0000","last_fullsized":"2024-09-21T06:36:05.835006+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:29.763272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.628935+0000","last_change":"2024-09-21T06:34:39.714620+0000","last_active":"2024-09-21T06:36:05.628935+0000","last_peered":"2024-09-21T06:36:05.628935+0000","last_clean":"2024-09-21T06:36:05.628935+0000","last_became_active":"2024-09-21T06:34:39.714199+0000","last_became_peered":"2024-09-21T06:34:39.714199+0000","last_unstale":"2024-09-21T06:36:05.628935+0000","last_undegraded":"2024-09-21T06:36:05.628935+0000","last_fullsized":"2024-09-21T06:36:05.628935+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:37.041249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.236401+0000","last_change":"2024-09-21T06:34:32.645001+0000","last_active":"2024-09-21T06:36:34.236401+0000","last_peered":"2024-09-21T06:36:34.236401+0000","last_clean":"2024-09-21T06:36:34.236401+0000","last_became_active":"2024-09-21T06:34:32.644584+0000","last_became_peered":"2024-09-21T06:34:32.644584+0000","last_unstale":"2024-09-21T06:36:34.236401+0000","last_undegraded":"2024-09-21T06:36:34.236401+0000","last_fullsized":"2024-09-21T06:36:34.236401+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:47.098850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835010+0000","last_change":"2024-09-21T06:34:30.632593+0000","last_active":"2024-09-21T06:36:05.835010+0000","last_peered":"2024-09-21T06:36:05.835010+0000","last_clean":"2024-09-21T06:36:05.835010+0000","last_became_active":"2024-09-21T06:34:30.632170+0000","last_became_peered":"2024-09-21T06:34:30.632170+0000","last_unstale":"2024-09-21T06:36:05.835010+0000","last_undegraded":"2024-09-21T06:36:05.835010+0000","last_fullsized":"2024-09-21T06:36:05.835010+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:56.170655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834214+0000","last_change":"2024-09-21T06:34:39.710172+0000","last_active":"2024-09-21T06:36:05.834214+0000","last_peered":"2024-09-21T06:36:05.834214+0000","last_clean":"2024-09-21T06:36:05.834214+0000","last_became_active":"2024-09-21T06:34:39.708876+0000","last_became_peered":"2024-09-21T06:34:39.708876+0000","last_unstale":"2024-09-21T06:36:05.834214+0000","last_undegraded":"2024-09-21T06:36:05.834214+0000","last_fullsized":"2024-09-21T06:36:05.834214+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:32.371873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.226698+0000","last_change":"2024-09-21T06:34:32.651056+0000","last_active":"2024-09-21T06:36:34.226698+0000","last_peered":"2024-09-21T06:36:34.226698+0000","last_clean":"2024-09-21T06:36:34.226698+0000","last_became_active":"2024-09-21T06:34:32.650676+0000","last_became_peered":"2024-09-21T06:34:32.650676+0000","last_unstale":"2024-09-21T06:36:34.226698+0000","last_undegraded":"2024-09-21T06:36:34.226698+0000","last_fullsized":"2024-09-21T06:36:34.226698+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:34.599654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.625707+0000","last_change":"2024-09-21T06:34:30.632307+0000","last_active":"2024-09-21T06:36:05.625707+0000","last_peered":"2024-09-21T06:36:05.625707+0000","last_clean":"2024-09-21T06:36:05.625707+0000","last_became_active":"2024-09-21T06:34:30.631920+0000","last_became_peered":"2024-09-21T06:34:30.631920+0000","last_unstale":"2024-09-21T06:36:05.625707+0000","last_undegraded":"2024-09-21T06:36:05.625707+0000","last_fullsized":"2024-09-21T06:36:05.625707+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:46.940898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834440+0000","last_change":"2024-09-21T06:34:39.715056+0000","last_active":"2024-09-21T06:36:05.834440+0000","last_peered":"2024-09-21T06:36:05.834440+0000","last_clean":"2024-09-21T06:36:05.834440+0000","last_became_active":"2024-09-21T06:34:39.714563+0000","last_became_peered":"2024-09-21T06:34:39.714563+0000","last_unstale":"2024-09-21T06:36:05.834440+0000","last_undegraded":"2024-09-21T06:36:05.834440+0000","last_fullsized":"2024-09-21T06:36:05.834440+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:48.567057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.246651+0000","last_change":"2024-09-21T06:34:32.646934+0000","last_active":"2024-09-21T06:36:34.246651+0000","last_peered":"2024-09-21T06:36:34.246651+0000","last_clean":"2024-09-21T06:36:34.246651+0000","last_became_active":"2024-09-21T06:34:32.646559+0000","last_became_peered":"2024-09-21T06:34:32.646559+0000","last_unstale":"2024-09-21T06:36:34.246651+0000","last_undegraded":"2024-09-21T06:36:34.246651+0000","last_fullsized":"2024-09-21T06:36:34.246651+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:37.802590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":65,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.843557+0000","last_change":"2024-09-21T06:36:05.843018+0000","last_active":"2024-09-21T06:36:05.843557+0000","last_peered":"2024-09-21T06:36:05.843557+0000","last_clean":"2024-09-21T06:36:05.843557+0000","last_became_active":"2024-09-21T06:36:05.836284+0000","last_became_peered":"2024-09-21T06:36:05.836284+0000","last_unstale":"2024-09-21T06:36:05.843557+0000","last_undegraded":"2024-09-21T06:36:05.843557+0000","last_fullsized":"2024-09-21T06:36:05.843557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:51.539804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834952+0000","last_change":"2024-09-21T06:34:39.713398+0000","last_active":"2024-09-21T06:36:05.834952+0000","last_peered":"2024-09-21T06:36:05.834952+0000","last_clean":"2024-09-21T06:36:05.834952+0000","last_became_active":"2024-09-21T06:34:39.712954+0000","last_became_peered":"2024-09-21T06:34:39.712954+0000","last_unstale":"2024-09-21T06:36:05.834952+0000","last_undegraded":"2024-09-21T06:36:05.834952+0000","last_fullsized":"2024-09-21T06:36:05.834952+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:58.881014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.236221+0000","last_change":"2024-09-21T06:34:32.640879+0000","last_active":"2024-09-21T06:36:34.236221+0000","last_peered":"2024-09-21T06:36:34.236221+0000","last_clean":"2024-09-21T06:36:34.236221+0000","last_became_active":"2024-09-21T06:34:32.640526+0000","last_became_peered":"2024-09-21T06:34:32.640526+0000","last_unstale":"2024-09-21T06:36:34.236221+0000","last_undegraded":"2024-09-21T06:36:34.236221+0000","last_fullsized":"2024-09-21T06:36:34.236221+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:12.327195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771987+0000","last_change":"2024-09-21T06:34:30.631449+0000","last_active":"2024-09-21T06:36:05.771987+0000","last_peered":"2024-09-21T06:36:05.771987+0000","last_clean":"2024-09-21T06:36:05.771987+0000","last_became_active":"2024-09-21T06:34:30.631081+0000","last_became_peered":"2024-09-21T06:34:30.631081+0000","last_unstale":"2024-09-21T06:36:05.771987+0000","last_undegraded":"2024-09-21T06:36:05.771987+0000","last_fullsized":"2024-09-21T06:36:05.771987+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:09.202696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621660+0000","last_change":"2024-09-21T06:34:39.707379+0000","last_active":"2024-09-21T06:36:05.621660+0000","last_peered":"2024-09-21T06:36:05.621660+0000","last_clean":"2024-09-21T06:36:05.621660+0000","last_became_active":"2024-09-21T06:34:39.706979+0000","last_became_peered":"2024-09-21T06:34:39.706979+0000","last_unstale":"2024-09-21T06:36:05.621660+0000","last_undegraded":"2024-09-21T06:36:05.621660+0000","last_fullsized":"2024-09-21T06:36:05.621660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:24.394233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":50,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.243242+0000","last_change":"2024-09-21T06:34:32.649458+0000","last_active":"2024-09-21T06:36:34.243242+0000","last_peered":"2024-09-21T06:36:34.243242+0000","last_clean":"2024-09-21T06:36:34.243242+0000","last_became_active":"2024-09-21T06:34:32.649051+0000","last_became_peered":"2024-09-21T06:34:32.649051+0000","last_unstale":"2024-09-21T06:36:34.243242+0000","last_undegraded":"2024-09-21T06:36:34.243242+0000","last_fullsized":"2024-09-21T06:36:34.243242+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:12.631446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834534+0000","last_change":"2024-09-21T06:34:30.634592+0000","last_active":"2024-09-21T06:36:05.834534+0000","last_peered":"2024-09-21T06:36:05.834534+0000","last_clean":"2024-09-21T06:36:05.834534+0000","last_became_active":"2024-09-21T06:34:30.634251+0000","last_became_peered":"2024-09-21T06:34:30.634251+0000","last_unstale":"2024-09-21T06:36:05.834534+0000","last_undegraded":"2024-09-21T06:36:05.834534+0000","last_fullsized":"2024-09-21T06:36:05.834534+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:32.817467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624233+0000","last_change":"2024-09-21T06:34:39.719416+0000","last_active":"2024-09-21T06:36:05.624233+0000","last_peered":"2024-09-21T06:36:05.624233+0000","last_clean":"2024-09-21T06:36:05.624233+0000","last_became_active":"2024-09-21T06:34:39.718910+0000","last_became_peered":"2024-09-21T06:34:39.718910+0000","last_unstale":"2024-09-21T06:36:05.624233+0000","last_undegraded":"2024-09-21T06:36:05.624233+0000","last_fullsized":"2024-09-21T06:36:05.624233+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:41.413250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.235830+0000","last_change":"2024-09-21T06:34:32.640727+0000","last_active":"2024-09-21T06:36:34.235830+0000","last_peered":"2024-09-21T06:36:34.235830+0000","last_clean":"2024-09-21T06:36:34.235830+0000","last_became_active":"2024-09-21T06:34:32.640296+0000","last_became_peered":"2024-09-21T06:34:32.640296+0000","last_unstale":"2024-09-21T06:36:34.235830+0000","last_undegraded":"2024-09-21T06:36:34.235830+0000","last_fullsized":"2024-09-21T06:36:34.235830+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:35.124059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622197+0000","last_change":"2024-09-21T06:34:30.633332+0000","last_active":"2024-09-21T06:36:05.622197+0000","last_peered":"2024-09-21T06:36:05.622197+0000","last_clean":"2024-09-21T06:36:05.622197+0000","last_became_active":"2024-09-21T06:34:30.632641+0000","last_became_peered":"2024-09-21T06:34:30.632641+0000","last_unstale":"2024-09-21T06:36:05.622197+0000","last_undegraded":"2024-09-21T06:36:05.622197+0000","last_fullsized":"2024-09-21T06:36:05.622197+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:30:23.930313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834659+0000","last_change":"2024-09-21T06:34:39.712504+0000","last_active":"2024-09-21T06:36:05.834659+0000","last_peered":"2024-09-21T06:36:05.834659+0000","last_clean":"2024-09-21T06:36:05.834659+0000","last_became_active":"2024-09-21T06:34:39.711638+0000","last_became_peered":"2024-09-21T06:34:39.711638+0000","last_unstale":"2024-09-21T06:36:05.834659+0000","last_undegraded":"2024-09-21T06:36:05.834659+0000","last_fullsized":"2024-09-21T06:36:05.834659+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:26:09.003367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.237201+0000","last_change":"2024-09-21T06:34:32.639582+0000","last_active":"2024-09-21T06:36:34.237201+0000","last_peered":"2024-09-21T06:36:34.237201+0000","last_clean":"2024-09-21T06:36:34.237201+0000","last_became_active":"2024-09-21T06:34:32.639023+0000","last_became_peered":"2024-09-21T06:34:32.639023+0000","last_unstale":"2024-09-21T06:36:34.237201+0000","last_undegraded":"2024-09-21T06:36:34.237201+0000","last_fullsized":"2024-09-21T06:36:34.237201+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:26.773389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"85'33","reported_seq":287,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:50.204582+0000","last_change":"2024-09-21T06:34:30.626280+0000","last_active":"2024-09-21T06:36:50.204582+0000","last_peered":"2024-09-21T06:36:50.204582+0000","last_clean":"2024-09-21T06:36:50.204582+0000","last_became_active":"2024-09-21T06:34:30.625662+0000","last_became_peered":"2024-09-21T06:34:30.625662+0000","last_unstale":"2024-09-21T06:36:50.204582+0000","last_undegraded":"2024-09-21T06:36:50.204582+0000","last_fullsized":"2024-09-21T06:36:50.204582+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:10.541881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69034,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":214,"num_read_kb":175,"num_write":33,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621183+0000","last_change":"2024-09-21T06:34:39.708999+0000","last_active":"2024-09-21T06:36:05.621183+0000","last_peered":"2024-09-21T06:36:05.621183+0000","last_clean":"2024-09-21T06:36:05.621183+0000","last_became_active":"2024-09-21T06:34:39.708580+0000","last_became_peered":"2024-09-21T06:34:39.708580+0000","last_unstale":"2024-09-21T06:36:05.621183+0000","last_undegraded":"2024-09-21T06:36:05.621183+0000","last_fullsized":"2024-09-21T06:36:05.621183+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:07.648358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.247017+0000","last_change":"2024-09-21T06:34:32.651157+0000","last_active":"2024-09-21T06:36:34.247017+0000","last_peered":"2024-09-21T06:36:34.247017+0000","last_clean":"2024-09-21T06:36:34.247017+0000","last_became_active":"2024-09-21T06:34:32.650804+0000","last_became_peered":"2024-09-21T06:34:32.650804+0000","last_unstale":"2024-09-21T06:36:34.247017+0000","last_undegraded":"2024-09-21T06:36:34.247017+0000","last_fullsized":"2024-09-21T06:36:34.247017+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:49.758496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834880+0000","last_change":"2024-09-21T06:34:30.947142+0000","last_active":"2024-09-21T06:36:05.834880+0000","last_peered":"2024-09-21T06:36:05.834880+0000","last_clean":"2024-09-21T06:36:05.834880+0000","last_became_active":"2024-09-21T06:34:30.946748+0000","last_became_peered":"2024-09-21T06:34:30.946748+0000","last_unstale":"2024-09-21T06:36:05.834880+0000","last_undegraded":"2024-09-21T06:36:05.834880+0000","last_fullsized":"2024-09-21T06:36:05.834880+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:39.222286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.771885+0000","last_change":"2024-09-21T06:34:39.709266+0000","last_active":"2024-09-21T06:36:05.771885+0000","last_peered":"2024-09-21T06:36:05.771885+0000","last_clean":"2024-09-21T06:36:05.771885+0000","last_became_active":"2024-09-21T06:34:39.708753+0000","last_became_peered":"2024-09-21T06:34:39.708753+0000","last_unstale":"2024-09-21T06:36:05.771885+0000","last_undegraded":"2024-09-21T06:36:05.771885+0000","last_fullsized":"2024-09-21T06:36:05.771885+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:48.711681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.241271+0000","last_change":"2024-09-21T06:34:32.637976+0000","last_active":"2024-09-21T06:36:34.241271+0000","last_peered":"2024-09-21T06:36:34.241271+0000","last_clean":"2024-09-21T06:36:34.241271+0000","last_became_active":"2024-09-21T06:34:32.637267+0000","last_became_peered":"2024-09-21T06:34:32.637267+0000","last_unstale":"2024-09-21T06:36:34.241271+0000","last_undegraded":"2024-09-21T06:36:34.241271+0000","last_fullsized":"2024-09-21T06:36:34.241271+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:31.340583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.836482+0000","last_change":"2024-09-21T06:36:05.836382+0000","last_active":"2024-09-21T06:36:05.836482+0000","last_peered":"2024-09-21T06:36:05.836482+0000","last_clean":"2024-09-21T06:36:05.836482+0000","last_became_active":"2024-09-21T06:36:05.835918+0000","last_became_peered":"2024-09-21T06:36:05.835918+0000","last_unstale":"2024-09-21T06:36:05.836482+0000","last_undegraded":"2024-09-21T06:36:05.836482+0000","last_fullsized":"2024-09-21T06:36:05.836482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:47.346400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.621023+0000","last_change":"2024-09-21T06:34:39.713831+0000","last_active":"2024-09-21T06:36:05.621023+0000","last_peered":"2024-09-21T06:36:05.621023+0000","last_clean":"2024-09-21T06:36:05.621023+0000","last_became_active":"2024-09-21T06:34:39.713403+0000","last_became_peered":"2024-09-21T06:34:39.713403+0000","last_unstale":"2024-09-21T06:36:05.621023+0000","last_undegraded":"2024-09-21T06:36:05.621023+0000","last_fullsized":"2024-09-21T06:36:05.621023+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:22.201063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.246305+0000","last_change":"2024-09-21T06:34:32.646924+0000","last_active":"2024-09-21T06:36:34.246305+0000","last_peered":"2024-09-21T06:36:34.246305+0000","last_clean":"2024-09-21T06:36:34.246305+0000","last_became_active":"2024-09-21T06:34:32.646531+0000","last_became_peered":"2024-09-21T06:34:32.646531+0000","last_unstale":"2024-09-21T06:36:34.246305+0000","last_undegraded":"2024-09-21T06:36:34.246305+0000","last_fullsized":"2024-09-21T06:36:34.246305+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:48.550193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.835325+0000","last_change":"2024-09-21T06:34:30.631901+0000","last_active":"2024-09-21T06:36:05.835325+0000","last_peered":"2024-09-21T06:36:05.835325+0000","last_clean":"2024-09-21T06:36:05.835325+0000","last_became_active":"2024-09-21T06:34:30.631450+0000","last_became_peered":"2024-09-21T06:34:30.631450+0000","last_unstale":"2024-09-21T06:36:05.835325+0000","last_undegraded":"2024-09-21T06:36:05.835325+0000","last_fullsized":"2024-09-21T06:36:05.835325+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:20.709456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834441+0000","last_change":"2024-09-21T06:34:39.711951+0000","last_active":"2024-09-21T06:36:05.834441+0000","last_peered":"2024-09-21T06:36:05.834441+0000","last_clean":"2024-09-21T06:36:05.834441+0000","last_became_active":"2024-09-21T06:34:39.710783+0000","last_became_peered":"2024-09-21T06:34:39.710783+0000","last_unstale":"2024-09-21T06:36:05.834441+0000","last_undegraded":"2024-09-21T06:36:05.834441+0000","last_fullsized":"2024-09-21T06:36:05.834441+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:27:58.501155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.629078+0000","last_change":"2024-09-21T06:34:39.709639+0000","last_active":"2024-09-21T06:36:05.629078+0000","last_peered":"2024-09-21T06:36:05.629078+0000","last_clean":"2024-09-21T06:36:05.629078+0000","last_became_active":"2024-09-21T06:34:39.709241+0000","last_became_peered":"2024-09-21T06:34:39.709241+0000","last_unstale":"2024-09-21T06:36:05.629078+0000","last_undegraded":"2024-09-21T06:36:05.629078+0000","last_fullsized":"2024-09-21T06:36:05.629078+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.357399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.240180+0000","last_change":"2024-09-21T06:34:32.635365+0000","last_active":"2024-09-21T06:36:34.240180+0000","last_peered":"2024-09-21T06:36:34.240180+0000","last_clean":"2024-09-21T06:36:34.240180+0000","last_became_active":"2024-09-21T06:34:32.635012+0000","last_became_peered":"2024-09-21T06:34:32.635012+0000","last_unstale":"2024-09-21T06:36:34.240180+0000","last_undegraded":"2024-09-21T06:36:34.240180+0000","last_fullsized":"2024-09-21T06:36:34.240180+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:54.671074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834581+0000","last_change":"2024-09-21T06:34:30.633347+0000","last_active":"2024-09-21T06:36:05.834581+0000","last_peered":"2024-09-21T06:36:05.834581+0000","last_clean":"2024-09-21T06:36:05.834581+0000","last_became_active":"2024-09-21T06:34:30.632513+0000","last_became_peered":"2024-09-21T06:34:30.632513+0000","last_unstale":"2024-09-21T06:36:05.834581+0000","last_undegraded":"2024-09-21T06:36:05.834581+0000","last_fullsized":"2024-09-21T06:36:05.834581+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:12.439109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.624854+0000","last_change":"2024-09-21T06:34:39.710204+0000","last_active":"2024-09-21T06:36:05.624854+0000","last_peered":"2024-09-21T06:36:05.624854+0000","last_clean":"2024-09-21T06:36:05.624854+0000","last_became_active":"2024-09-21T06:34:39.709744+0000","last_became_peered":"2024-09-21T06:34:39.709744+0000","last_unstale":"2024-09-21T06:36:05.624854+0000","last_undegraded":"2024-09-21T06:36:05.624854+0000","last_fullsized":"2024-09-21T06:36:05.624854+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:45.484088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.229155+0000","last_change":"2024-09-21T06:34:32.634170+0000","last_active":"2024-09-21T06:36:34.229155+0000","last_peered":"2024-09-21T06:36:34.229155+0000","last_clean":"2024-09-21T06:36:34.229155+0000","last_became_active":"2024-09-21T06:34:32.633831+0000","last_became_peered":"2024-09-21T06:34:32.633831+0000","last_unstale":"2024-09-21T06:36:34.229155+0000","last_undegraded":"2024-09-21T06:36:34.229155+0000","last_fullsized":"2024-09-21T06:36:34.229155+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:36.738279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.833746+0000","last_change":"2024-09-21T06:34:30.629422+0000","last_active":"2024-09-21T06:36:05.833746+0000","last_peered":"2024-09-21T06:36:05.833746+0000","last_clean":"2024-09-21T06:36:05.833746+0000","last_became_active":"2024-09-21T06:34:30.629025+0000","last_became_peered":"2024-09-21T06:34:30.629025+0000","last_unstale":"2024-09-21T06:36:05.833746+0000","last_undegraded":"2024-09-21T06:36:05.833746+0000","last_fullsized":"2024-09-21T06:36:05.833746+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:05.068816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834561+0000","last_change":"2024-09-21T06:34:30.634562+0000","last_active":"2024-09-21T06:36:05.834561+0000","last_peered":"2024-09-21T06:36:05.834561+0000","last_clean":"2024-09-21T06:36:05.834561+0000","last_became_active":"2024-09-21T06:34:30.634170+0000","last_became_peered":"2024-09-21T06:34:30.634170+0000","last_unstale":"2024-09-21T06:36:05.834561+0000","last_undegraded":"2024-09-21T06:36:05.834561+0000","last_fullsized":"2024-09-21T06:36:05.834561+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:15.243103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.234496+0000","last_change":"2024-09-21T06:34:32.645718+0000","last_active":"2024-09-21T06:36:34.234496+0000","last_peered":"2024-09-21T06:36:34.234496+0000","last_clean":"2024-09-21T06:36:34.234496+0000","last_became_active":"2024-09-21T06:34:32.645072+0000","last_became_peered":"2024-09-21T06:34:32.645072+0000","last_unstale":"2024-09-21T06:36:34.234496+0000","last_undegraded":"2024-09-21T06:36:34.234496+0000","last_fullsized":"2024-09-21T06:36:34.234496+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:54.109246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622307+0000","last_change":"2024-09-21T06:34:39.716947+0000","last_active":"2024-09-21T06:36:05.622307+0000","last_peered":"2024-09-21T06:36:05.622307+0000","last_clean":"2024-09-21T06:36:05.622307+0000","last_became_active":"2024-09-21T06:34:39.716540+0000","last_became_peered":"2024-09-21T06:34:39.716540+0000","last_unstale":"2024-09-21T06:36:05.622307+0000","last_undegraded":"2024-09-21T06:36:05.622307+0000","last_fullsized":"2024-09-21T06:36:05.622307+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:12.748444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.834526+0000","last_change":"2024-09-21T06:34:30.636389+0000","last_active":"2024-09-21T06:36:05.834526+0000","last_peered":"2024-09-21T06:36:05.834526+0000","last_clean":"2024-09-21T06:36:05.834526+0000","last_became_active":"2024-09-21T06:34:30.636008+0000","last_became_peered":"2024-09-21T06:34:30.636008+0000","last_unstale":"2024-09-21T06:36:05.834526+0000","last_undegraded":"2024-09-21T06:36:05.834526+0000","last_fullsized":"2024-09-21T06:36:05.834526+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:29.598926+0000","last_clean_scrub_stamp":"2024-09-21T06:34:29.598926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:45.476267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.219675+0000","last_change":"2024-09-21T06:34:32.635644+0000","last_active":"2024-09-21T06:36:34.219675+0000","last_peered":"2024-09-21T06:36:34.219675+0000","last_clean":"2024-09-21T06:36:34.219675+0000","last_became_active":"2024-09-21T06:34:32.635253+0000","last_became_peered":"2024-09-21T06:34:32.635253+0000","last_unstale":"2024-09-21T06:36:34.219675+0000","last_undegraded":"2024-09-21T06:36:34.219675+0000","last_fullsized":"2024-09-21T06:36:34.219675+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:34.713656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:05.622327+0000","last_change":"2024-09-21T06:34:39.710382+0000","last_active":"2024-09-21T06:36:05.622327+0000","last_peered":"2024-09-21T06:36:05.622327+0000","last_clean":"2024-09-21T06:36:05.622327+0000","last_became_active":"2024-09-21T06:34:39.709884+0000","last_became_peered":"2024-09-21T06:34:39.709884+0000","last_unstale":"2024-09-21T06:36:05.622327+0000","last_undegraded":"2024-09-21T06:36:05.622327+0000","last_fullsized":"2024-09-21T06:36:05.622327+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:38.672853+0000","last_clean_scrub_stamp":"2024-09-21T06:34:38.672853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:01.602366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.234789+0000","last_change":"2024-09-21T06:34:32.640025+0000","last_active":"2024-09-21T06:36:34.234789+0000","last_peered":"2024-09-21T06:36:34.234789+0000","last_clean":"2024-09-21T06:36:34.234789+0000","last_became_active":"2024-09-21T06:34:32.639648+0000","last_became_peered":"2024-09-21T06:34:32.639648+0000","last_unstale":"2024-09-21T06:36:34.234789+0000","last_undegraded":"2024-09-21T06:36:34.234789+0000","last_fullsized":"2024-09-21T06:36:34.234789+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:05.833065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":54,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.230047+0000","last_change":"2024-09-21T06:35:05.115344+0000","last_active":"2024-09-21T06:36:34.230047+0000","last_peered":"2024-09-21T06:36:34.230047+0000","last_clean":"2024-09-21T06:36:34.230047+0000","last_became_active":"2024-09-21T06:35:05.114678+0000","last_became_peered":"2024-09-21T06:35:05.114678+0000","last_unstale":"2024-09-21T06:36:34.230047+0000","last_undegraded":"2024-09-21T06:36:34.230047+0000","last_fullsized":"2024-09-21T06:36:34.230047+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:24.593145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":50,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.229653+0000","last_change":"2024-09-21T06:35:05.115301+0000","last_active":"2024-09-21T06:36:34.229653+0000","last_peered":"2024-09-21T06:36:34.229653+0000","last_clean":"2024-09-21T06:36:34.229653+0000","last_became_active":"2024-09-21T06:35:05.114706+0000","last_became_peered":"2024-09-21T06:35:05.114706+0000","last_unstale":"2024-09-21T06:36:34.229653+0000","last_undegraded":"2024-09-21T06:36:34.229653+0000","last_fullsized":"2024-09-21T06:36:34.229653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:09.513594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.241136+0000","last_change":"2024-09-21T06:34:32.634679+0000","last_active":"2024-09-21T06:36:34.241136+0000","last_peered":"2024-09-21T06:36:34.241136+0000","last_clean":"2024-09-21T06:36:34.241136+0000","last_became_active":"2024-09-21T06:34:32.634285+0000","last_became_peered":"2024-09-21T06:34:32.634285+0000","last_unstale":"2024-09-21T06:36:34.241136+0000","last_undegraded":"2024-09-21T06:36:34.241136+0000","last_fullsized":"2024-09-21T06:36:34.241136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:15.185453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.238589+0000","last_change":"2024-09-21T06:34:32.639037+0000","last_active":"2024-09-21T06:36:34.238589+0000","last_peered":"2024-09-21T06:36:34.238589+0000","last_clean":"2024-09-21T06:36:34.238589+0000","last_became_active":"2024-09-21T06:34:32.638689+0000","last_became_peered":"2024-09-21T06:34:32.638689+0000","last_unstale":"2024-09-21T06:36:34.238589+0000","last_undegraded":"2024-09-21T06:36:34.238589+0000","last_fullsized":"2024-09-21T06:36:34.238589+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:13.609191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.213709+0000","last_change":"2024-09-21T06:35:05.115051+0000","last_active":"2024-09-21T06:36:34.213709+0000","last_peered":"2024-09-21T06:36:34.213709+0000","last_clean":"2024-09-21T06:36:34.213709+0000","last_became_active":"2024-09-21T06:35:05.114602+0000","last_became_peered":"2024-09-21T06:35:05.114602+0000","last_unstale":"2024-09-21T06:36:34.213709+0000","last_undegraded":"2024-09-21T06:36:34.213709+0000","last_fullsized":"2024-09-21T06:36:34.213709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:48.778188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":50,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.233404+0000","last_change":"2024-09-21T06:34:32.637237+0000","last_active":"2024-09-21T06:36:34.233404+0000","last_peered":"2024-09-21T06:36:34.233404+0000","last_clean":"2024-09-21T06:36:34.233404+0000","last_became_active":"2024-09-21T06:34:32.634105+0000","last_became_peered":"2024-09-21T06:34:32.634105+0000","last_unstale":"2024-09-21T06:36:34.233404+0000","last_undegraded":"2024-09-21T06:36:34.233404+0000","last_fullsized":"2024-09-21T06:36:34.233404+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:52.530461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.238241+0000","last_change":"2024-09-21T06:34:32.624441+0000","last_active":"2024-09-21T06:36:34.238241+0000","last_peered":"2024-09-21T06:36:34.238241+0000","last_clean":"2024-09-21T06:36:34.238241+0000","last_became_active":"2024-09-21T06:34:32.623974+0000","last_became_peered":"2024-09-21T06:34:32.623974+0000","last_unstale":"2024-09-21T06:36:34.238241+0000","last_undegraded":"2024-09-21T06:36:34.238241+0000","last_fullsized":"2024-09-21T06:36:34.238241+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:12.378113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.228142+0000","last_change":"2024-09-21T06:34:32.633879+0000","last_active":"2024-09-21T06:36:34.228142+0000","last_peered":"2024-09-21T06:36:34.228142+0000","last_clean":"2024-09-21T06:36:34.228142+0000","last_became_active":"2024-09-21T06:34:32.631791+0000","last_became_peered":"2024-09-21T06:34:32.631791+0000","last_unstale":"2024-09-21T06:36:34.228142+0000","last_undegraded":"2024-09-21T06:36:34.228142+0000","last_fullsized":"2024-09-21T06:36:34.228142+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:46.246892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.238453+0000","last_change":"2024-09-21T06:34:32.639974+0000","last_active":"2024-09-21T06:36:34.238453+0000","last_peered":"2024-09-21T06:36:34.238453+0000","last_clean":"2024-09-21T06:36:34.238453+0000","last_became_active":"2024-09-21T06:34:32.639631+0000","last_became_peered":"2024-09-21T06:34:32.639631+0000","last_unstale":"2024-09-21T06:36:34.238453+0000","last_undegraded":"2024-09-21T06:36:34.238453+0000","last_fullsized":"2024-09-21T06:36:34.238453+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:50.069378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.243809+0000","last_change":"2024-09-21T06:34:32.645403+0000","last_active":"2024-09-21T06:36:34.243809+0000","last_peered":"2024-09-21T06:36:34.243809+0000","last_clean":"2024-09-21T06:36:34.243809+0000","last_became_active":"2024-09-21T06:34:32.644986+0000","last_became_peered":"2024-09-21T06:34:32.644986+0000","last_unstale":"2024-09-21T06:36:34.243809+0000","last_undegraded":"2024-09-21T06:36:34.243809+0000","last_fullsized":"2024-09-21T06:36:34.243809+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:43.640107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.237746+0000","last_change":"2024-09-21T06:34:32.639933+0000","last_active":"2024-09-21T06:36:34.237746+0000","last_peered":"2024-09-21T06:36:34.237746+0000","last_clean":"2024-09-21T06:36:34.237746+0000","last_became_active":"2024-09-21T06:34:32.639438+0000","last_became_peered":"2024-09-21T06:34:32.639438+0000","last_unstale":"2024-09-21T06:36:34.237746+0000","last_undegraded":"2024-09-21T06:36:34.237746+0000","last_fullsized":"2024-09-21T06:36:34.237746+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:18.860219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.226962+0000","last_change":"2024-09-21T06:34:32.648580+0000","last_active":"2024-09-21T06:36:34.226962+0000","last_peered":"2024-09-21T06:36:34.226962+0000","last_clean":"2024-09-21T06:36:34.226962+0000","last_became_active":"2024-09-21T06:34:32.648016+0000","last_became_peered":"2024-09-21T06:34:32.648016+0000","last_unstale":"2024-09-21T06:36:34.226962+0000","last_undegraded":"2024-09-21T06:36:34.226962+0000","last_fullsized":"2024-09-21T06:36:34.226962+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:52.858069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.240213+0000","last_change":"2024-09-21T06:34:32.633376+0000","last_active":"2024-09-21T06:36:34.240213+0000","last_peered":"2024-09-21T06:36:34.240213+0000","last_clean":"2024-09-21T06:36:34.240213+0000","last_became_active":"2024-09-21T06:34:32.631052+0000","last_became_peered":"2024-09-21T06:34:32.631052+0000","last_unstale":"2024-09-21T06:36:34.240213+0000","last_undegraded":"2024-09-21T06:36:34.240213+0000","last_fullsized":"2024-09-21T06:36:34.240213+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:46.229599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.244444+0000","last_change":"2024-09-21T06:34:32.636966+0000","last_active":"2024-09-21T06:36:34.244444+0000","last_peered":"2024-09-21T06:36:34.244444+0000","last_clean":"2024-09-21T06:36:34.244444+0000","last_became_active":"2024-09-21T06:34:32.636565+0000","last_became_peered":"2024-09-21T06:34:32.636565+0000","last_unstale":"2024-09-21T06:36:34.244444+0000","last_undegraded":"2024-09-21T06:36:34.244444+0000","last_fullsized":"2024-09-21T06:36:34.244444+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:36.233082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.246729+0000","last_change":"2024-09-21T06:34:32.646895+0000","last_active":"2024-09-21T06:36:34.246729+0000","last_peered":"2024-09-21T06:36:34.246729+0000","last_clean":"2024-09-21T06:36:34.246729+0000","last_became_active":"2024-09-21T06:34:32.646505+0000","last_became_peered":"2024-09-21T06:34:32.646505+0000","last_unstale":"2024-09-21T06:36:34.246729+0000","last_undegraded":"2024-09-21T06:36:34.246729+0000","last_fullsized":"2024-09-21T06:36:34.246729+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:43.441337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.246723+0000","last_change":"2024-09-21T06:34:32.645402+0000","last_active":"2024-09-21T06:36:34.246723+0000","last_peered":"2024-09-21T06:36:34.246723+0000","last_clean":"2024-09-21T06:36:34.246723+0000","last_became_active":"2024-09-21T06:34:32.645013+0000","last_became_peered":"2024-09-21T06:34:32.645013+0000","last_unstale":"2024-09-21T06:36:34.246723+0000","last_undegraded":"2024-09-21T06:36:34.246723+0000","last_fullsized":"2024-09-21T06:36:34.246723+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:33.475847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.240830+0000","last_change":"2024-09-21T06:34:32.643234+0000","last_active":"2024-09-21T06:36:34.240830+0000","last_peered":"2024-09-21T06:36:34.240830+0000","last_clean":"2024-09-21T06:36:34.240830+0000","last_became_active":"2024-09-21T06:34:32.642614+0000","last_became_peered":"2024-09-21T06:34:32.642614+0000","last_unstale":"2024-09-21T06:36:34.240830+0000","last_undegraded":"2024-09-21T06:36:34.240830+0000","last_fullsized":"2024-09-21T06:36:34.240830+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:50.903500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.242716+0000","last_change":"2024-09-21T06:34:32.641887+0000","last_active":"2024-09-21T06:36:34.242716+0000","last_peered":"2024-09-21T06:36:34.242716+0000","last_clean":"2024-09-21T06:36:34.242716+0000","last_became_active":"2024-09-21T06:34:32.641490+0000","last_became_peered":"2024-09-21T06:34:32.641490+0000","last_unstale":"2024-09-21T06:36:34.242716+0000","last_undegraded":"2024-09-21T06:36:34.242716+0000","last_fullsized":"2024-09-21T06:36:34.242716+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:34.550970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.234750+0000","last_change":"2024-09-21T06:34:32.641865+0000","last_active":"2024-09-21T06:36:34.234750+0000","last_peered":"2024-09-21T06:36:34.234750+0000","last_clean":"2024-09-21T06:36:34.234750+0000","last_became_active":"2024-09-21T06:34:32.641252+0000","last_became_peered":"2024-09-21T06:34:32.641252+0000","last_unstale":"2024-09-21T06:36:34.234750+0000","last_undegraded":"2024-09-21T06:36:34.234750+0000","last_fullsized":"2024-09-21T06:36:34.234750+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:44.459202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.236389+0000","last_change":"2024-09-21T06:34:32.641259+0000","last_active":"2024-09-21T06:36:34.236389+0000","last_peered":"2024-09-21T06:36:34.236389+0000","last_clean":"2024-09-21T06:36:34.236389+0000","last_became_active":"2024-09-21T06:34:32.640907+0000","last_became_peered":"2024-09-21T06:34:32.640907+0000","last_unstale":"2024-09-21T06:36:34.236389+0000","last_undegraded":"2024-09-21T06:36:34.236389+0000","last_fullsized":"2024-09-21T06:36:34.236389+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:59.966857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.246472+0000","last_change":"2024-09-21T06:34:32.646095+0000","last_active":"2024-09-21T06:36:34.246472+0000","last_peered":"2024-09-21T06:36:34.246472+0000","last_clean":"2024-09-21T06:36:34.246472+0000","last_became_active":"2024-09-21T06:34:32.645604+0000","last_became_peered":"2024-09-21T06:34:32.645604+0000","last_unstale":"2024-09-21T06:36:34.246472+0000","last_undegraded":"2024-09-21T06:36:34.246472+0000","last_fullsized":"2024-09-21T06:36:34.246472+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:30.872428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":50,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.235849+0000","last_change":"2024-09-21T06:34:32.649303+0000","last_active":"2024-09-21T06:36:34.235849+0000","last_peered":"2024-09-21T06:36:34.235849+0000","last_clean":"2024-09-21T06:36:34.235849+0000","last_became_active":"2024-09-21T06:34:32.648965+0000","last_became_peered":"2024-09-21T06:34:32.648965+0000","last_unstale":"2024-09-21T06:36:34.235849+0000","last_undegraded":"2024-09-21T06:36:34.235849+0000","last_fullsized":"2024-09-21T06:36:34.235849+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:51.335949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.247103+0000","last_change":"2024-09-21T06:34:32.637261+0000","last_active":"2024-09-21T06:36:34.247103+0000","last_peered":"2024-09-21T06:36:34.247103+0000","last_clean":"2024-09-21T06:36:34.247103+0000","last_became_active":"2024-09-21T06:34:32.636861+0000","last_became_peered":"2024-09-21T06:34:32.636861+0000","last_unstale":"2024-09-21T06:36:34.247103+0000","last_undegraded":"2024-09-21T06:36:34.247103+0000","last_fullsized":"2024-09-21T06:36:34.247103+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:55.271027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.237027+0000","last_change":"2024-09-21T06:34:32.635156+0000","last_active":"2024-09-21T06:36:34.237027+0000","last_peered":"2024-09-21T06:36:34.237027+0000","last_clean":"2024-09-21T06:36:34.237027+0000","last_became_active":"2024-09-21T06:34:32.634802+0000","last_became_peered":"2024-09-21T06:34:32.634802+0000","last_unstale":"2024-09-21T06:36:34.237027+0000","last_undegraded":"2024-09-21T06:36:34.237027+0000","last_fullsized":"2024-09-21T06:36:34.237027+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:16.139829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.228891+0000","last_change":"2024-09-21T06:34:32.637509+0000","last_active":"2024-09-21T06:36:34.228891+0000","last_peered":"2024-09-21T06:36:34.228891+0000","last_clean":"2024-09-21T06:36:34.228891+0000","last_became_active":"2024-09-21T06:34:32.637136+0000","last_became_peered":"2024-09-21T06:34:32.637136+0000","last_unstale":"2024-09-21T06:36:34.228891+0000","last_undegraded":"2024-09-21T06:36:34.228891+0000","last_fullsized":"2024-09-21T06:36:34.228891+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:48.088988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.233952+0000","last_change":"2024-09-21T06:35:05.114819+0000","last_active":"2024-09-21T06:36:34.233952+0000","last_peered":"2024-09-21T06:36:34.233952+0000","last_clean":"2024-09-21T06:36:34.233952+0000","last_became_active":"2024-09-21T06:35:05.114424+0000","last_became_peered":"2024-09-21T06:35:05.114424+0000","last_unstale":"2024-09-21T06:36:34.233952+0000","last_undegraded":"2024-09-21T06:36:34.233952+0000","last_fullsized":"2024-09-21T06:36:34.233952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:59.323643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.225869+0000","last_change":"2024-09-21T06:34:32.636103+0000","last_active":"2024-09-21T06:36:34.225869+0000","last_peered":"2024-09-21T06:36:34.225869+0000","last_clean":"2024-09-21T06:36:34.225869+0000","last_became_active":"2024-09-21T06:34:32.635728+0000","last_became_peered":"2024-09-21T06:34:32.635728+0000","last_unstale":"2024-09-21T06:36:34.225869+0000","last_undegraded":"2024-09-21T06:36:34.225869+0000","last_fullsized":"2024-09-21T06:36:34.225869+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:16.731402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.235424+0000","last_change":"2024-09-21T06:34:32.635654+0000","last_active":"2024-09-21T06:36:34.235424+0000","last_peered":"2024-09-21T06:36:34.235424+0000","last_clean":"2024-09-21T06:36:34.235424+0000","last_became_active":"2024-09-21T06:34:32.635265+0000","last_became_peered":"2024-09-21T06:34:32.635265+0000","last_unstale":"2024-09-21T06:36:34.235424+0000","last_undegraded":"2024-09-21T06:36:34.235424+0000","last_fullsized":"2024-09-21T06:36:34.235424+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:28.169625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.246376+0000","last_change":"2024-09-21T06:34:32.644640+0000","last_active":"2024-09-21T06:36:34.246376+0000","last_peered":"2024-09-21T06:36:34.246376+0000","last_clean":"2024-09-21T06:36:34.246376+0000","last_became_active":"2024-09-21T06:34:32.644334+0000","last_became_peered":"2024-09-21T06:34:32.644334+0000","last_unstale":"2024-09-21T06:36:34.246376+0000","last_undegraded":"2024-09-21T06:36:34.246376+0000","last_fullsized":"2024-09-21T06:36:34.246376+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:25.572178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.242912+0000","last_change":"2024-09-21T06:34:32.643705+0000","last_active":"2024-09-21T06:36:34.242912+0000","last_peered":"2024-09-21T06:36:34.242912+0000","last_clean":"2024-09-21T06:36:34.242912+0000","last_became_active":"2024-09-21T06:34:32.642705+0000","last_became_peered":"2024-09-21T06:34:32.642705+0000","last_unstale":"2024-09-21T06:36:34.242912+0000","last_undegraded":"2024-09-21T06:36:34.242912+0000","last_fullsized":"2024-09-21T06:36:34.242912+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:22.542559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"85'8","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T06:36:34.246617+0000","last_change":"2024-09-21T06:34:32.646539+0000","last_active":"2024-09-21T06:36:34.246617+0000","last_peered":"2024-09-21T06:36:34.246617+0000","last_clean":"2024-09-21T06:36:34.246617+0000","last_became_active":"2024-09-21T06:34:32.645992+0000","last_became_peered":"2024-09-21T06:34:32.645992+0000","last_unstale":"2024-09-21T06:36:34.246617+0000","last_undegraded":"2024-09-21T06:36:34.246617+0000","last_fullsized":"2024-09-21T06:36:34.246617+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:34:31.613076+0000","last_clean_scrub_stamp":"2024-09-21T06:34:31.613076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:38.856001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69832,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":218,"num_read_kb":179,"num_write":90,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":209989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282016,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743466,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":485699,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939694,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":1141199,"data_compressed":65460,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103220,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":490377,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299449,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":486303,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495678,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":987136,"data_stored":1080745,"data_compressed":65460,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659204,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":481603,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855433,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":1076249,"data_compressed":65460,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018958,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":485721,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215186,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":933888,"data_stored":481707,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":550671,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":550637,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":929792,"data_stored":481693,"data_compressed":29599,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35861,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":69068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:36:56.592 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:36:56.856 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:36:57.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.333+0000 7f9153a4a640 1 -- 172.21.15.67:0/3089112558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f914c1083c0 msgr2=0x7f914c1087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:57.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.333+0000 7f9153a4a640 1 --2- 172.21.15.67:0/3089112558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f914c1083c0 0x7f914c1087d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f913c009900 tx=0x7f913c02a410 comp rx=0 tx=0).stop 2024-09-21T06:36:57.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.334+0000 7f9153a4a640 1 -- 172.21.15.67:0/3089112558 shutdown_connections 2024-09-21T06:36:57.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.334+0000 7f9153a4a640 1 --2- 172.21.15.67:0/3089112558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f914c108d10 0x7f914c10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:57.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.334+0000 7f9153a4a640 1 --2- 172.21.15.67:0/3089112558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f914c1083c0 0x7f914c1087d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:57.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.334+0000 7f9153a4a640 1 --2- 172.21.15.67:0/3089112558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f914c102a80 0x7f914c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:57.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.334+0000 7f9153a4a640 1 -- 172.21.15.67:0/3089112558 >> 172.21.15.67:0/3089112558 conn(0x7f914c079740 msgr2=0x7f914c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:57.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.334+0000 7f9153a4a640 1 -- 172.21.15.67:0/3089112558 shutdown_connections 2024-09-21T06:36:57.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.334+0000 7f9153a4a640 1 -- 172.21.15.67:0/3089112558 wait complete. 2024-09-21T06:36:57.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.334+0000 7f9153a4a640 1 Processor -- start 2024-09-21T06:36:57.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.335+0000 7f9153a4a640 1 -- start start 2024-09-21T06:36:57.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.335+0000 7f9153a4a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f914c102a80 0x7f914c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:57.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.335+0000 7f9153a4a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f914c1083c0 0x7f914c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:57.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.335+0000 7f9153a4a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f914c108d10 0x7f914c19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:57.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.335+0000 7f9153a4a640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f914c112830 con 0x7f914c1083c0 2024-09-21T06:36:57.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.335+0000 7f9153a4a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f914c1126b0 con 0x7f914c108d10 2024-09-21T06:36:57.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.335+0000 7f9153a4a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f914c1129b0 con 0x7f914c102a80 2024-09-21T06:36:57.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.335+0000 7f9150fe5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f914c1083c0 0x7f914c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:57.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.335+0000 7f9150fe5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f914c1083c0 0x7f914c19aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54708/0 (socket says 172.21.15.67:54708) 2024-09-21T06:36:57.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.335+0000 7f9150fe5640 1 -- 172.21.15.67:0/1828643971 learned_addr learned my addr 172.21.15.67:0/1828643971 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:57.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.335+0000 7f91517e6640 1 --2- 172.21.15.67:0/1828643971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f914c102a80 0x7f914c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:57.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.335+0000 7f9151fe7640 1 --2- 172.21.15.67:0/1828643971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f914c108d10 0x7f914c19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:57.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.335+0000 7f9150fe5640 1 -- 172.21.15.67:0/1828643971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f914c102a80 msgr2=0x7f914c19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:57.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.335+0000 7f9150fe5640 1 --2- 172.21.15.67:0/1828643971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f914c102a80 0x7f914c19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:57.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.335+0000 7f9150fe5640 1 -- 172.21.15.67:0/1828643971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f914c108d10 msgr2=0x7f914c19fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:57.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.335+0000 7f9150fe5640 1 --2- 172.21.15.67:0/1828643971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f914c108d10 0x7f914c19fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:57.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.335+0000 7f9150fe5640 1 -- 172.21.15.67:0/1828643971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f914c1a0400 con 0x7f914c1083c0 2024-09-21T06:36:57.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.335+0000 7f9151fe7640 1 --2- 172.21.15.67:0/1828643971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f914c108d10 0x7f914c19fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:36:57.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.336+0000 7f9150fe5640 1 --2- 172.21.15.67:0/1828643971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f914c1083c0 0x7f914c19aa30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f913c005940 tx=0x7f913c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:57.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.336+0000 7f91427fc640 1 -- 172.21.15.67:0/1828643971 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913c03f030 con 0x7f914c1083c0 2024-09-21T06:36:57.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.336+0000 7f91427fc640 1 -- 172.21.15.67:0/1828643971 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f913c003c90 con 0x7f914c1083c0 2024-09-21T06:36:57.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.336+0000 7f9153a4a640 1 -- 172.21.15.67:0/1828643971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f914c1a0690 con 0x7f914c1083c0 2024-09-21T06:36:57.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.336+0000 7f91427fc640 1 -- 172.21.15.67:0/1828643971 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913c035070 con 0x7f914c1083c0 2024-09-21T06:36:57.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.336+0000 7f9153a4a640 1 -- 172.21.15.67:0/1828643971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f914c1a6e50 con 0x7f914c1083c0 2024-09-21T06:36:57.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.337+0000 7f9153a4a640 1 -- 172.21.15.67:0/1828643971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f914c103d00 con 0x7f914c1083c0 2024-09-21T06:36:57.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.338+0000 7f91427fc640 1 -- 172.21.15.67:0/1828643971 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f913c042030 con 0x7f914c1083c0 2024-09-21T06:36:57.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.339+0000 7f91427fc640 1 --2- 172.21.15.67:0/1828643971 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f91380b6590 0x7f91380b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:57.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.339+0000 7f91517e6640 1 --2- 172.21.15.67:0/1828643971 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f91380b6590 0x7f91380b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:57.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.339+0000 7f91427fc640 1 -- 172.21.15.67:0/1828643971 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f913c107f80 con 0x7f914c1083c0 2024-09-21T06:36:57.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.340+0000 7f91517e6640 1 --2- 172.21.15.67:0/1828643971 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f91380b6590 0x7f91380b8aa0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f914800bbe0 tx=0x7f914800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:57.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.342+0000 7f91427fc640 1 -- 172.21.15.67:0/1828643971 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f913c0ce440 con 0x7f914c1083c0 2024-09-21T06:36:57.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:57 smithi067 ceph-mon[31232]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:36:57.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:57 smithi067 ceph-mon[31232]: pgmap v371: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-21T06:36:57.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.568+0000 7f9153a4a640 1 -- 172.21.15.67:0/1828643971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f914c1a7020 con 0x7f914c1083c0 2024-09-21T06:36:57.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.569+0000 7f91427fc640 1 -- 172.21.15.67:0/1828643971 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7f913c0d2820 con 0x7f914c1083c0 2024-09-21T06:36:57.572 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:36:57.572 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:36:57.572 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:36:57.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.571+0000 7f9153a4a640 1 -- 172.21.15.67:0/1828643971 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f91380b6590 msgr2=0x7f91380b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:57.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.571+0000 7f9153a4a640 1 --2- 172.21.15.67:0/1828643971 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f91380b6590 0x7f91380b8aa0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f914800bbe0 tx=0x7f914800b390 comp rx=0 tx=0).stop 2024-09-21T06:36:57.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.571+0000 7f9153a4a640 1 -- 172.21.15.67:0/1828643971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f914c1083c0 msgr2=0x7f914c19aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:57.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.571+0000 7f9153a4a640 1 --2- 172.21.15.67:0/1828643971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f914c1083c0 0x7f914c19aa30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f913c005940 tx=0x7f913c0042f0 comp rx=0 tx=0).stop 2024-09-21T06:36:57.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.571+0000 7f9153a4a640 1 -- 172.21.15.67:0/1828643971 shutdown_connections 2024-09-21T06:36:57.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.571+0000 7f9153a4a640 1 --2- 172.21.15.67:0/1828643971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f914c108d10 0x7f914c19fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:57.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.571+0000 7f9153a4a640 1 --2- 172.21.15.67:0/1828643971 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f91380b6590 0x7f91380b8aa0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:57.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.571+0000 7f9153a4a640 1 --2- 172.21.15.67:0/1828643971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f914c1083c0 0x7f914c19aa30 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:57.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.571+0000 7f9153a4a640 1 --2- 172.21.15.67:0/1828643971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f914c102a80 0x7f914c19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:57.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.571+0000 7f9153a4a640 1 -- 172.21.15.67:0/1828643971 >> 172.21.15.67:0/1828643971 conn(0x7f914c079740 msgr2=0x7f914c101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:57.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.571+0000 7f9153a4a640 1 -- 172.21.15.67:0/1828643971 shutdown_connections 2024-09-21T06:36:57.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:57.571+0000 7f9153a4a640 1 -- 172.21.15.67:0/1828643971 wait complete. 2024-09-21T06:36:57.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:57 smithi100 ceph-mon[33401]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:36:57.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:57 smithi100 ceph-mon[33401]: pgmap v371: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-21T06:36:57.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:57 smithi203 ceph-mon[33735]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:36:57.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:57 smithi203 ceph-mon[33735]: pgmap v371: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-21T06:36:58.263 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:36:58.522 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:36:58.541 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:58 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1828643971' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:36:58.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:58 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1828643971' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:36:58.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:58 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1828643971' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:36:59.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.019+0000 7fc8e6597640 1 -- 172.21.15.67:0/3897171711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8e0104840 msgr2=0x7fc8e0104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:59.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.019+0000 7fc8e6597640 1 --2- 172.21.15.67:0/3897171711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8e0104840 0x7fc8e0104c50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009900 tx=0x7fc8d002a3f0 comp rx=0 tx=0).stop 2024-09-21T06:36:59.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.019+0000 7fc8e6597640 1 -- 172.21.15.67:0/3897171711 shutdown_connections 2024-09-21T06:36:59.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.019+0000 7fc8e6597640 1 --2- 172.21.15.67:0/3897171711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8e0105270 0x7fc8e010d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:59.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.019+0000 7fc8e6597640 1 --2- 172.21.15.67:0/3897171711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8e010ddf0 0x7fc8e0110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:59.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.019+0000 7fc8e6597640 1 --2- 172.21.15.67:0/3897171711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8e0104840 0x7fc8e0104c50 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:59.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.019+0000 7fc8e6597640 1 -- 172.21.15.67:0/3897171711 >> 172.21.15.67:0/3897171711 conn(0x7fc8e0079730 msgr2=0x7fc8e0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:59.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.019+0000 7fc8e6597640 1 -- 172.21.15.67:0/3897171711 shutdown_connections 2024-09-21T06:36:59.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.019+0000 7fc8e6597640 1 -- 172.21.15.67:0/3897171711 wait complete. 2024-09-21T06:36:59.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.020+0000 7fc8e6597640 1 Processor -- start 2024-09-21T06:36:59.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.020+0000 7fc8e6597640 1 -- start start 2024-09-21T06:36:59.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.020+0000 7fc8e6597640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8e0104840 0x7fc8e019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:59.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.020+0000 7fc8e6597640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8e0105270 0x7fc8e019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:59.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.020+0000 7fc8e6597640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8e010ddf0 0x7fc8e019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:59.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.020+0000 7fc8e6597640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e0113940 con 0x7fc8e010ddf0 2024-09-21T06:36:59.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.020+0000 7fc8e6597640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e01137c0 con 0x7fc8e0105270 2024-09-21T06:36:59.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.020+0000 7fc8e6597640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e0113ac0 con 0x7fc8e0104840 2024-09-21T06:36:59.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.020+0000 7fc8e4b34640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8e010ddf0 0x7fc8e019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:59.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.020+0000 7fc8e4b34640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8e010ddf0 0x7fc8e019fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54024/0 (socket says 172.21.15.67:54024) 2024-09-21T06:36:59.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.020+0000 7fc8e4b34640 1 -- 172.21.15.67:0/1271364058 learned_addr learned my addr 172.21.15.67:0/1271364058 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:36:59.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.020+0000 7fc8d7fff640 1 --2- 172.21.15.67:0/1271364058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8e0105270 0x7fc8e019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:59.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.020+0000 7fc8dffff640 1 --2- 172.21.15.67:0/1271364058 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8e0104840 0x7fc8e019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:59.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.021+0000 7fc8e4b34640 1 -- 172.21.15.67:0/1271364058 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8e0104840 msgr2=0x7fc8e019a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:59.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.021+0000 7fc8e4b34640 1 --2- 172.21.15.67:0/1271364058 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8e0104840 0x7fc8e019a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:59.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.021+0000 7fc8e4b34640 1 -- 172.21.15.67:0/1271364058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8e0105270 msgr2=0x7fc8e019a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:59.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.021+0000 7fc8e4b34640 1 --2- 172.21.15.67:0/1271364058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8e0105270 0x7fc8e019a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:59.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.021+0000 7fc8e4b34640 1 -- 172.21.15.67:0/1271364058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e01a02c0 con 0x7fc8e010ddf0 2024-09-21T06:36:59.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.021+0000 7fc8e4b34640 1 --2- 172.21.15.67:0/1271364058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8e010ddf0 0x7fc8e019fc50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800cad0 tx=0x7fc8d800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:59.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.021+0000 7fc8ddffb640 1 -- 172.21.15.67:0/1271364058 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d800ad60 con 0x7fc8e010ddf0 2024-09-21T06:36:59.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.021+0000 7fc8ddffb640 1 -- 172.21.15.67:0/1271364058 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8d8005020 con 0x7fc8e010ddf0 2024-09-21T06:36:59.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.021+0000 7fc8e6597640 1 -- 172.21.15.67:0/1271364058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e01a05b0 con 0x7fc8e010ddf0 2024-09-21T06:36:59.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.021+0000 7fc8ddffb640 1 -- 172.21.15.67:0/1271364058 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d800f6b0 con 0x7fc8e010ddf0 2024-09-21T06:36:59.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.021+0000 7fc8e6597640 1 -- 172.21.15.67:0/1271364058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e01a6dd0 con 0x7fc8e010ddf0 2024-09-21T06:36:59.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.022+0000 7fc8e6597640 1 -- 172.21.15.67:0/1271364058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8e0104c50 con 0x7fc8e010ddf0 2024-09-21T06:36:59.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.024+0000 7fc8ddffb640 1 -- 172.21.15.67:0/1271364058 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fc8d80051d0 con 0x7fc8e010ddf0 2024-09-21T06:36:59.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.024+0000 7fc8ddffb640 1 --2- 172.21.15.67:0/1271364058 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc8cc0b6540 0x7fc8cc0b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:36:59.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.024+0000 7fc8dffff640 1 --2- 172.21.15.67:0/1271364058 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc8cc0b6540 0x7fc8cc0b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:36:59.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.024+0000 7fc8ddffb640 1 -- 172.21.15.67:0/1271364058 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fc8d80e7960 con 0x7fc8e010ddf0 2024-09-21T06:36:59.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.025+0000 7fc8dffff640 1 --2- 172.21.15.67:0/1271364058 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc8cc0b6540 0x7fc8cc0b8a50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0005ec0 tx=0x7fc8d00058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:36:59.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.028+0000 7fc8ddffb640 1 -- 172.21.15.67:0/1271364058 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc8d8014030 con 0x7fc8e010ddf0 2024-09-21T06:36:59.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.252+0000 7fc8e6597640 1 -- 172.21.15.67:0/1271364058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8e019b540 con 0x7fc8e010ddf0 2024-09-21T06:36:59.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.253+0000 7fc8ddffb640 1 -- 172.21.15.67:0/1271364058 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7fc8e019b540 con 0x7fc8e010ddf0 2024-09-21T06:36:59.256 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:36:59.256 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:36:59.256 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:36:59.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.255+0000 7fc8e6597640 1 -- 172.21.15.67:0/1271364058 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc8cc0b6540 msgr2=0x7fc8cc0b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:59.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.255+0000 7fc8e6597640 1 --2- 172.21.15.67:0/1271364058 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc8cc0b6540 0x7fc8cc0b8a50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0005ec0 tx=0x7fc8d00058d0 comp rx=0 tx=0).stop 2024-09-21T06:36:59.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.255+0000 7fc8e6597640 1 -- 172.21.15.67:0/1271364058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8e010ddf0 msgr2=0x7fc8e019fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:36:59.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.255+0000 7fc8e6597640 1 --2- 172.21.15.67:0/1271364058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8e010ddf0 0x7fc8e019fc50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800cad0 tx=0x7fc8d800ced0 comp rx=0 tx=0).stop 2024-09-21T06:36:59.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.255+0000 7fc8e6597640 1 -- 172.21.15.67:0/1271364058 shutdown_connections 2024-09-21T06:36:59.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.255+0000 7fc8e6597640 1 --2- 172.21.15.67:0/1271364058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8e0105270 0x7fc8e019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:59.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.255+0000 7fc8e6597640 1 --2- 172.21.15.67:0/1271364058 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc8cc0b6540 0x7fc8cc0b8a50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:59.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.255+0000 7fc8e6597640 1 --2- 172.21.15.67:0/1271364058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8e010ddf0 0x7fc8e019fc50 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:59.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.255+0000 7fc8e6597640 1 --2- 172.21.15.67:0/1271364058 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8e0104840 0x7fc8e019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:36:59.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.255+0000 7fc8e6597640 1 -- 172.21.15.67:0/1271364058 >> 172.21.15.67:0/1271364058 conn(0x7fc8e0079730 msgr2=0x7fc8e0100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:36:59.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.256+0000 7fc8e6597640 1 -- 172.21.15.67:0/1271364058 shutdown_connections 2024-09-21T06:36:59.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:36:59.256+0000 7fc8e6597640 1 -- 172.21.15.67:0/1271364058 wait complete. 2024-09-21T06:36:59.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:59 smithi067 ceph-mon[31232]: pgmap v372: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:36:59.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:36:59 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1271364058' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:36:59.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:59 smithi100 ceph-mon[33401]: pgmap v372: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:36:59.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:36:59 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1271364058' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:36:59.917 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:36:59.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:59 smithi203 ceph-mon[33735]: pgmap v372: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:36:59.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:36:59 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1271364058' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:37:00.178 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:37:00.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.736+0000 7efebef12640 1 -- 172.21.15.67:0/2400570391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeb81052f0 msgr2=0x7efeb810d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:00.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.736+0000 7efebef12640 1 --2- 172.21.15.67:0/2400570391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeb81052f0 0x7efeb810d930 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7efea8009900 tx=0x7efea802a410 comp rx=0 tx=0).stop 2024-09-21T06:37:00.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.736+0000 7efebef12640 1 -- 172.21.15.67:0/2400570391 shutdown_connections 2024-09-21T06:37:00.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.736+0000 7efebef12640 1 --2- 172.21.15.67:0/2400570391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efeb810de70 0x7efeb8110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:00.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.736+0000 7efebef12640 1 --2- 172.21.15.67:0/2400570391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeb81052f0 0x7efeb810d930 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:00.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.736+0000 7efebef12640 1 --2- 172.21.15.67:0/2400570391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeb81048c0 0x7efeb8104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:00.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.736+0000 7efebef12640 1 -- 172.21.15.67:0/2400570391 >> 172.21.15.67:0/2400570391 conn(0x7efeb8079740 msgr2=0x7efeb8079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:37:00.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.736+0000 7efebef12640 1 -- 172.21.15.67:0/2400570391 shutdown_connections 2024-09-21T06:37:00.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.736+0000 7efebef12640 1 -- 172.21.15.67:0/2400570391 wait complete. 2024-09-21T06:37:00.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.736+0000 7efebef12640 1 Processor -- start 2024-09-21T06:37:00.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.736+0000 7efebef12640 1 -- start start 2024-09-21T06:37:00.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.737+0000 7efebef12640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeb81048c0 0x7efeb81027d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:00.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.737+0000 7efebef12640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efeb81052f0 0x7efeb8102d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:00.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.737+0000 7efebef12640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeb810de70 0x7efeb8103250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:00.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.737+0000 7efebef12640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efeb8113930 con 0x7efeb810de70 2024-09-21T06:37:00.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.737+0000 7efebef12640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efeb81137b0 con 0x7efeb81048c0 2024-09-21T06:37:00.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.737+0000 7efebef12640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7efeb8113ab0 con 0x7efeb81052f0 2024-09-21T06:37:00.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.737+0000 7efebd4af640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeb810de70 0x7efeb8103250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:00.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.737+0000 7efebd4af640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeb810de70 0x7efeb8103250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54048/0 (socket says 172.21.15.67:54048) 2024-09-21T06:37:00.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.737+0000 7efebd4af640 1 -- 172.21.15.67:0/927183257 learned_addr learned my addr 172.21.15.67:0/927183257 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:37:00.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.737+0000 7efeaffff640 1 --2- 172.21.15.67:0/927183257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efeb81052f0 0x7efeb8102d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:00.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.737+0000 7efebccae640 1 --2- 172.21.15.67:0/927183257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeb81048c0 0x7efeb81027d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:00.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.737+0000 7efebd4af640 1 -- 172.21.15.67:0/927183257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efeb81052f0 msgr2=0x7efeb8102d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:00.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.737+0000 7efebd4af640 1 --2- 172.21.15.67:0/927183257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efeb81052f0 0x7efeb8102d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:00.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.737+0000 7efebd4af640 1 -- 172.21.15.67:0/927183257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeb81048c0 msgr2=0x7efeb81027d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.737+0000 7efebd4af640 1 --2- 172.21.15.67:0/927183257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeb81048c0 0x7efeb81027d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.737+0000 7efebd4af640 1 -- 172.21.15.67:0/927183257 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb81a8670 con 0x7efeb810de70 2024-09-21T06:37:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.737+0000 7efebd4af640 1 --2- 172.21.15.67:0/927183257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeb810de70 0x7efeb8103250 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7efeb000eaa0 tx=0x7efeb000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:37:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.738+0000 7efeadffb640 1 -- 172.21.15.67:0/927183257 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb000cdb0 con 0x7efeb810de70 2024-09-21T06:37:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.738+0000 7efeadffb640 1 -- 172.21.15.67:0/927183257 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efeb0005020 con 0x7efeb810de70 2024-09-21T06:37:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.738+0000 7efeadffb640 1 -- 172.21.15.67:0/927183257 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb00116d0 con 0x7efeb810de70 2024-09-21T06:37:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.738+0000 7efebef12640 1 -- 172.21.15.67:0/927183257 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb81a8960 con 0x7efeb810de70 2024-09-21T06:37:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.738+0000 7efebef12640 1 -- 172.21.15.67:0/927183257 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb81a8e70 con 0x7efeb810de70 2024-09-21T06:37:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.739+0000 7efebef12640 1 -- 172.21.15.67:0/927183257 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeb806c180 con 0x7efeb810de70 2024-09-21T06:37:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.741+0000 7efeadffb640 1 -- 172.21.15.67:0/927183257 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7efeb00051d0 con 0x7efeb810de70 2024-09-21T06:37:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.742+0000 7efeadffb640 1 --2- 172.21.15.67:0/927183257 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7efea00b6510 0x7efea00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.742+0000 7efebccae640 1 --2- 172.21.15.67:0/927183257 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7efea00b6510 0x7efea00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.742+0000 7efeadffb640 1 -- 172.21.15.67:0/927183257 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7efeb00e7b10 con 0x7efeb810de70 2024-09-21T06:37:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.743+0000 7efebccae640 1 --2- 172.21.15.67:0/927183257 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7efea00b6510 0x7efea00b8a20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efeb8104220 tx=0x7efeb4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:37:00.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.745+0000 7efeadffb640 1 -- 172.21.15.67:0/927183257 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efeb0016030 con 0x7efeb810de70 2024-09-21T06:37:00.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.935+0000 7efebef12640 1 -- 172.21.15.67:0/927183257 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efeb8104cd0 con 0x7efeb810de70 2024-09-21T06:37:00.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.937+0000 7efeadffb640 1 -- 172.21.15.67:0/927183257 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24954 (secure 0 0 0) 0x7efeb00b23b0 con 0x7efeb810de70 2024-09-21T06:37:00.940 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:37:00.940 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":85,"fsid":"2a791768-77e2-11ef-baf5-efdc52797490","created":"2024-09-21T06:26:15.222283+0000","modified":"2024-09-21T06:36:05.617384+0000","last_up_change":"2024-09-21T06:33:40.886310+0000","last_in_change":"2024-09-21T06:33:17.339486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:29:50.212825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:34:29.082881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:34:31.301094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:34:37.681356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"18778364-82ed-41c0-92da-ce44cdad7358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6803","nonce":2310759946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6805","nonce":2310759946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6809","nonce":2310759946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6807","nonce":2310759946}]},"public_addr":"172.21.15.67:6803/2310759946","cluster_addr":"172.21.15.67:6805/2310759946","heartbeat_back_addr":"172.21.15.67:6809/2310759946","heartbeat_front_addr":"172.21.15.67:6807/2310759946","state":["exists","up"]},{"osd":1,"uuid":"84e8d0c2-1c45-4673-ad49-f61df667d7f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6801","nonce":161807364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6803","nonce":161807364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6807","nonce":161807364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6805","nonce":161807364}]},"public_addr":"172.21.15.100:6801/161807364","cluster_addr":"172.21.15.100:6803/161807364","heartbeat_back_addr":"172.21.15.100:6807/161807364","heartbeat_front_addr":"172.21.15.100:6805/161807364","state":["exists","up"]},{"osd":2,"uuid":"e9a4d919-a779-4a1b-9b5b-d03a65f28ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6801","nonce":2754787169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6803","nonce":2754787169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6807","nonce":2754787169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6805","nonce":2754787169}]},"public_addr":"172.21.15.203:6801/2754787169","cluster_addr":"172.21.15.203:6803/2754787169","heartbeat_back_addr":"172.21.15.203:6807/2754787169","heartbeat_front_addr":"172.21.15.203:6805/2754787169","state":["exists","up"]},{"osd":3,"uuid":"3f7d9abd-587c-47ad-bf43-11f7dcdf71c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6811","nonce":4007517951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6813","nonce":4007517951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6817","nonce":4007517951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6815","nonce":4007517951}]},"public_addr":"172.21.15.67:6811/4007517951","cluster_addr":"172.21.15.67:6813/4007517951","heartbeat_back_addr":"172.21.15.67:6817/4007517951","heartbeat_front_addr":"172.21.15.67:6815/4007517951","state":["exists","up"]},{"osd":4,"uuid":"94f3a075-6d8a-4907-8418-a8513b35779a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6809","nonce":3773646501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6811","nonce":3773646501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6815","nonce":3773646501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6813","nonce":3773646501}]},"public_addr":"172.21.15.100:6809/3773646501","cluster_addr":"172.21.15.100:6811/3773646501","heartbeat_back_addr":"172.21.15.100:6815/3773646501","heartbeat_front_addr":"172.21.15.100:6813/3773646501","state":["exists","up"]},{"osd":5,"uuid":"5d842edf-06ad-49ae-ac71-863e5657270c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6809","nonce":2096171166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6811","nonce":2096171166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6815","nonce":2096171166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6813","nonce":2096171166}]},"public_addr":"172.21.15.203:6809/2096171166","cluster_addr":"172.21.15.203:6811/2096171166","heartbeat_back_addr":"172.21.15.203:6815/2096171166","heartbeat_front_addr":"172.21.15.203:6813/2096171166","state":["exists","up"]},{"osd":6,"uuid":"85b9beed-c93d-48ba-bc7a-c4cd816fbdff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6819","nonce":3229936280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6821","nonce":3229936280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6825","nonce":3229936280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6823","nonce":3229936280}]},"public_addr":"172.21.15.67:6819/3229936280","cluster_addr":"172.21.15.67:6821/3229936280","heartbeat_back_addr":"172.21.15.67:6825/3229936280","heartbeat_front_addr":"172.21.15.67:6823/3229936280","state":["exists","up"]},{"osd":7,"uuid":"86de30c2-b8ee-4c9d-9c17-8a56517934ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6817","nonce":1622917909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6819","nonce":1622917909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6823","nonce":1622917909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6821","nonce":1622917909}]},"public_addr":"172.21.15.100:6817/1622917909","cluster_addr":"172.21.15.100:6819/1622917909","heartbeat_back_addr":"172.21.15.100:6823/1622917909","heartbeat_front_addr":"172.21.15.100:6821/1622917909","state":["exists","up"]},{"osd":8,"uuid":"0e96cdad-a135-4115-9e2d-9a28306700e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6817","nonce":3452445943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6819","nonce":3452445943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6823","nonce":3452445943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6821","nonce":3452445943}]},"public_addr":"172.21.15.203:6817/3452445943","cluster_addr":"172.21.15.203:6819/3452445943","heartbeat_back_addr":"172.21.15.203:6823/3452445943","heartbeat_front_addr":"172.21.15.203:6821/3452445943","state":["exists","up"]},{"osd":9,"uuid":"95704ed6-6286-4094-9c6d-23919c7005ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6827","nonce":2998307533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6829","nonce":2998307533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6833","nonce":2998307533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6831","nonce":2998307533}]},"public_addr":"172.21.15.67:6827/2998307533","cluster_addr":"172.21.15.67:6829/2998307533","heartbeat_back_addr":"172.21.15.67:6833/2998307533","heartbeat_front_addr":"172.21.15.67:6831/2998307533","state":["exists","up"]},{"osd":10,"uuid":"bd2a7d33-b935-46c8-95c3-d76f28242ffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6825","nonce":2819057142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6827","nonce":2819057142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6831","nonce":2819057142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6829","nonce":2819057142}]},"public_addr":"172.21.15.100:6825/2819057142","cluster_addr":"172.21.15.100:6827/2819057142","heartbeat_back_addr":"172.21.15.100:6831/2819057142","heartbeat_front_addr":"172.21.15.100:6829/2819057142","state":["exists","up"]},{"osd":11,"uuid":"ea591507-10e2-4c08-8d29-d38ba9be96ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6825","nonce":566502600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6827","nonce":566502600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6831","nonce":566502600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6829","nonce":566502600}]},"public_addr":"172.21.15.203:6825/566502600","cluster_addr":"172.21.15.203:6827/566502600","heartbeat_back_addr":"172.21.15.203:6831/566502600","heartbeat_front_addr":"172.21.15.203:6829/566502600","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:28:58.945733+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:29:23.597118+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:29:48.303563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:30:14.139139+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:30:39.021508+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:04.981241+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:30.853841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:56.452967+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:32:22.239036+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:32:47.998803+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:33:13.850759+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:33:39.282150+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6800/1181476774":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/3100321138":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/2309364298":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/3458717076":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/4110206171":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6801/1181476774":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/2910038982":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6801/2622729103":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6800/2622729103":"2024-09-22T06:26:37.041377+0000","172.21.15.67:0/3346784528":"2024-09-22T06:26:37.041377+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:37:00.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.939+0000 7efebef12640 1 -- 172.21.15.67:0/927183257 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7efea00b6510 msgr2=0x7efea00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:00.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.939+0000 7efebef12640 1 --2- 172.21.15.67:0/927183257 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7efea00b6510 0x7efea00b8a20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efeb8104220 tx=0x7efeb4005f90 comp rx=0 tx=0).stop 2024-09-21T06:37:00.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.939+0000 7efebef12640 1 -- 172.21.15.67:0/927183257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeb810de70 msgr2=0x7efeb8103250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:00.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.939+0000 7efebef12640 1 --2- 172.21.15.67:0/927183257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeb810de70 0x7efeb8103250 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7efeb000eaa0 tx=0x7efeb000ef60 comp rx=0 tx=0).stop 2024-09-21T06:37:00.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.939+0000 7efebef12640 1 -- 172.21.15.67:0/927183257 shutdown_connections 2024-09-21T06:37:00.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.939+0000 7efebef12640 1 --2- 172.21.15.67:0/927183257 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7efea00b6510 0x7efea00b8a20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:00.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.939+0000 7efebef12640 1 --2- 172.21.15.67:0/927183257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeb810de70 0x7efeb8103250 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:00.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.939+0000 7efebef12640 1 --2- 172.21.15.67:0/927183257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efeb81052f0 0x7efeb8102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:00.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.939+0000 7efebef12640 1 --2- 172.21.15.67:0/927183257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeb81048c0 0x7efeb81027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:00.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.939+0000 7efebef12640 1 -- 172.21.15.67:0/927183257 >> 172.21.15.67:0/927183257 conn(0x7efeb8079740 msgr2=0x7efeb80ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:37:00.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.940+0000 7efebef12640 1 -- 172.21.15.67:0/927183257 shutdown_connections 2024-09-21T06:37:00.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:00.940+0000 7efebef12640 1 -- 172.21.15.67:0/927183257 wait complete. 2024-09-21T06:37:01.594 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph daemon mon.a --format=json config get run_dir 2024-09-21T06:37:01.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:01 smithi067 ceph-mon[31232]: pgmap v373: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:37:01.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:01 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/927183257' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:37:01.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:01 smithi100 ceph-mon[33401]: pgmap v373: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:37:01.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:01 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/927183257' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:37:01.855 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:37:01.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:01 smithi203 ceph-mon[33735]: pgmap v373: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:37:01.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:01 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/927183257' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:37:03.107 INFO:teuthology.orchestra.run.smithi067.stdout:{"run_dir":"/var/run/ceph"} 2024-09-21T06:37:03.107 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-21T06:37:03.107 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 7864 2024-09-21T06:37:03.107 DEBUG:tasks.quiescer:Yielding 2024-09-21T06:37:03.108 INFO:teuthology.run_tasks:Running task workunit... 2024-09-21T06:37:03.118 INFO:tasks.workunit:Pulling workunits from ref 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:37:03.118 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-21T06:37:03.118 INFO:tasks.workunit:timeout=3h 2024-09-21T06:37:03.118 INFO:tasks.workunit:cleanup=True 2024-09-21T06:37:03.118 DEBUG:teuthology.orchestra.run.smithi067:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-21T06:37:03.119 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:37:03.136 INFO:teuthology.orchestra.run.smithi067.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-21T06:37:03.136 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-21T06:37:03.136 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 78h/120d Inode: 1 Links: 2 2024-09-21T06:37:03.136 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:37:03.136 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:fusefs_t:s0 2024-09-21T06:37:03.136 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-09-21 06:35:13.670357580 +0000 2024-09-21T06:37:03.136 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-09-21 06:35:13.670357580 +0000 2024-09-21T06:37:03.136 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-09-21 06:36:51.416230301 +0000 2024-09-21T06:37:03.137 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: - 2024-09-21T06:37:03.137 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-21T06:37:03.137 DEBUG:teuthology.orchestra.run.smithi067:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-21T06:37:03.209 DEBUG:teuthology.orchestra.run.smithi067:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:37:03.265 INFO:tasks.workunit.client.0.smithi067.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-21T06:37:03.380 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:37:03.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:03 smithi067 ceph-mon[31232]: pgmap v374: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 3 op/s 2024-09-21T06:37:03.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:03 smithi100 ceph-mon[33401]: pgmap v374: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 3 op/s 2024-09-21T06:37:03.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.882+0000 7f10d1d9a640 1 -- 172.21.15.67:0/2183881900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10cc10de10 msgr2=0x7f10cc110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:03.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.882+0000 7f10d1d9a640 1 --2- 172.21.15.67:0/2183881900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10cc10de10 0x7f10cc110300 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f10c4009ca0 tx=0x7f10c402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:37:03.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.882+0000 7f10d1d9a640 1 -- 172.21.15.67:0/2183881900 shutdown_connections 2024-09-21T06:37:03.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.882+0000 7f10d1d9a640 1 --2- 172.21.15.67:0/2183881900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10cc105300 0x7f10cc10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:03.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.882+0000 7f10d1d9a640 1 --2- 172.21.15.67:0/2183881900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10cc10de10 0x7f10cc110300 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:03.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.882+0000 7f10d1d9a640 1 --2- 172.21.15.67:0/2183881900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10cc1049b0 0x7f10cc104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:03.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.882+0000 7f10d1d9a640 1 -- 172.21.15.67:0/2183881900 >> 172.21.15.67:0/2183881900 conn(0x7f10cc079730 msgr2=0x7f10cc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:37:03.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.882+0000 7f10d1d9a640 1 -- 172.21.15.67:0/2183881900 shutdown_connections 2024-09-21T06:37:03.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.882+0000 7f10d1d9a640 1 -- 172.21.15.67:0/2183881900 wait complete. 2024-09-21T06:37:03.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.883+0000 7f10d1d9a640 1 Processor -- start 2024-09-21T06:37:03.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.883+0000 7f10d1d9a640 1 -- start start 2024-09-21T06:37:03.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.884+0000 7f10d1d9a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10cc1049b0 0x7f10cc19a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:03.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.884+0000 7f10d1d9a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10cc105300 0x7f10cc19a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:03.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.884+0000 7f10d1d9a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10cc10de10 0x7f10cc19fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:03.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.884+0000 7f10d1d9a640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc113990 con 0x7f10cc10de10 2024-09-21T06:37:03.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.884+0000 7f10d1d9a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc113810 con 0x7f10cc1049b0 2024-09-21T06:37:03.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.884+0000 7f10d1d9a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc113b10 con 0x7f10cc105300 2024-09-21T06:37:03.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.884+0000 7f10cbfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10cc10de10 0x7f10cc19fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:03.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.884+0000 7f10cbfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10cc10de10 0x7f10cc19fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54064/0 (socket says 172.21.15.67:54064) 2024-09-21T06:37:03.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.884+0000 7f10cbfff640 1 -- 172.21.15.67:0/3046388886 learned_addr learned my addr 172.21.15.67:0/3046388886 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:37:03.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.884+0000 7f10cb7fe640 1 --2- 172.21.15.67:0/3046388886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10cc1049b0 0x7f10cc19a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:03.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.884+0000 7f10caffd640 1 --2- 172.21.15.67:0/3046388886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10cc105300 0x7f10cc19a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:03.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.884+0000 7f10cbfff640 1 -- 172.21.15.67:0/3046388886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10cc105300 msgr2=0x7f10cc19a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:03.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.884+0000 7f10cbfff640 1 --2- 172.21.15.67:0/3046388886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10cc105300 0x7f10cc19a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:03.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.884+0000 7f10cbfff640 1 -- 172.21.15.67:0/3046388886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10cc1049b0 msgr2=0x7f10cc19a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:03.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.884+0000 7f10cbfff640 1 --2- 172.21.15.67:0/3046388886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10cc1049b0 0x7f10cc19a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:03.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.884+0000 7f10cbfff640 1 -- 172.21.15.67:0/3046388886 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc1a02d0 con 0x7f10cc10de10 2024-09-21T06:37:03.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.884+0000 7f10caffd640 1 --2- 172.21.15.67:0/3046388886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10cc105300 0x7f10cc19a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:37:03.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.884+0000 7f10cbfff640 1 --2- 172.21.15.67:0/3046388886 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10cc10de10 0x7f10cc19fba0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f10c4005cc0 tx=0x7f10c4032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:37:03.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.885+0000 7f10c8ff9640 1 -- 172.21.15.67:0/3046388886 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c4032e00 con 0x7f10cc10de10 2024-09-21T06:37:03.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.885+0000 7f10c8ff9640 1 -- 172.21.15.67:0/3046388886 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10c40044d0 con 0x7f10cc10de10 2024-09-21T06:37:03.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.885+0000 7f10c8ff9640 1 -- 172.21.15.67:0/3046388886 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c4030b70 con 0x7f10cc10de10 2024-09-21T06:37:03.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.885+0000 7f10d1d9a640 1 -- 172.21.15.67:0/3046388886 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10cc1a0560 con 0x7f10cc10de10 2024-09-21T06:37:03.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.885+0000 7f10d1d9a640 1 -- 172.21.15.67:0/3046388886 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10cc1a6d20 con 0x7f10cc10de10 2024-09-21T06:37:03.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.886+0000 7f10d1d9a640 1 -- 172.21.15.67:0/3046388886 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10cc06b560 con 0x7f10cc10de10 2024-09-21T06:37:03.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.887+0000 7f10c8ff9640 1 -- 172.21.15.67:0/3046388886 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f10c4030d10 con 0x7f10cc10de10 2024-09-21T06:37:03.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.888+0000 7f10c8ff9640 1 --2- 172.21.15.67:0/3046388886 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f10b80b6510 0x7f10b80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:03.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.888+0000 7f10cb7fe640 1 --2- 172.21.15.67:0/3046388886 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f10b80b6510 0x7f10b80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:03.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.888+0000 7f10c8ff9640 1 -- 172.21.15.67:0/3046388886 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f10c4108ab0 con 0x7f10cc10de10 2024-09-21T06:37:03.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.889+0000 7f10cb7fe640 1 --2- 172.21.15.67:0/3046388886 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f10b80b6510 0x7f10b80b8a20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f10bc005fd0 tx=0x7f10bc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:37:03.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:03.891+0000 7f10c8ff9640 1 -- 172.21.15.67:0/3046388886 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f10c40cef70 con 0x7f10cc10de10 2024-09-21T06:37:03.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:03 smithi203 ceph-mon[33735]: pgmap v374: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 3 op/s 2024-09-21T06:37:04.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:04.121+0000 7f10d1d9a640 1 -- 172.21.15.67:0/3046388886 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10cc1a6ef0 con 0x7f10cc10de10 2024-09-21T06:37:04.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:04.122+0000 7f10c8ff9640 1 -- 172.21.15.67:0/3046388886 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7f10c40d3350 con 0x7f10cc10de10 2024-09-21T06:37:04.123 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:37:04.123 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:37:04.123 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:37:04.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:04.124+0000 7f10d1d9a640 1 -- 172.21.15.67:0/3046388886 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f10b80b6510 msgr2=0x7f10b80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:04.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:04.124+0000 7f10d1d9a640 1 --2- 172.21.15.67:0/3046388886 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f10b80b6510 0x7f10b80b8a20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f10bc005fd0 tx=0x7f10bc00b390 comp rx=0 tx=0).stop 2024-09-21T06:37:04.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:04.124+0000 7f10d1d9a640 1 -- 172.21.15.67:0/3046388886 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10cc10de10 msgr2=0x7f10cc19fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:04.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:04.124+0000 7f10d1d9a640 1 --2- 172.21.15.67:0/3046388886 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10cc10de10 0x7f10cc19fba0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f10c4005cc0 tx=0x7f10c4032a10 comp rx=0 tx=0).stop 2024-09-21T06:37:04.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:04.124+0000 7f10d1d9a640 1 -- 172.21.15.67:0/3046388886 shutdown_connections 2024-09-21T06:37:04.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:04.124+0000 7f10d1d9a640 1 --2- 172.21.15.67:0/3046388886 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f10b80b6510 0x7f10b80b8a20 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:04.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:04.124+0000 7f10d1d9a640 1 --2- 172.21.15.67:0/3046388886 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10cc10de10 0x7f10cc19fba0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:04.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:04.124+0000 7f10d1d9a640 1 --2- 172.21.15.67:0/3046388886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10cc105300 0x7f10cc19a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:04.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:04.124+0000 7f10d1d9a640 1 --2- 172.21.15.67:0/3046388886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10cc1049b0 0x7f10cc19a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:04.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:04.124+0000 7f10d1d9a640 1 -- 172.21.15.67:0/3046388886 >> 172.21.15.67:0/3046388886 conn(0x7f10cc079730 msgr2=0x7f10cc0ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:37:04.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:04.124+0000 7f10d1d9a640 1 -- 172.21.15.67:0/3046388886 shutdown_connections 2024-09-21T06:37:04.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:04.124+0000 7f10d1d9a640 1 -- 172.21.15.67:0/3046388886 wait complete. 2024-09-21T06:37:04.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:04 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3046388886' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:37:04.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:04 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3046388886' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:37:04.834 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T06:34:33.306959+0000', 'modified': '2024-09-21T06:35:09.805211+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24671}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24641': {'gid': 24641, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.100:6837/259395872', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6836', 'nonce': 259395872}, {'type': 'v1', 'addr': '172.21.15.100:6837', 'nonce': 259395872}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24671': {'gid': 24671, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.100:6839/778060936', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6838', 'nonce': 778060936}, {'type': 'v1', 'addr': '172.21.15.100:6839', 'nonce': 778060936}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24671, 'qdb_cluster': [24671]} 2024-09-21T06:37:04.834 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T06:37:04.834 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T06:37:04.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:04 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3046388886' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:37:05.134 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:37:05.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.667+0000 7f06fc765640 1 -- 172.21.15.67:0/2418810106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06f4108d00 msgr2=0x7f06f410f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:05.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.667+0000 7f06fc765640 1 --2- 172.21.15.67:0/2418810106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06f4108d00 0x7f06f410f530 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f06ec009ca0 tx=0x7f06ec02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.667+0000 7f06fc765640 1 -- 172.21.15.67:0/2418810106 shutdown_connections 2024-09-21T06:37:05.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.667+0000 7f06fc765640 1 --2- 172.21.15.67:0/2418810106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06f4108d00 0x7f06f410f530 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.667+0000 7f06fc765640 1 --2- 172.21.15.67:0/2418810106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06f41083b0 0x7f06f41087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.667+0000 7f06fc765640 1 --2- 172.21.15.67:0/2418810106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06f4102a70 0x7f06f4102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.667+0000 7f06fc765640 1 -- 172.21.15.67:0/2418810106 >> 172.21.15.67:0/2418810106 conn(0x7f06f4079710 msgr2=0x7f06f4079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:37:05.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.667+0000 7f06fc765640 1 -- 172.21.15.67:0/2418810106 shutdown_connections 2024-09-21T06:37:05.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.667+0000 7f06fc765640 1 -- 172.21.15.67:0/2418810106 wait complete. 2024-09-21T06:37:05.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.668+0000 7f06fc765640 1 Processor -- start 2024-09-21T06:37:05.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.668+0000 7f06fc765640 1 -- start start 2024-09-21T06:37:05.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.668+0000 7f06fc765640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06f4102a70 0x7f06f419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:05.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.668+0000 7f06fc765640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06f41083b0 0x7f06f419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:05.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.668+0000 7f06fc765640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06f4108d00 0x7f06f419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:05.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.669+0000 7f06fc765640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f06f41139c0 con 0x7f06f4102a70 2024-09-21T06:37:05.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.669+0000 7f06fc765640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f06f4113840 con 0x7f06f4108d00 2024-09-21T06:37:05.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.669+0000 7f06fc765640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f06f4113b40 con 0x7f06f41083b0 2024-09-21T06:37:05.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.669+0000 7f06fa501640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06f4102a70 0x7f06f419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:05.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.669+0000 7f06fa501640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06f4102a70 0x7f06f419a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54084/0 (socket says 172.21.15.67:54084) 2024-09-21T06:37:05.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.669+0000 7f06fa501640 1 -- 172.21.15.67:0/1825839579 learned_addr learned my addr 172.21.15.67:0/1825839579 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:37:05.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.669+0000 7f06f9d00640 1 --2- 172.21.15.67:0/1825839579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06f41083b0 0x7f06f419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:05.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.669+0000 7f06fad02640 1 --2- 172.21.15.67:0/1825839579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06f4108d00 0x7f06f419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:05.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.669+0000 7f06fa501640 1 -- 172.21.15.67:0/1825839579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06f41083b0 msgr2=0x7f06f419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:05.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.669+0000 7f06fa501640 1 --2- 172.21.15.67:0/1825839579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06f41083b0 0x7f06f419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.669+0000 7f06fa501640 1 -- 172.21.15.67:0/1825839579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06f4108d00 msgr2=0x7f06f419fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:05.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.669+0000 7f06fa501640 1 --2- 172.21.15.67:0/1825839579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06f4108d00 0x7f06f419fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.669+0000 7f06fa501640 1 -- 172.21.15.67:0/1825839579 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f41a02c0 con 0x7f06f4102a70 2024-09-21T06:37:05.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.669+0000 7f06fa501640 1 --2- 172.21.15.67:0/1825839579 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06f4102a70 0x7f06f419a410 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f06f000c920 tx=0x7f06f000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:37:05.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.669+0000 7f06e77fe640 1 -- 172.21.15.67:0/1825839579 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f0007c70 con 0x7f06f4102a70 2024-09-21T06:37:05.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.669+0000 7f06fc765640 1 -- 172.21.15.67:0/1825839579 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06f41a05b0 con 0x7f06f4102a70 2024-09-21T06:37:05.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.669+0000 7f06e77fe640 1 -- 172.21.15.67:0/1825839579 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f06f0007e10 con 0x7f06f4102a70 2024-09-21T06:37:05.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.669+0000 7f06e77fe640 1 -- 172.21.15.67:0/1825839579 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f00186f0 con 0x7f06f4102a70 2024-09-21T06:37:05.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.670+0000 7f06fc765640 1 -- 172.21.15.67:0/1825839579 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06f41a6dd0 con 0x7f06f4102a70 2024-09-21T06:37:05.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.672+0000 7f0702cd9740 1 Processor -- start 2024-09-21T06:37:05.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.672+0000 7f0702cd9740 1 -- start start 2024-09-21T06:37:05.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.673+0000 7f06e77fe640 1 -- 172.21.15.67:0/1825839579 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f06f000f450 con 0x7f06f4102a70 2024-09-21T06:37:05.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.673+0000 7f0702cd9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac34303f60 0x55ac346a9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:05.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.673+0000 7f0702cd9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac342db0a0 0x55ac346aa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:05.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.673+0000 7f0702cd9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac342e3280 0x55ac346aa980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:05.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.673+0000 7f0702cd9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ac34621c10 con 0x55ac342e3280 2024-09-21T06:37:05.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.673+0000 7f0702cd9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ac34611040 con 0x55ac34303f60 2024-09-21T06:37:05.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.673+0000 7f0702cd9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ac34614e50 con 0x55ac342db0a0 2024-09-21T06:37:05.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.673+0000 7f06f9d00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac342e3280 0x55ac346aa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:05.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.673+0000 7f06f9d00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac342e3280 0x55ac346aa980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54090/0 (socket says 172.21.15.67:54090) 2024-09-21T06:37:05.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.673+0000 7f06f9d00640 1 -- 172.21.15.67:0/1261112801 learned_addr learned my addr 172.21.15.67:0/1261112801 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:37:05.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.673+0000 7f06fa501640 1 --2- 172.21.15.67:0/1261112801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac342db0a0 0x55ac346aa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:05.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.673+0000 7f06fad02640 1 --2- 172.21.15.67:0/1261112801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac34303f60 0x55ac346a9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:05.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.673+0000 7f06f9d00640 1 -- 172.21.15.67:0/1261112801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac342db0a0 msgr2=0x55ac346aa440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:37:05.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.673+0000 7f06f9d00640 1 --2- 172.21.15.67:0/1261112801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac342db0a0 0x55ac346aa440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.673+0000 7f06f9d00640 1 -- 172.21.15.67:0/1261112801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac34303f60 msgr2=0x55ac346a9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:37:05.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.673+0000 7f06f9d00640 1 --2- 172.21.15.67:0/1261112801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac34303f60 0x55ac346a9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.673+0000 7f06f9d00640 1 -- 172.21.15.67:0/1261112801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac3436f540 con 0x55ac342e3280 2024-09-21T06:37:05.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.674+0000 7f06f9d00640 1 --2- 172.21.15.67:0/1261112801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac342e3280 0x55ac346aa980 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f06e800bbd0 tx=0x7f06e800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532031bb00c26ceb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:37:05.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.674+0000 7f06e77fe640 1 --2- 172.21.15.67:0/1825839579 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f06e00b6540 0x7f06e00b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:05.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.674+0000 7f06e4ff9640 1 -- 172.21.15.67:0/1261112801 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8015070 con 0x55ac342e3280 2024-09-21T06:37:05.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.674+0000 7f06e4ff9640 1 -- 172.21.15.67:0/1261112801 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f06e801a8f0 con 0x55ac342e3280 2024-09-21T06:37:05.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.674+0000 7f06e4ff9640 1 -- 172.21.15.67:0/1261112801 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8011780 con 0x55ac342e3280 2024-09-21T06:37:05.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.674+0000 7f06fad02640 1 --2- 172.21.15.67:0/1825839579 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f06e00b6540 0x7f06e00b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:05.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.674+0000 7f06e77fe640 1 -- 172.21.15.67:0/1825839579 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f06f00e78f0 con 0x7f06f4102a70 2024-09-21T06:37:05.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.675+0000 7f0702cd9740 1 -- 172.21.15.67:0/1261112801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac342e3280 msgr2=0x55ac346aa980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:05.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.675+0000 7f06fad02640 1 --2- 172.21.15.67:0/1825839579 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f06e00b6540 0x7f06e00b8a50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f06ec009a60 tx=0x7f06ec005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:37:05.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.675+0000 7f0702cd9740 1 --2- 172.21.15.67:0/1261112801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac342e3280 0x55ac346aa980 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f06e800bbd0 tx=0x7f06e800f730 comp rx=0 tx=0).stop 2024-09-21T06:37:05.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.675+0000 7f0702cd9740 1 -- 172.21.15.67:0/1261112801 shutdown_connections 2024-09-21T06:37:05.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.675+0000 7f0702cd9740 1 --2- 172.21.15.67:0/1261112801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac342e3280 0x55ac346aa980 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.675+0000 7f0702cd9740 1 --2- 172.21.15.67:0/1261112801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac342db0a0 0x55ac346aa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.675+0000 7f0702cd9740 1 --2- 172.21.15.67:0/1261112801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac34303f60 0x55ac346a9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.675+0000 7f0702cd9740 1 -- 172.21.15.67:0/1261112801 >> 172.21.15.67:0/1261112801 conn(0x55ac342f8350 msgr2=0x55ac346a7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:37:05.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.675+0000 7f0702cd9740 1 -- 172.21.15.67:0/1261112801 shutdown_connections 2024-09-21T06:37:05.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.675+0000 7f0702cd9740 1 -- 172.21.15.67:0/1261112801 wait complete. 2024-09-21T06:37:05.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.675+0000 7f0702cd9740 1 -- start start 2024-09-21T06:37:05.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.675+0000 7f0702cd9740 1 Processor -- start 2024-09-21T06:37:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.676+0000 7f0702cd9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac342db0a0 0x55ac346b9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.676+0000 7f0702cd9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac342e3280 0x55ac346ba150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.676+0000 7f0702cd9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac34303f60 0x55ac346c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.676+0000 7f0702cd9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ac344725f0 con 0x55ac342db0a0 2024-09-21T06:37:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.676+0000 7f0702cd9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ac345ac010 con 0x55ac342e3280 2024-09-21T06:37:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.676+0000 7f0702cd9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ac346bca10 con 0x55ac34303f60 2024-09-21T06:37:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.676+0000 7f0702cd9740 10 client.0 _ll_register_callbacks cb 0x55ac3460b340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:37:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.676+0000 7f06fa501640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac342db0a0 0x55ac346b9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.676+0000 7f06fa501640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac342db0a0 0x55ac346b9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54092/0 (socket says 172.21.15.67:54092) 2024-09-21T06:37:05.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.676+0000 7f06fa501640 1 -- 172.21.15.67:0/3988498396 learned_addr learned my addr 172.21.15.67:0/3988498396 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:37:05.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.676+0000 7f06fad02640 1 --2- 172.21.15.67:0/3988498396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac34303f60 0x55ac346c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:05.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.676+0000 7f06f9d00640 1 --2- 172.21.15.67:0/3988498396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac342e3280 0x55ac346ba150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:05.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.676+0000 7f06fa501640 1 -- 172.21.15.67:0/3988498396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac34303f60 msgr2=0x55ac346c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:37:05.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.676+0000 7f06fa501640 1 --2- 172.21.15.67:0/3988498396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac34303f60 0x55ac346c5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.676+0000 7f06fa501640 1 -- 172.21.15.67:0/3988498396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac342e3280 msgr2=0x55ac346ba150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:37:05.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.676+0000 7f06fa501640 1 --2- 172.21.15.67:0/3988498396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac342e3280 0x55ac346ba150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.676+0000 7f06fa501640 1 -- 172.21.15.67:0/3988498396 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac345d9a00 con 0x55ac342db0a0 2024-09-21T06:37:05.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.676+0000 7f06f9d00640 1 --2- 172.21.15.67:0/3988498396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac342e3280 0x55ac346ba150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:37:05.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.677+0000 7f06fa501640 1 --2- 172.21.15.67:0/3988498396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac342db0a0 0x55ac346b9b70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f06f00ad640 tx=0x7f06f00a9dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79fa801af1f89e2b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:37:05.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.677+0000 7f06ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:37:05.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.677+0000 7f06ddffb640 1 -- 172.21.15.67:0/3988498396 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f00ace20 con 0x55ac342db0a0 2024-09-21T06:37:05.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.677+0000 7f06ddffb640 1 -- 172.21.15.67:0/3988498396 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f06f00b6e40 con 0x55ac342db0a0 2024-09-21T06:37:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.677+0000 7f0702cd9740 1 -- 172.21.15.67:0/3988498396 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06e801a8f0 con 0x55ac342db0a0 2024-09-21T06:37:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.677+0000 7f06ddffb640 1 -- 172.21.15.67:0/3988498396 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f06f00ad1c0 con 0x55ac342db0a0 2024-09-21T06:37:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.677+0000 7f0702cd9740 10 client.15432 fetch_fsmap learned FSMap version 21 2024-09-21T06:37:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.677+0000 7f0702cd9740 1 -- 172.21.15.67:0/3988498396 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ac346c89f0 con 0x55ac342db0a0 2024-09-21T06:37:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.677+0000 7f06ddffb640 1 -- 172.21.15.67:0/3988498396 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f06e801a8f0 con 0x55ac342db0a0 2024-09-21T06:37:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.677+0000 7f06ddffb640 1 -- 172.21.15.67:0/3988498396 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f06f00a9090 con 0x55ac342db0a0 2024-09-21T06:37:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.677+0000 7f0702cd9740 10 client.15432 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:37:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.677+0000 7f0702cd9740 10 client.15432 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:37:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.677+0000 7f0702cd9740 20 client.15432 populate_metadata read hostname 'smithi067' 2024-09-21T06:37:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.677+0000 7f0702cd9740 1 --2- 172.21.15.67:0/3988498396 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55ac342c28e0 0x55ac346cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.677+0000 7f0702cd9740 4 client.15432 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:37:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.677+0000 7f0702cd9740 1 -- 172.21.15.67:0/3988498396 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac34623b60 con 0x55ac342c28e0 2024-09-21T06:37:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.678+0000 7f06f9d00640 1 --2- 172.21.15.67:0/3988498396 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55ac342c28e0 0x55ac346cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.678+0000 7f06f9d00640 1 --2- 172.21.15.67:0/3988498396 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55ac342c28e0 0x55ac346cc3d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f5dc0e50acf94ab server_cookie=78f7296ef1a78b69 in_seq=0 out_seq=0 2024-09-21T06:37:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.678+0000 7f06ddffb640 10 client.15432 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:37:05.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.679+0000 7f06ddffb640 1 -- 172.21.15.67:0/3988498396 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ac34623b60 con 0x55ac342c28e0 2024-09-21T06:37:05.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.679+0000 7f06ddffb640 10 client.15432 handle_command_reply: tid=0 2024-09-21T06:37:05.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.679+0000 7f0702cd9740 1 client.15432 shutdown 2024-09-21T06:37:05.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.679+0000 7f0702cd9740 1 -- 172.21.15.67:0/3988498396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac342db0a0 msgr2=0x55ac346b9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:05.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.679+0000 7f0702cd9740 1 --2- 172.21.15.67:0/3988498396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac342db0a0 0x55ac346b9b70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f06f00ad640 tx=0x7f06f00a9dc0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.679+0000 7f0702cd9740 1 -- 172.21.15.67:0/3988498396 shutdown_connections 2024-09-21T06:37:05.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.679+0000 7f0702cd9740 1 --2- 172.21.15.67:0/3988498396 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55ac342c28e0 0x55ac346cc3d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.679+0000 7f06ddffb640 0 client.15432 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:37:05.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.679+0000 7f0702cd9740 1 --2- 172.21.15.67:0/3988498396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac342e3280 0x55ac346ba150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.679+0000 7f0702cd9740 1 --2- 172.21.15.67:0/3988498396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac34303f60 0x55ac346c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.679+0000 7f0702cd9740 1 --2- 172.21.15.67:0/3988498396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac342db0a0 0x55ac346b9b70 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.679+0000 7f0702cd9740 1 -- 172.21.15.67:0/3988498396 >> 172.21.15.67:0/3988498396 conn(0x55ac342f8350 msgr2=0x55ac346ad090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:37:05.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.680+0000 7f0702cd9740 1 -- 172.21.15.67:0/3988498396 shutdown_connections 2024-09-21T06:37:05.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.680+0000 7f0702cd9740 1 -- 172.21.15.67:0/3988498396 wait complete. 2024-09-21T06:37:05.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.680+0000 7f0702cd9740 20 client.15432 trim_cache size 0 max 16384 2024-09-21T06:37:05.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.698+0000 7f0702cd9740 1 Processor -- start 2024-09-21T06:37:05.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.698+0000 7f0702cd9740 1 -- start start 2024-09-21T06:37:05.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.698+0000 7f0702cd9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac342c28e0 0x55ac346c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:05.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.698+0000 7f0702cd9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac342db0a0 0x55ac346c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:05.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.698+0000 7f0702cd9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac342e3280 0x55ac346c6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:05.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.698+0000 7f0702cd9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ac346ca710 con 0x55ac342e3280 2024-09-21T06:37:05.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.698+0000 7f0702cd9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ac346cade0 con 0x55ac342c28e0 2024-09-21T06:37:05.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.698+0000 7f0702cd9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ac346cb4b0 con 0x55ac342db0a0 2024-09-21T06:37:05.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.698+0000 7f06fad02640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac342e3280 0x55ac346c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:05.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.698+0000 7f06fad02640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac342e3280 0x55ac346c6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54108/0 (socket says 172.21.15.67:54108) 2024-09-21T06:37:05.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.698+0000 7f06fad02640 1 -- 172.21.15.67:0/3257065600 learned_addr learned my addr 172.21.15.67:0/3257065600 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:37:05.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.698+0000 7f06fa501640 1 --2- 172.21.15.67:0/3257065600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac342c28e0 0x55ac346c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:05.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.699+0000 7f06fad02640 1 -- 172.21.15.67:0/3257065600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac342db0a0 msgr2=0x55ac346c5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:37:05.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.699+0000 7f06fad02640 1 --2- 172.21.15.67:0/3257065600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac342db0a0 0x55ac346c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.699+0000 7f06fad02640 1 -- 172.21.15.67:0/3257065600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac342c28e0 msgr2=0x55ac346c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:37:05.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.699+0000 7f06fad02640 1 --2- 172.21.15.67:0/3257065600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac342c28e0 0x55ac346c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.699+0000 7f06fad02640 1 -- 172.21.15.67:0/3257065600 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac346a8780 con 0x55ac342e3280 2024-09-21T06:37:05.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.699+0000 7f06fa501640 1 --2- 172.21.15.67:0/3257065600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac342c28e0 0x55ac346c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:37:05.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.699+0000 7f06fad02640 1 --2- 172.21.15.67:0/3257065600 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac342e3280 0x55ac346c6220 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f06ec009ca0 tx=0x7f06ec044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5ba7106efc6c0b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:37:05.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.699+0000 7f06dffff640 1 -- 172.21.15.67:0/3257065600 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ec003d50 con 0x55ac342e3280 2024-09-21T06:37:05.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.699+0000 7f06dffff640 1 -- 172.21.15.67:0/3257065600 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f06ec036040 con 0x55ac342e3280 2024-09-21T06:37:05.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.699+0000 7f06dffff640 1 -- 172.21.15.67:0/3257065600 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ec0308f0 con 0x55ac342e3280 2024-09-21T06:37:05.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.700+0000 7f0702cd9740 1 -- 172.21.15.67:0/3257065600 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac342e3280 msgr2=0x55ac346c6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:05.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.700+0000 7f0702cd9740 1 --2- 172.21.15.67:0/3257065600 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac342e3280 0x55ac346c6220 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f06ec009ca0 tx=0x7f06ec044000 comp rx=0 tx=0).stop 2024-09-21T06:37:05.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.700+0000 7f0702cd9740 1 -- 172.21.15.67:0/3257065600 shutdown_connections 2024-09-21T06:37:05.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.700+0000 7f0702cd9740 1 --2- 172.21.15.67:0/3257065600 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac342e3280 0x55ac346c6220 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.700+0000 7f0702cd9740 1 --2- 172.21.15.67:0/3257065600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac342db0a0 0x55ac346c5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.700+0000 7f0702cd9740 1 --2- 172.21.15.67:0/3257065600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac342c28e0 0x55ac346c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.700+0000 7f0702cd9740 1 -- 172.21.15.67:0/3257065600 >> 172.21.15.67:0/3257065600 conn(0x55ac342f8350 msgr2=0x55ac346b83f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:37:05.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.700+0000 7f0702cd9740 1 -- 172.21.15.67:0/3257065600 shutdown_connections 2024-09-21T06:37:05.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.700+0000 7f0702cd9740 1 -- 172.21.15.67:0/3257065600 wait complete. 2024-09-21T06:37:05.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.700+0000 7f0702cd9740 1 -- start start 2024-09-21T06:37:05.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.700+0000 7f0702cd9740 1 Processor -- start 2024-09-21T06:37:05.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.701+0000 7f0702cd9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac342c28e0 0x55ac346ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:05.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.701+0000 7f0702cd9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac342db0a0 0x55ac346cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:05.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.701+0000 7f0702cd9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac342e3280 0x55ac346d42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:05.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.701+0000 7f0702cd9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ac346cc250 con 0x55ac342db0a0 2024-09-21T06:37:05.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.701+0000 7f0702cd9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ac346cbb80 con 0x55ac342e3280 2024-09-21T06:37:05.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.701+0000 7f0702cd9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ac346cc920 con 0x55ac342c28e0 2024-09-21T06:37:05.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.701+0000 7f0702cd9740 10 client.0 _ll_register_callbacks cb 0x7f06f000cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:37:05.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.701+0000 7f06f9d00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac342db0a0 0x55ac346cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:05.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.701+0000 7f06f9d00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac342db0a0 0x55ac346cf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54110/0 (socket says 172.21.15.67:54110) 2024-09-21T06:37:05.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.701+0000 7f06f9d00640 1 -- 172.21.15.67:0/3933209191 learned_addr learned my addr 172.21.15.67:0/3933209191 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:37:05.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.701+0000 7f06fa501640 1 --2- 172.21.15.67:0/3933209191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac342c28e0 0x55ac346ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:05.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.701+0000 7f06fad02640 1 --2- 172.21.15.67:0/3933209191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac342e3280 0x55ac346d42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:05.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.701+0000 7f06f9d00640 1 -- 172.21.15.67:0/3933209191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac342c28e0 msgr2=0x55ac346ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:37:05.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.701+0000 7f06f9d00640 1 --2- 172.21.15.67:0/3933209191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac342c28e0 0x55ac346ced80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.701+0000 7f06f9d00640 1 -- 172.21.15.67:0/3933209191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac342e3280 msgr2=0x55ac346d42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:37:05.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.701+0000 7f06f9d00640 1 --2- 172.21.15.67:0/3933209191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac342e3280 0x55ac346d42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.701+0000 7f06f9d00640 1 -- 172.21.15.67:0/3933209191 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac346d4800 con 0x55ac342db0a0 2024-09-21T06:37:05.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.701+0000 7f06f9d00640 1 --2- 172.21.15.67:0/3933209191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac342db0a0 0x55ac346cf2c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f06e8005d80 tx=0x7f06e800b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc4bfe395112ffc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:37:05.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.701+0000 7f06ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:37:05.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.702+0000 7f06ddffb640 1 -- 172.21.15.67:0/3933209191 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8015050 con 0x55ac342db0a0 2024-09-21T06:37:05.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.702+0000 7f06ddffb640 1 -- 172.21.15.67:0/3933209191 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f06e8004440 con 0x55ac342db0a0 2024-09-21T06:37:05.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.702+0000 7f0702cd9740 1 -- 172.21.15.67:0/3933209191 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06ec036040 con 0x55ac342db0a0 2024-09-21T06:37:05.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.702+0000 7f06ddffb640 1 -- 172.21.15.67:0/3933209191 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f06e8028040 con 0x55ac342db0a0 2024-09-21T06:37:05.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.702+0000 7f0702cd9740 10 client.15438 fetch_fsmap learned FSMap version 21 2024-09-21T06:37:05.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.702+0000 7f0702cd9740 1 -- 172.21.15.67:0/3933209191 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ac34738020 con 0x55ac342db0a0 2024-09-21T06:37:05.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.702+0000 7f06ddffb640 1 -- 172.21.15.67:0/3933209191 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f06ec036040 con 0x55ac342db0a0 2024-09-21T06:37:05.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.702+0000 7f06ddffb640 1 -- 172.21.15.67:0/3933209191 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f06e8004a90 con 0x55ac342db0a0 2024-09-21T06:37:05.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.702+0000 7f0702cd9740 10 client.15438 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:37:05.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.702+0000 7f0702cd9740 10 client.15438 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:37:05.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.702+0000 7f0702cd9740 20 client.15438 populate_metadata read hostname 'smithi067' 2024-09-21T06:37:05.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.702+0000 7f0702cd9740 1 --2- 172.21.15.67:0/3933209191 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55ac34303f60 0x55ac3473b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:05.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.702+0000 7f0702cd9740 4 client.15438 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:37:05.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.702+0000 7f0702cd9740 1 -- 172.21.15.67:0/3933209191 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "status"}) -- 0x55ac3461f610 con 0x55ac34303f60 2024-09-21T06:37:05.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.703+0000 7f06fa501640 1 --2- 172.21.15.67:0/3933209191 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55ac34303f60 0x55ac3473b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:05.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.703+0000 7f06fa501640 1 --2- 172.21.15.67:0/3933209191 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55ac34303f60 0x55ac3473b660 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74abd3efa72091a6 server_cookie=e4e3d6fac0e210d4 in_seq=0 out_seq=0 2024-09-21T06:37:05.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.703+0000 7f06ddffb640 10 client.15438 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:37:05.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.703+0000 7f06ddffb640 1 -- 172.21.15.67:0/3933209191 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55ac3461f610 con 0x55ac34303f60 2024-09-21T06:37:05.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.703+0000 7f06ddffb640 10 client.15438 handle_command_reply: tid=0 2024-09-21T06:37:05.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.703+0000 7f0702cd9740 1 client.15438 shutdown 2024-09-21T06:37:05.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.704+0000 7f0702cd9740 1 -- 172.21.15.67:0/3933209191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac342db0a0 msgr2=0x55ac346cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:05.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.704+0000 7f0702cd9740 1 --2- 172.21.15.67:0/3933209191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac342db0a0 0x55ac346cf2c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f06e8005d80 tx=0x7f06e800b6b0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.704+0000 7f0702cd9740 1 -- 172.21.15.67:0/3933209191 shutdown_connections 2024-09-21T06:37:05.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.704+0000 7f0702cd9740 1 --2- 172.21.15.67:0/3933209191 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55ac34303f60 0x55ac3473b660 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.704+0000 7f06ddffb640 0 client.15438 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:37:05.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.704+0000 7f0702cd9740 1 --2- 172.21.15.67:0/3933209191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac342e3280 0x55ac346d42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.704+0000 7f0702cd9740 1 --2- 172.21.15.67:0/3933209191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac342db0a0 0x55ac346cf2c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.704+0000 7f0702cd9740 1 --2- 172.21.15.67:0/3933209191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac342c28e0 0x55ac346ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.704+0000 7f0702cd9740 1 -- 172.21.15.67:0/3933209191 >> 172.21.15.67:0/3933209191 conn(0x55ac342f8350 msgr2=0x55ac346b8030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:37:05.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.704+0000 7f0702cd9740 1 -- 172.21.15.67:0/3933209191 shutdown_connections 2024-09-21T06:37:05.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.704+0000 7f0702cd9740 1 -- 172.21.15.67:0/3933209191 wait complete. 2024-09-21T06:37:05.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.704+0000 7f0702cd9740 20 client.15438 trim_cache size 0 max 16384 2024-09-21T06:37:05.709 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-21T06:37:05.710 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "2a791768-77e2-11ef-baf5-efdc52797490", 2024-09-21T06:37:05.710 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-09-21T06:37:05.710 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24671, 2024-09-21T06:37:05.710 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-09-21T06:37:05.710 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-09-21T06:37:05.710 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-09-21T06:37:05.710 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 152.055208515, 2024-09-21T06:37:05.710 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 21, 2024-09-21T06:37:05.710 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 85, 2024-09-21T06:37:05.710 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 83, 2024-09-21T06:37:05.710 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 163.635114958 2024-09-21T06:37:05.710 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-21T06:37:05.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.704+0000 7f06fc765640 1 -- 172.21.15.67:0/1825839579 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f06e00b6540 msgr2=0x7f06e00b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:05.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.704+0000 7f06fc765640 1 --2- 172.21.15.67:0/1825839579 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f06e00b6540 0x7f06e00b8a50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f06ec009a60 tx=0x7f06ec005c50 comp rx=0 tx=0).stop 2024-09-21T06:37:05.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.704+0000 7f06fc765640 1 -- 172.21.15.67:0/1825839579 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06f4102a70 msgr2=0x7f06f419a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:05.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.704+0000 7f06fc765640 1 --2- 172.21.15.67:0/1825839579 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06f4102a70 0x7f06f419a410 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f06f000c920 tx=0x7f06f000cde0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.704+0000 7f06fc765640 1 -- 172.21.15.67:0/1825839579 shutdown_connections 2024-09-21T06:37:05.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.705+0000 7f06fc765640 1 --2- 172.21.15.67:0/1825839579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06f4108d00 0x7f06f419fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.705+0000 7f06fc765640 1 --2- 172.21.15.67:0/1825839579 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f06e00b6540 0x7f06e00b8a50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.705+0000 7f06fc765640 1 --2- 172.21.15.67:0/1825839579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06f41083b0 0x7f06f419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.705+0000 7f06fc765640 1 --2- 172.21.15.67:0/1825839579 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06f4102a70 0x7f06f419a410 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:05.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.705+0000 7f06fc765640 1 -- 172.21.15.67:0/1825839579 >> 172.21.15.67:0/1825839579 conn(0x7f06f4079710 msgr2=0x7f06f41064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:37:05.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.705+0000 7f06fc765640 1 -- 172.21.15.67:0/1825839579 shutdown_connections 2024-09-21T06:37:05.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:05.705+0000 7f06fc765640 1 -- 172.21.15.67:0/1825839579 wait complete. 2024-09-21T06:37:05.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:05 smithi067 ceph-mon[31232]: pgmap v375: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:37:05.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:05 smithi100 ceph-mon[33401]: pgmap v375: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:37:05.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:05 smithi203 ceph-mon[33735]: pgmap v375: 129 pgs: 129 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:37:06.395 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-21T06:37:07.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:07 smithi067 ceph-mon[31232]: pgmap v376: 129 pgs: 129 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-21T06:37:07.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:07 smithi100 ceph-mon[33401]: pgmap v376: 129 pgs: 129 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-21T06:37:07.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:07 smithi203 ceph-mon[33735]: pgmap v376: 129 pgs: 129 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-21T06:37:09.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:09 smithi067 ceph-mon[31232]: pgmap v377: 129 pgs: 129 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:37:09.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:09 smithi100 ceph-mon[33401]: pgmap v377: 129 pgs: 129 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:37:09.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:09 smithi203 ceph-mon[33735]: pgmap v377: 129 pgs: 129 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:37:11.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:11 smithi067 ceph-mon[31232]: pgmap v378: 129 pgs: 129 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 1 op/s 2024-09-21T06:37:11.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:11 smithi100 ceph-mon[33401]: pgmap v378: 129 pgs: 129 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 1 op/s 2024-09-21T06:37:11.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:11 smithi203 ceph-mon[33735]: pgmap v378: 129 pgs: 129 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 1 op/s 2024-09-21T06:37:13.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:13 smithi067 ceph-mon[31232]: pgmap v379: 129 pgs: 129 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:37:13.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:13 smithi100 ceph-mon[33401]: pgmap v379: 129 pgs: 129 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:37:13.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:13 smithi203 ceph-mon[33735]: pgmap v379: 129 pgs: 129 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:37:15.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:15 smithi067 ceph-mon[31232]: pgmap v380: 129 pgs: 129 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:37:15.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:15 smithi100 ceph-mon[33401]: pgmap v380: 129 pgs: 129 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:37:15.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:15 smithi203 ceph-mon[33735]: pgmap v380: 129 pgs: 129 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:37:17.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:17 smithi203 ceph-mon[33735]: pgmap v381: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-21T06:37:17.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:17 smithi067 ceph-mon[31232]: pgmap v381: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-21T06:37:17.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:17 smithi100 ceph-mon[33401]: pgmap v381: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-21T06:37:19.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:19 smithi067 ceph-mon[31232]: pgmap v382: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-21T06:37:19.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:19 smithi100 ceph-mon[33401]: pgmap v382: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-21T06:37:19.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:19 smithi203 ceph-mon[33735]: pgmap v382: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-21T06:37:21.395 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 14.5 2024-09-21T06:37:21.396 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:37:21.675 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:37:21.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:21 smithi067 ceph-mon[31232]: pgmap v383: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:37:21.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:21 smithi100 ceph-mon[33401]: pgmap v383: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:37:21.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:21 smithi203 ceph-mon[33735]: pgmap v383: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:37:22.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.253+0000 7fe7e5f42640 1 -- 172.21.15.67:0/2604032656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7e0109e40 msgr2=0x7fe7e0110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:22.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.253+0000 7fe7e5f42640 1 --2- 172.21.15.67:0/2604032656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7e0109e40 0x7fe7e0110670 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009ca0 tx=0x7fe7d802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:37:22.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.253+0000 7fe7e5f42640 1 -- 172.21.15.67:0/2604032656 shutdown_connections 2024-09-21T06:37:22.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.253+0000 7fe7e5f42640 1 --2- 172.21.15.67:0/2604032656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7e01094f0 0x7fe7e0109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:22.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.253+0000 7fe7e5f42640 1 --2- 172.21.15.67:0/2604032656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7e0109e40 0x7fe7e0110670 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:22.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.253+0000 7fe7e5f42640 1 --2- 172.21.15.67:0/2604032656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7e0102110 0x7fe7e0102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:22.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.253+0000 7fe7e5f42640 1 -- 172.21.15.67:0/2604032656 >> 172.21.15.67:0/2604032656 conn(0x7fe7e0077cc0 msgr2=0x7fe7e0078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:37:22.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.253+0000 7fe7e5f42640 1 -- 172.21.15.67:0/2604032656 shutdown_connections 2024-09-21T06:37:22.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.253+0000 7fe7e5f42640 1 -- 172.21.15.67:0/2604032656 wait complete. 2024-09-21T06:37:22.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.254+0000 7fe7e5f42640 1 Processor -- start 2024-09-21T06:37:22.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.254+0000 7fe7e5f42640 1 -- start start 2024-09-21T06:37:22.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.254+0000 7fe7e5f42640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7e0102110 0x7fe7e019c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:22.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.254+0000 7fe7e5f42640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7e01094f0 0x7fe7e019cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:22.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.254+0000 7fe7e5f42640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7e0109e40 0x7fe7e01a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:22.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.254+0000 7fe7e5f42640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e01139b0 con 0x7fe7e0102110 2024-09-21T06:37:22.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.254+0000 7fe7e5f42640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e0113830 con 0x7fe7e0109e40 2024-09-21T06:37:22.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.254+0000 7fe7e5f42640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e0113b30 con 0x7fe7e01094f0 2024-09-21T06:37:22.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.255+0000 7fe7df7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7e0102110 0x7fe7e019c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:22.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.255+0000 7fe7df7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7e0102110 0x7fe7e019c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41800/0 (socket says 172.21.15.67:41800) 2024-09-21T06:37:22.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.255+0000 7fe7df7fe640 1 -- 172.21.15.67:0/3070271638 learned_addr learned my addr 172.21.15.67:0/3070271638 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:37:22.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.255+0000 7fe7deffd640 1 --2- 172.21.15.67:0/3070271638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7e01094f0 0x7fe7e019cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:22.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.255+0000 7fe7dffff640 1 --2- 172.21.15.67:0/3070271638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7e0109e40 0x7fe7e01a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:22.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.255+0000 7fe7df7fe640 1 -- 172.21.15.67:0/3070271638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7e01094f0 msgr2=0x7fe7e019cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:22.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.255+0000 7fe7df7fe640 1 --2- 172.21.15.67:0/3070271638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7e01094f0 0x7fe7e019cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:22.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.255+0000 7fe7df7fe640 1 -- 172.21.15.67:0/3070271638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7e0109e40 msgr2=0x7fe7e01a1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:22.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.255+0000 7fe7df7fe640 1 --2- 172.21.15.67:0/3070271638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7e0109e40 0x7fe7e01a1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:22.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.255+0000 7fe7df7fe640 1 -- 172.21.15.67:0/3070271638 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e01a24f0 con 0x7fe7e0102110 2024-09-21T06:37:22.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.255+0000 7fe7deffd640 1 --2- 172.21.15.67:0/3070271638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7e01094f0 0x7fe7e019cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:37:22.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.255+0000 7fe7dffff640 1 --2- 172.21.15.67:0/3070271638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7e0109e40 0x7fe7e01a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:37:22.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.255+0000 7fe7df7fe640 1 --2- 172.21.15.67:0/3070271638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7e0102110 0x7fe7e019c580 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000e9f0 tx=0x7fe7d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:37:22.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.256+0000 7fe7dcff9640 1 -- 172.21.15.67:0/3070271638 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d000cd80 con 0x7fe7e0102110 2024-09-21T06:37:22.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.256+0000 7fe7e5f42640 1 -- 172.21.15.67:0/3070271638 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7e01a27e0 con 0x7fe7e0102110 2024-09-21T06:37:22.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.256+0000 7fe7e5f42640 1 -- 172.21.15.67:0/3070271638 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7e01a8f40 con 0x7fe7e0102110 2024-09-21T06:37:22.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.256+0000 7fe7dcff9640 1 -- 172.21.15.67:0/3070271638 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7d00054b0 con 0x7fe7e0102110 2024-09-21T06:37:22.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.256+0000 7fe7dcff9640 1 -- 172.21.15.67:0/3070271638 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0010430 con 0x7fe7e0102110 2024-09-21T06:37:22.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.257+0000 7fe7e5f42640 1 -- 172.21.15.67:0/3070271638 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7cc005290 con 0x7fe7e0102110 2024-09-21T06:37:22.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.258+0000 7fe7dcff9640 1 -- 172.21.15.67:0/3070271638 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fe7d0005020 con 0x7fe7e0102110 2024-09-21T06:37:22.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.259+0000 7fe7dcff9640 1 --2- 172.21.15.67:0/3070271638 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe7c80b6590 0x7fe7c80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:22.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.260+0000 7fe7dcff9640 1 -- 172.21.15.67:0/3070271638 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fe7d00e7950 con 0x7fe7e0102110 2024-09-21T06:37:22.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.260+0000 7fe7deffd640 1 --2- 172.21.15.67:0/3070271638 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe7c80b6590 0x7fe7c80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:22.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.260+0000 7fe7deffd640 1 --2- 172.21.15.67:0/3070271638 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe7c80b6590 0x7fe7c80b8aa0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4005d00 tx=0x7fe7d4005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:37:22.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.268+0000 7fe7dcff9640 1 -- 172.21.15.67:0/3070271638 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe7d00ade10 con 0x7fe7e0102110 2024-09-21T06:37:22.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.506+0000 7fe7e5f42640 1 -- 172.21.15.67:0/3070271638 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7cc005580 con 0x7fe7e0102110 2024-09-21T06:37:22.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.506+0000 7fe7dcff9640 1 -- 172.21.15.67:0/3070271638 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7fe7d00b21f0 con 0x7fe7e0102110 2024-09-21T06:37:22.508 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:37:22.508 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:37:22.508 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:37:22.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.510+0000 7fe7e5f42640 1 -- 172.21.15.67:0/3070271638 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe7c80b6590 msgr2=0x7fe7c80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:22.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.510+0000 7fe7e5f42640 1 --2- 172.21.15.67:0/3070271638 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe7c80b6590 0x7fe7c80b8aa0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4005d00 tx=0x7fe7d4005c70 comp rx=0 tx=0).stop 2024-09-21T06:37:22.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.510+0000 7fe7e5f42640 1 -- 172.21.15.67:0/3070271638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7e0102110 msgr2=0x7fe7e019c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:22.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.510+0000 7fe7e5f42640 1 --2- 172.21.15.67:0/3070271638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7e0102110 0x7fe7e019c580 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000e9f0 tx=0x7fe7d000edf0 comp rx=0 tx=0).stop 2024-09-21T06:37:22.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.510+0000 7fe7e5f42640 1 -- 172.21.15.67:0/3070271638 shutdown_connections 2024-09-21T06:37:22.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.510+0000 7fe7e5f42640 1 --2- 172.21.15.67:0/3070271638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7e0109e40 0x7fe7e01a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:22.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.510+0000 7fe7e5f42640 1 --2- 172.21.15.67:0/3070271638 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe7c80b6590 0x7fe7c80b8aa0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:22.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.510+0000 7fe7e5f42640 1 --2- 172.21.15.67:0/3070271638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7e01094f0 0x7fe7e019cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:22.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.510+0000 7fe7e5f42640 1 --2- 172.21.15.67:0/3070271638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7e0102110 0x7fe7e019c580 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:22.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.510+0000 7fe7e5f42640 1 -- 172.21.15.67:0/3070271638 >> 172.21.15.67:0/3070271638 conn(0x7fe7e0077cc0 msgr2=0x7fe7e0105280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:37:22.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.510+0000 7fe7e5f42640 1 -- 172.21.15.67:0/3070271638 shutdown_connections 2024-09-21T06:37:22.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:22.510+0000 7fe7e5f42640 1 -- 172.21.15.67:0/3070271638 wait complete. 2024-09-21T06:37:22.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:22 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3070271638' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:37:22.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:22 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3070271638' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:37:22.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:22 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3070271638' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:37:23.341 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db / --timeout 90 --expiration 134.5 --await-for 45.0' 2024-09-21T06:37:23.341 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24671 quiesce db / --timeout 90 --expiration 134.5 --await-for 45.0 2024-09-21T06:37:23.617 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:37:23.739 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:23 smithi067 ceph-mon[31232]: pgmap v384: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:37:23.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:23 smithi100 ceph-mon[33401]: pgmap v384: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:37:23.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:23 smithi203 ceph-mon[33735]: pgmap v384: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:37:24.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.163+0000 7f8721119640 1 -- 172.21.15.67:0/3201984971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f871c107b40 msgr2=0x7f871c107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:24.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.163+0000 7f8721119640 1 --2- 172.21.15.67:0/3201984971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f871c107b40 0x7f871c107f50 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f871000aa10 tx=0x7f871002b720 comp rx=0 tx=0).stop 2024-09-21T06:37:24.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.163+0000 7f8721119640 1 -- 172.21.15.67:0/3201984971 shutdown_connections 2024-09-21T06:37:24.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.163+0000 7f8721119640 1 --2- 172.21.15.67:0/3201984971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f871c108490 0x7f871c10ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.163+0000 7f8721119640 1 --2- 172.21.15.67:0/3201984971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f871c107b40 0x7f871c107f50 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.163+0000 7f8721119640 1 --2- 172.21.15.67:0/3201984971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f871c102200 0x7f871c102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.163+0000 7f8721119640 1 -- 172.21.15.67:0/3201984971 >> 172.21.15.67:0/3201984971 conn(0x7f871c079f50 msgr2=0x7f871c07a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:37:24.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.163+0000 7f8721119640 1 -- 172.21.15.67:0/3201984971 shutdown_connections 2024-09-21T06:37:24.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.163+0000 7f8721119640 1 -- 172.21.15.67:0/3201984971 wait complete. 2024-09-21T06:37:24.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.164+0000 7f8721119640 1 Processor -- start 2024-09-21T06:37:24.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.164+0000 7f8721119640 1 -- start start 2024-09-21T06:37:24.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.164+0000 7f8721119640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f871c102200 0x7f871c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:24.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.164+0000 7f8721119640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f871c107b40 0x7f871c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:24.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.164+0000 7f8721119640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f871c108490 0x7f871c19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:24.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.164+0000 7f8721119640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f871c111780 con 0x7f871c102200 2024-09-21T06:37:24.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.165+0000 7f8721119640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f871c111600 con 0x7f871c108490 2024-09-21T06:37:24.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.165+0000 7f8721119640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f871c111900 con 0x7f871c107b40 2024-09-21T06:37:24.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.165+0000 7f871ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f871c102200 0x7f871c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:24.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.165+0000 7f871ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f871c102200 0x7f871c19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41822/0 (socket says 172.21.15.67:41822) 2024-09-21T06:37:24.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.165+0000 7f871ad9d640 1 -- 172.21.15.67:0/2453451092 learned_addr learned my addr 172.21.15.67:0/2453451092 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:37:24.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.165+0000 7f871b59e640 1 --2- 172.21.15.67:0/2453451092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f871c108490 0x7f871c19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:24.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.165+0000 7f871a59c640 1 --2- 172.21.15.67:0/2453451092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f871c107b40 0x7f871c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:24.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.165+0000 7f871ad9d640 1 -- 172.21.15.67:0/2453451092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f871c107b40 msgr2=0x7f871c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:24.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.165+0000 7f871ad9d640 1 --2- 172.21.15.67:0/2453451092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f871c107b40 0x7f871c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.165+0000 7f871ad9d640 1 -- 172.21.15.67:0/2453451092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f871c108490 msgr2=0x7f871c19fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:24.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.165+0000 7f871ad9d640 1 --2- 172.21.15.67:0/2453451092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f871c108490 0x7f871c19fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.165+0000 7f871ad9d640 1 -- 172.21.15.67:0/2453451092 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871c1a03d0 con 0x7f871c102200 2024-09-21T06:37:24.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.165+0000 7f871a59c640 1 --2- 172.21.15.67:0/2453451092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f871c107b40 0x7f871c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:37:24.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.165+0000 7f871ad9d640 1 --2- 172.21.15.67:0/2453451092 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f871c102200 0x7f871c19a460 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f870c00e9f0 tx=0x7f870c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:37:24.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.166+0000 7f8703fff640 1 -- 172.21.15.67:0/2453451092 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c00cd80 con 0x7f871c102200 2024-09-21T06:37:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.166+0000 7f8721119640 1 -- 172.21.15.67:0/2453451092 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f871c1a06c0 con 0x7f871c102200 2024-09-21T06:37:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.166+0000 7f8703fff640 1 -- 172.21.15.67:0/2453451092 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f870c0108c0 con 0x7f871c102200 2024-09-21T06:37:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.166+0000 7f8703fff640 1 -- 172.21.15.67:0/2453451092 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c0187c0 con 0x7f871c102200 2024-09-21T06:37:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.166+0000 7f8721119640 1 -- 172.21.15.67:0/2453451092 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f871c1a6e20 con 0x7f871c102200 2024-09-21T06:37:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.169+0000 7f872768d740 1 Processor -- start 2024-09-21T06:37:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.169+0000 7f872768d740 1 -- start start 2024-09-21T06:37:24.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.169+0000 7f872768d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2e334c840 0x55a2e36f3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:24.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.169+0000 7f8703fff640 1 -- 172.21.15.67:0/2453451092 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f870c018980 con 0x7f871c102200 2024-09-21T06:37:24.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.169+0000 7f872768d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e3323980 0x55a2e36f3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:24.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.169+0000 7f872768d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e332bb60 0x55a2e36f4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:24.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.169+0000 7f872768d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a2e3669b90 con 0x55a2e334c840 2024-09-21T06:37:24.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.169+0000 7f872768d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55a2e3658fc0 con 0x55a2e3323980 2024-09-21T06:37:24.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.169+0000 7f872768d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a2e365cdd0 con 0x55a2e332bb60 2024-09-21T06:37:24.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.169+0000 7f871b59e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2e334c840 0x55a2e36f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:24.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.169+0000 7f871b59e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2e334c840 0x55a2e36f3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41832/0 (socket says 172.21.15.67:41832) 2024-09-21T06:37:24.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.169+0000 7f871b59e640 1 -- 172.21.15.67:0/3969722256 learned_addr learned my addr 172.21.15.67:0/3969722256 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:37:24.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.169+0000 7f871a59c640 1 --2- 172.21.15.67:0/3969722256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e332bb60 0x55a2e36f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:24.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.169+0000 7f871ad9d640 1 --2- 172.21.15.67:0/3969722256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e3323980 0x55a2e36f3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:24.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.170+0000 7f871b59e640 1 -- 172.21.15.67:0/3969722256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e332bb60 msgr2=0x55a2e36f4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:37:24.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.170+0000 7f871b59e640 1 --2- 172.21.15.67:0/3969722256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e332bb60 0x55a2e36f4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.170+0000 7f871b59e640 1 -- 172.21.15.67:0/3969722256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e3323980 msgr2=0x55a2e36f3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:37:24.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.170+0000 7f871b59e640 1 --2- 172.21.15.67:0/3969722256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e3323980 0x55a2e36f3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.170+0000 7f871b59e640 1 -- 172.21.15.67:0/3969722256 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2e35e0060 con 0x55a2e334c840 2024-09-21T06:37:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.170+0000 7f871b59e640 1 --2- 172.21.15.67:0/3969722256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2e334c840 0x55a2e36f3600 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f871400ae10 tx=0x7f871400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75fda473667a6529 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:37:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.170+0000 7f87017fa640 1 -- 172.21.15.67:0/3969722256 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8714003b70 con 0x55a2e334c840 2024-09-21T06:37:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.170+0000 7f87017fa640 1 -- 172.21.15.67:0/3969722256 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8714003d10 con 0x55a2e334c840 2024-09-21T06:37:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.170+0000 7f87017fa640 1 -- 172.21.15.67:0/3969722256 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8714010930 con 0x55a2e334c840 2024-09-21T06:37:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.171+0000 7f8703fff640 1 --2- 172.21.15.67:0/2453451092 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f87040b6510 0x7f87040b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.171+0000 7f871ad9d640 1 --2- 172.21.15.67:0/2453451092 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f87040b6510 0x7f87040b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.171+0000 7f8703fff640 1 -- 172.21.15.67:0/2453451092 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f870c0e7b30 con 0x7f871c102200 2024-09-21T06:37:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.171+0000 7f872768d740 1 -- 172.21.15.67:0/3969722256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2e334c840 msgr2=0x55a2e36f3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.171+0000 7f872768d740 1 --2- 172.21.15.67:0/3969722256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2e334c840 0x55a2e36f3600 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f871400ae10 tx=0x7f871400f790 comp rx=0 tx=0).stop 2024-09-21T06:37:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.171+0000 7f872768d740 1 -- 172.21.15.67:0/3969722256 shutdown_connections 2024-09-21T06:37:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.172+0000 7f872768d740 1 --2- 172.21.15.67:0/3969722256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e3323980 0x55a2e36f3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.172+0000 7f872768d740 1 --2- 172.21.15.67:0/3969722256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e332bb60 0x55a2e36f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.172+0000 7f872768d740 1 --2- 172.21.15.67:0/3969722256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2e334c840 0x55a2e36f3600 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.172+0000 7f872768d740 1 -- 172.21.15.67:0/3969722256 >> 172.21.15.67:0/3969722256 conn(0x55a2e3340990 msgr2=0x55a2e36f14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:37:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.172+0000 7f871ad9d640 1 --2- 172.21.15.67:0/2453451092 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f87040b6510 0x7f87040b8a20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55a2e36f2280 tx=0x7f870c0eb610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:37:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.172+0000 7f872768d740 1 -- 172.21.15.67:0/3969722256 shutdown_connections 2024-09-21T06:37:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.172+0000 7f872768d740 1 -- 172.21.15.67:0/3969722256 wait complete. 2024-09-21T06:37:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.172+0000 7f872768d740 1 -- start start 2024-09-21T06:37:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.172+0000 7f872768d740 1 Processor -- start 2024-09-21T06:37:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.173+0000 7f872768d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e3323980 0x55a2e36ff1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.173+0000 7f872768d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2e332bb60 0x55a2e36ff6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.173+0000 7f872768d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e334c840 0x55a2e370b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.173+0000 7f872768d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a2e34ba7a0 con 0x55a2e332bb60 2024-09-21T06:37:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.173+0000 7f872768d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55a2e35fe0a0 con 0x55a2e3323980 2024-09-21T06:37:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.173+0000 7f872768d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a2e3701e30 con 0x55a2e334c840 2024-09-21T06:37:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.173+0000 7f871a59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2e332bb60 0x55a2e36ff6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.173+0000 7f872768d740 10 client.0 _ll_register_callbacks cb 0x55a2e35db3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:37:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.173+0000 7f871a59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2e332bb60 0x55a2e36ff6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41848/0 (socket says 172.21.15.67:41848) 2024-09-21T06:37:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.173+0000 7f871a59c640 1 -- 172.21.15.67:0/2153621605 learned_addr learned my addr 172.21.15.67:0/2153621605 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:37:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.173+0000 7f871b59e640 1 --2- 172.21.15.67:0/2153621605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e3323980 0x55a2e36ff1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.173+0000 7f871b59e640 1 --2- 172.21.15.67:0/2153621605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e334c840 0x55a2e370b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.173+0000 7f871a59c640 1 -- 172.21.15.67:0/2153621605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e334c840 msgr2=0x55a2e370b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:37:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.173+0000 7f871a59c640 1 --2- 172.21.15.67:0/2153621605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e334c840 0x55a2e370b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.173+0000 7f871a59c640 1 -- 172.21.15.67:0/2153621605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e3323980 msgr2=0x55a2e36ff1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:37:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.173+0000 7f871a59c640 1 --2- 172.21.15.67:0/2153621605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e3323980 0x55a2e36ff1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.173+0000 7f871a59c640 1 -- 172.21.15.67:0/2153621605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2e35ec880 con 0x55a2e332bb60 2024-09-21T06:37:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.174+0000 7f871a59c640 1 --2- 172.21.15.67:0/2153621605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2e332bb60 0x55a2e36ff6e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8710006010 tx=0x7f8710030740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5011b44f3463e490 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:37:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.174+0000 7f86feff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:37:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.174+0000 7f86feff5640 1 -- 172.21.15.67:0/2153621605 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8710036040 con 0x55a2e332bb60 2024-09-21T06:37:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.174+0000 7f86feff5640 1 -- 172.21.15.67:0/2153621605 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f8710004250 con 0x55a2e332bb60 2024-09-21T06:37:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.174+0000 7f872768d740 1 -- 172.21.15.67:0/2153621605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8714003d10 con 0x55a2e332bb60 2024-09-21T06:37:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.174+0000 7f86feff5640 1 -- 172.21.15.67:0/2153621605 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f871004d050 con 0x55a2e332bb60 2024-09-21T06:37:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.174+0000 7f872768d740 10 client.15456 fetch_fsmap learned FSMap version 21 2024-09-21T06:37:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.174+0000 7f872768d740 1 -- 172.21.15.67:0/2153621605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a2e33c6b90 con 0x55a2e332bb60 2024-09-21T06:37:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.174+0000 7f86feff5640 1 -- 172.21.15.67:0/2153621605 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8714003d10 con 0x55a2e332bb60 2024-09-21T06:37:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.175+0000 7f86feff5640 1 -- 172.21.15.67:0/2153621605 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f871000b040 con 0x55a2e332bb60 2024-09-21T06:37:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.175+0000 7f872768d740 10 client.15456 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:37:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.175+0000 7f872768d740 10 client.15456 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:37:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.175+0000 7f872768d740 20 client.15456 populate_metadata read hostname 'smithi067' 2024-09-21T06:37:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.175+0000 7f872768d740 1 --2- 172.21.15.67:0/2153621605 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55a2e330b1c0 0x55a2e3711160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.175+0000 7f872768d740 4 client.15456 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:37:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.175+0000 7f872768d740 1 -- 172.21.15.67:0/2153621605 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2e366bae0 con 0x55a2e330b1c0 2024-09-21T06:37:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.175+0000 7f871b59e640 1 --2- 172.21.15.67:0/2153621605 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55a2e330b1c0 0x55a2e3711160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.176+0000 7f86feff5640 10 client.15456 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:37:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.176+0000 7f871b59e640 1 --2- 172.21.15.67:0/2153621605 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55a2e330b1c0 0x55a2e3711160 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17bd4f553c520938 server_cookie=f788930d70be3a72 in_seq=0 out_seq=0 2024-09-21T06:37:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.177+0000 7f86feff5640 1 -- 172.21.15.67:0/2153621605 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a2e366bae0 con 0x55a2e330b1c0 2024-09-21T06:37:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.177+0000 7f86feff5640 10 client.15456 handle_command_reply: tid=0 2024-09-21T06:37:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.177+0000 7f872768d740 1 client.15456 shutdown 2024-09-21T06:37:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.177+0000 7f872768d740 1 -- 172.21.15.67:0/2153621605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2e332bb60 msgr2=0x55a2e36ff6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.177+0000 7f872768d740 1 --2- 172.21.15.67:0/2153621605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2e332bb60 0x55a2e36ff6e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8710006010 tx=0x7f8710030740 comp rx=0 tx=0).stop 2024-09-21T06:37:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.177+0000 7f872768d740 1 -- 172.21.15.67:0/2153621605 shutdown_connections 2024-09-21T06:37:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.177+0000 7f872768d740 1 --2- 172.21.15.67:0/2153621605 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55a2e330b1c0 0x55a2e3711160 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.177+0000 7f872768d740 1 --2- 172.21.15.67:0/2153621605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e334c840 0x55a2e370b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.177+0000 7f86feff5640 0 client.15456 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:37:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.177+0000 7f872768d740 1 --2- 172.21.15.67:0/2153621605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2e332bb60 0x55a2e36ff6e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.177+0000 7f872768d740 1 --2- 172.21.15.67:0/2153621605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e3323980 0x55a2e36ff1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.177+0000 7f872768d740 1 -- 172.21.15.67:0/2153621605 >> 172.21.15.67:0/2153621605 conn(0x55a2e3340990 msgr2=0x55a2e36f4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:37:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.177+0000 7f872768d740 1 -- 172.21.15.67:0/2153621605 shutdown_connections 2024-09-21T06:37:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.177+0000 7f872768d740 1 -- 172.21.15.67:0/2153621605 wait complete. 2024-09-21T06:37:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.177+0000 7f872768d740 20 client.15456 trim_cache size 0 max 16384 2024-09-21T06:37:24.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.195+0000 7f872768d740 1 Processor -- start 2024-09-21T06:37:24.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.195+0000 7f872768d740 1 -- start start 2024-09-21T06:37:24.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.195+0000 7f872768d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2e330b1c0 0x55a2e36f4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:24.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.195+0000 7f872768d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e3323980 0x55a2e370b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:24.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.195+0000 7f872768d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e332bb60 0x55a2e370b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:24.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.195+0000 7f872768d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f871c1aec40 con 0x55a2e330b1c0 2024-09-21T06:37:24.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.195+0000 7f872768d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f871c1b14c0 con 0x55a2e332bb60 2024-09-21T06:37:24.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.195+0000 7f872768d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f871c1af2b0 con 0x55a2e3323980 2024-09-21T06:37:24.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.195+0000 7f871b59e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2e330b1c0 0x55a2e36f4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:24.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.195+0000 7f871b59e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2e330b1c0 0x55a2e36f4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41852/0 (socket says 172.21.15.67:41852) 2024-09-21T06:37:24.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.195+0000 7f871b59e640 1 -- 172.21.15.67:0/999425180 learned_addr learned my addr 172.21.15.67:0/999425180 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:37:24.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.195+0000 7f871a59c640 1 --2- 172.21.15.67:0/999425180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e3323980 0x55a2e370b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:24.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.195+0000 7f871b59e640 1 --2- 172.21.15.67:0/999425180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e332bb60 0x55a2e370b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:24.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.196+0000 7f871b59e640 1 -- 172.21.15.67:0/999425180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e3323980 msgr2=0x55a2e370b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:37:24.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.196+0000 7f871b59e640 1 --2- 172.21.15.67:0/999425180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e3323980 0x55a2e370b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.196+0000 7f871b59e640 1 -- 172.21.15.67:0/999425180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e332bb60 msgr2=0x55a2e370b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:37:24.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.196+0000 7f871b59e640 1 --2- 172.21.15.67:0/999425180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e332bb60 0x55a2e370b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.196+0000 7f871b59e640 1 -- 172.21.15.67:0/999425180 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2e36fd4a0 con 0x55a2e330b1c0 2024-09-21T06:37:24.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.196+0000 7f871b59e640 1 --2- 172.21.15.67:0/999425180 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2e330b1c0 0x55a2e36f4d40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f871400ae10 tx=0x7f871400a4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1aa69276f140d4a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:37:24.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.196+0000 7f8700ff9640 1 -- 172.21.15.67:0/999425180 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8714004c00 con 0x55a2e330b1c0 2024-09-21T06:37:24.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.196+0000 7f8700ff9640 1 -- 172.21.15.67:0/999425180 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87140114d0 con 0x55a2e330b1c0 2024-09-21T06:37:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.196+0000 7f8700ff9640 1 -- 172.21.15.67:0/999425180 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8714011dd0 con 0x55a2e330b1c0 2024-09-21T06:37:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.197+0000 7f872768d740 1 -- 172.21.15.67:0/999425180 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2e330b1c0 msgr2=0x55a2e36f4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.197+0000 7f872768d740 1 --2- 172.21.15.67:0/999425180 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2e330b1c0 0x55a2e36f4d40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f871400ae10 tx=0x7f871400a4d0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.197+0000 7f872768d740 1 -- 172.21.15.67:0/999425180 shutdown_connections 2024-09-21T06:37:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.197+0000 7f872768d740 1 --2- 172.21.15.67:0/999425180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e332bb60 0x55a2e370b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.197+0000 7f872768d740 1 --2- 172.21.15.67:0/999425180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e3323980 0x55a2e370b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.197+0000 7f872768d740 1 --2- 172.21.15.67:0/999425180 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2e330b1c0 0x55a2e36f4d40 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.197+0000 7f872768d740 1 -- 172.21.15.67:0/999425180 >> 172.21.15.67:0/999425180 conn(0x55a2e3340990 msgr2=0x55a2e36f4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:37:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.197+0000 7f872768d740 1 -- 172.21.15.67:0/999425180 shutdown_connections 2024-09-21T06:37:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.197+0000 7f872768d740 1 -- 172.21.15.67:0/999425180 wait complete. 2024-09-21T06:37:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.198+0000 7f872768d740 1 -- start start 2024-09-21T06:37:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.198+0000 7f872768d740 1 Processor -- start 2024-09-21T06:37:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.198+0000 7f872768d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e330b1c0 0x55a2e3707800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.198+0000 7f872768d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e3323980 0x55a2e3703d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:24.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.198+0000 7f872768d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2e332bb60 0x55a2e3704250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:24.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.198+0000 7f872768d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f871c1af980 con 0x55a2e332bb60 2024-09-21T06:37:24.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.198+0000 7f872768d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f871c1b0df0 con 0x55a2e330b1c0 2024-09-21T06:37:24.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.198+0000 7f872768d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f871c1b0050 con 0x55a2e3323980 2024-09-21T06:37:24.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.198+0000 7f872768d740 10 client.0 _ll_register_callbacks cb 0x7f871000a890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:37:24.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.198+0000 7f871b59e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2e332bb60 0x55a2e3704250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:24.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.198+0000 7f871b59e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2e332bb60 0x55a2e3704250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41862/0 (socket says 172.21.15.67:41862) 2024-09-21T06:37:24.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.198+0000 7f871b59e640 1 -- 172.21.15.67:0/1358323000 learned_addr learned my addr 172.21.15.67:0/1358323000 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:37:24.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.199+0000 7f871a59c640 1 --2- 172.21.15.67:0/1358323000 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e3323980 0x55a2e3703d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:24.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.199+0000 7f871b59e640 1 --2- 172.21.15.67:0/1358323000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e330b1c0 0x55a2e3707800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:24.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.199+0000 7f871b59e640 1 -- 172.21.15.67:0/1358323000 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e3323980 msgr2=0x55a2e3703d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:37:24.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.199+0000 7f871b59e640 1 --2- 172.21.15.67:0/1358323000 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e3323980 0x55a2e3703d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.199+0000 7f871b59e640 1 -- 172.21.15.67:0/1358323000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e330b1c0 msgr2=0x55a2e3707800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:37:24.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.199+0000 7f871b59e640 1 --2- 172.21.15.67:0/1358323000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e330b1c0 0x55a2e3707800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.199+0000 7f871b59e640 1 -- 172.21.15.67:0/1358323000 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2e3704940 con 0x55a2e332bb60 2024-09-21T06:37:24.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.199+0000 7f871b59e640 1 --2- 172.21.15.67:0/1358323000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2e332bb60 0x55a2e3704250 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8714011ab0 tx=0x7f8714004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17fda1d1b91f96f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:37:24.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.199+0000 7f86feff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:37:24.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.199+0000 7f86feff5640 1 -- 172.21.15.67:0/1358323000 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8714042060 con 0x55a2e332bb60 2024-09-21T06:37:24.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.199+0000 7f86feff5640 1 -- 172.21.15.67:0/1358323000 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f8714004a00 con 0x55a2e332bb60 2024-09-21T06:37:24.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.199+0000 7f872768d740 1 -- 172.21.15.67:0/1358323000 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87140114d0 con 0x55a2e332bb60 2024-09-21T06:37:24.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.199+0000 7f86feff5640 1 -- 172.21.15.67:0/1358323000 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8714004c00 con 0x55a2e332bb60 2024-09-21T06:37:24.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.199+0000 7f872768d740 10 client.15462 fetch_fsmap learned FSMap version 21 2024-09-21T06:37:24.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.199+0000 7f872768d740 1 -- 172.21.15.67:0/1358323000 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a2e377c9f0 con 0x55a2e332bb60 2024-09-21T06:37:24.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.200+0000 7f86feff5640 1 -- 172.21.15.67:0/1358323000 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87140114d0 con 0x55a2e332bb60 2024-09-21T06:37:24.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.200+0000 7f86feff5640 1 -- 172.21.15.67:0/1358323000 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f8714034070 con 0x55a2e332bb60 2024-09-21T06:37:24.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.200+0000 7f872768d740 10 client.15462 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:37:24.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.200+0000 7f872768d740 10 client.15462 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:37:24.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.200+0000 7f872768d740 20 client.15462 populate_metadata read hostname 'smithi067' 2024-09-21T06:37:24.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.200+0000 7f872768d740 1 --2- 172.21.15.67:0/1358323000 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55a2e334c840 0x55a2e377eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:24.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.200+0000 7f872768d740 4 client.15462 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 134.5, "await_for": 45.0}] 2024-09-21T06:37:24.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.200+0000 7f872768d740 1 -- 172.21.15.67:0/1358323000 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 134.5, "await_for": 45.0}) -- 0x55a2e3667590 con 0x55a2e334c840 2024-09-21T06:37:24.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.200+0000 7f871a59c640 1 --2- 172.21.15.67:0/1358323000 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55a2e334c840 0x55a2e377eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:24.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.201+0000 7f86feff5640 10 client.15462 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:37:24.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.201+0000 7f871a59c640 1 --2- 172.21.15.67:0/1358323000 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55a2e334c840 0x55a2e377eea0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96523012025d80f0 server_cookie=f9d4c8781a1b6ebc in_seq=0 out_seq=0 2024-09-21T06:37:24.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.207+0000 7f86feff5640 1 -- 172.21.15.67:0/1358323000 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55a2e3667590 con 0x55a2e334c840 2024-09-21T06:37:24.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.207+0000 7f86feff5640 10 client.15462 handle_command_reply: tid=0 2024-09-21T06:37:24.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.207+0000 7f872768d740 1 client.15462 shutdown 2024-09-21T06:37:24.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.207+0000 7f872768d740 1 -- 172.21.15.67:0/1358323000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2e332bb60 msgr2=0x55a2e3704250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:24.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.207+0000 7f872768d740 1 --2- 172.21.15.67:0/1358323000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2e332bb60 0x55a2e3704250 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8714011ab0 tx=0x7f8714004920 comp rx=0 tx=0).stop 2024-09-21T06:37:24.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.207+0000 7f872768d740 1 -- 172.21.15.67:0/1358323000 shutdown_connections 2024-09-21T06:37:24.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.207+0000 7f872768d740 1 --2- 172.21.15.67:0/1358323000 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55a2e334c840 0x55a2e377eea0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.207+0000 7f872768d740 1 --2- 172.21.15.67:0/1358323000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2e332bb60 0x55a2e3704250 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.207+0000 7f872768d740 1 --2- 172.21.15.67:0/1358323000 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e3323980 0x55a2e3703d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.207+0000 7f872768d740 1 --2- 172.21.15.67:0/1358323000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e330b1c0 0x55a2e3707800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.207+0000 7f872768d740 1 -- 172.21.15.67:0/1358323000 >> 172.21.15.67:0/1358323000 conn(0x55a2e3340990 msgr2=0x55a2e36f4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:37:24.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.207+0000 7f86feff5640 0 client.15462 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:37:24.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.208+0000 7f872768d740 1 -- 172.21.15.67:0/1358323000 shutdown_connections 2024-09-21T06:37:24.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.208+0000 7f872768d740 1 -- 172.21.15.67:0/1358323000 wait complete. 2024-09-21T06:37:24.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.208+0000 7f872768d740 20 client.15462 trim_cache size 0 max 16384 2024-09-21T06:37:24.210 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-21T06:37:24.210 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 21, 2024-09-21T06:37:24.210 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-09-21T06:37:24.211 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 2, 2024-09-21T06:37:24.211 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-21T06:37:24.211 INFO:teuthology.orchestra.run.smithi067.stdout: "6c8619e7": { 2024-09-21T06:37:24.211 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 2, 2024-09-21T06:37:24.211 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-21T06:37:24.211 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:37:24.211 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-21T06:37:24.211 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:37:24.211 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:37:24.211 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-21T06:37:24.211 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 134.5, 2024-09-21T06:37:24.211 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-21T06:37:24.211 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-21T06:37:24.211 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-21T06:37:24.211 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:37:24.212 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-21T06:37:24.212 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:37:24.212 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:37:24.212 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:37:24.212 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:37:24.212 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:37:24.212 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:37:24.212 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-21T06:37:24.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.208+0000 7f8721119640 1 -- 172.21.15.67:0/2453451092 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f87040b6510 msgr2=0x7f87040b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:24.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.208+0000 7f8721119640 1 --2- 172.21.15.67:0/2453451092 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f87040b6510 0x7f87040b8a20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55a2e36f2280 tx=0x7f870c0eb610 comp rx=0 tx=0).stop 2024-09-21T06:37:24.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.208+0000 7f8721119640 1 -- 172.21.15.67:0/2453451092 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f871c102200 msgr2=0x7f871c19a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:24.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.208+0000 7f8721119640 1 --2- 172.21.15.67:0/2453451092 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f871c102200 0x7f871c19a460 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f870c00e9f0 tx=0x7f870c00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.209+0000 7f8721119640 1 -- 172.21.15.67:0/2453451092 shutdown_connections 2024-09-21T06:37:24.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.209+0000 7f8721119640 1 --2- 172.21.15.67:0/2453451092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f871c108490 0x7f871c19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.209+0000 7f8721119640 1 --2- 172.21.15.67:0/2453451092 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f87040b6510 0x7f87040b8a20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.209+0000 7f8721119640 1 --2- 172.21.15.67:0/2453451092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f871c107b40 0x7f871c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.209+0000 7f8721119640 1 --2- 172.21.15.67:0/2453451092 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f871c102200 0x7f871c19a460 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:24.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.209+0000 7f8721119640 1 -- 172.21.15.67:0/2453451092 >> 172.21.15.67:0/2453451092 conn(0x7f871c079f50 msgr2=0x7f871c103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:37:24.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.209+0000 7f8721119640 1 -- 172.21.15.67:0/2453451092 shutdown_connections 2024-09-21T06:37:24.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:24.209+0000 7f8721119640 1 -- 172.21.15.67:0/2453451092 wait complete. 2024-09-21T06:37:25.068 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6c8619e7', quiesce took 3.7 seconds. Will release after: 10.8 2024-09-21T06:37:25.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:25 smithi067 ceph-mon[31232]: pgmap v385: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 1 op/s 2024-09-21T06:37:25.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:25 smithi100 ceph-mon[33401]: pgmap v385: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 1 op/s 2024-09-21T06:37:25.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:25 smithi203 ceph-mon[33735]: pgmap v385: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 1 op/s 2024-09-21T06:37:27.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:27 smithi100 ceph-mon[33401]: pgmap v386: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-21T06:37:27.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:27 smithi203 ceph-mon[33735]: pgmap v386: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-21T06:37:28.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:27 smithi067 ceph-mon[31232]: pgmap v386: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-21T06:37:29.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:29 smithi100 ceph-mon[33401]: pgmap v387: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-21T06:37:29.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:29 smithi203 ceph-mon[33735]: pgmap v387: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-21T06:37:30.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:29 smithi067 ceph-mon[31232]: pgmap v387: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-21T06:37:31.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:31 smithi100 ceph-mon[33401]: pgmap v388: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-21T06:37:31.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:31 smithi203 ceph-mon[33735]: pgmap v388: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-21T06:37:32.041 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:31 smithi067 ceph-mon[31232]: pgmap v388: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-21T06:37:33.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:33 smithi100 ceph-mon[33401]: pgmap v389: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T06:37:33.984 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:33 smithi203 ceph-mon[33735]: pgmap v389: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T06:37:34.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:33 smithi067 ceph-mon[31232]: pgmap v389: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T06:37:34.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:34 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:37:34.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:34 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:37:34.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:34 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:37:34.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:34 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:37:34.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:34 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:37:34.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:34 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:37:34.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:34 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:37:35.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:34 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:37:35.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:34 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:37:35.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:34 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:37:35.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:34 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:37:35.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:34 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:37:35.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:34 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:37:35.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:34 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:37:35.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:34 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:37:35.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:34 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:37:35.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:34 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:37:35.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:34 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:37:35.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:34 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:37:35.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:34 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:37:35.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:34 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:37:35.869 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:37:35.892 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:35 smithi067 ceph-mon[31232]: pgmap v390: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-21T06:37:35.892 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:37:35.892 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:37:35.892 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:37:35.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:35 smithi203 ceph-mon[33735]: pgmap v390: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-21T06:37:35.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:35 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:37:35.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:35 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:37:35.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:35 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:37:36.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:35 smithi100 ceph-mon[33401]: pgmap v390: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-21T06:37:36.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:35 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:37:36.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:35 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:37:36.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:35 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:37:36.159 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:37:36.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.699+0000 7ff46ed86640 1 -- 172.21.15.67:0/1908140964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff468108d30 msgr2=0x7ff46810f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:36.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.699+0000 7ff46ed86640 1 --2- 172.21.15.67:0/1908140964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff468108d30 0x7ff46810f560 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff460009ca0 tx=0x7ff46002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:37:36.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.699+0000 7ff46ed86640 1 -- 172.21.15.67:0/1908140964 shutdown_connections 2024-09-21T06:37:36.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.699+0000 7ff46ed86640 1 --2- 172.21.15.67:0/1908140964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff468108d30 0x7ff46810f560 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:36.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.699+0000 7ff46ed86640 1 --2- 172.21.15.67:0/1908140964 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4681083e0 0x7ff4681087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:36.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.699+0000 7ff46ed86640 1 --2- 172.21.15.67:0/1908140964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff468102aa0 0x7ff468102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:36.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.699+0000 7ff46ed86640 1 -- 172.21.15.67:0/1908140964 >> 172.21.15.67:0/1908140964 conn(0x7ff468079750 msgr2=0x7ff468079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:37:36.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.700+0000 7ff46ed86640 1 -- 172.21.15.67:0/1908140964 shutdown_connections 2024-09-21T06:37:36.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.700+0000 7ff46ed86640 1 -- 172.21.15.67:0/1908140964 wait complete. 2024-09-21T06:37:36.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.701+0000 7ff46ed86640 1 Processor -- start 2024-09-21T06:37:36.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.701+0000 7ff46ed86640 1 -- start start 2024-09-21T06:37:36.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.701+0000 7ff46ed86640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff468102aa0 0x7ff46806c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:36.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.701+0000 7ff46ed86640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4681083e0 0x7ff468070180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:36.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.701+0000 7ff46ed86640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff468108d30 0x7ff468070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:36.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.701+0000 7ff46ed86640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff4681128f0 con 0x7ff468102aa0 2024-09-21T06:37:36.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.701+0000 7ff46ed86640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff468112770 con 0x7ff468108d30 2024-09-21T06:37:36.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.701+0000 7ff46ed86640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff468112a70 con 0x7ff4681083e0 2024-09-21T06:37:36.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.701+0000 7ff46cb22640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff468102aa0 0x7ff46806c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:36.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.701+0000 7ff46cb22640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff468102aa0 0x7ff46806c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44326/0 (socket says 172.21.15.67:44326) 2024-09-21T06:37:36.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.702+0000 7ff46cb22640 1 -- 172.21.15.67:0/3836844653 learned_addr learned my addr 172.21.15.67:0/3836844653 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:37:36.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.702+0000 7ff467fff640 1 --2- 172.21.15.67:0/3836844653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4681083e0 0x7ff468070180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:36.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.702+0000 7ff46d323640 1 --2- 172.21.15.67:0/3836844653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff468108d30 0x7ff468070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:36.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.702+0000 7ff46cb22640 1 -- 172.21.15.67:0/3836844653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4681083e0 msgr2=0x7ff468070180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:36.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.702+0000 7ff46cb22640 1 --2- 172.21.15.67:0/3836844653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4681083e0 0x7ff468070180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:36.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.702+0000 7ff46cb22640 1 -- 172.21.15.67:0/3836844653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff468108d30 msgr2=0x7ff468070770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:36.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.702+0000 7ff46cb22640 1 --2- 172.21.15.67:0/3836844653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff468108d30 0x7ff468070770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:36.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.702+0000 7ff46cb22640 1 -- 172.21.15.67:0/3836844653 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff468070f90 con 0x7ff468102aa0 2024-09-21T06:37:36.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.702+0000 7ff46cb22640 1 --2- 172.21.15.67:0/3836844653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff468102aa0 0x7ff46806c360 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff45c00d950 tx=0x7ff45c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:37:36.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.702+0000 7ff465ffb640 1 -- 172.21.15.67:0/3836844653 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c017070 con 0x7ff468102aa0 2024-09-21T06:37:36.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.703+0000 7ff46ed86640 1 -- 172.21.15.67:0/3836844653 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4681aef20 con 0x7ff468102aa0 2024-09-21T06:37:36.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.703+0000 7ff465ffb640 1 -- 172.21.15.67:0/3836844653 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff45c009c30 con 0x7ff468102aa0 2024-09-21T06:37:36.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.703+0000 7ff465ffb640 1 -- 172.21.15.67:0/3836844653 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c00f620 con 0x7ff468102aa0 2024-09-21T06:37:36.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.703+0000 7ff46ed86640 1 -- 172.21.15.67:0/3836844653 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4681af420 con 0x7ff468102aa0 2024-09-21T06:37:36.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.704+0000 7ff46ed86640 1 -- 172.21.15.67:0/3836844653 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff468102eb0 con 0x7ff468102aa0 2024-09-21T06:37:36.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.706+0000 7ff465ffb640 1 -- 172.21.15.67:0/3836844653 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7ff45c005030 con 0x7ff468102aa0 2024-09-21T06:37:36.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.707+0000 7ff465ffb640 1 --2- 172.21.15.67:0/3836844653 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7ff4580b6510 0x7ff4580b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:36.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.707+0000 7ff467fff640 1 --2- 172.21.15.67:0/3836844653 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7ff4580b6510 0x7ff4580b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:36.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.707+0000 7ff467fff640 1 --2- 172.21.15.67:0/3836844653 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7ff4580b6510 0x7ff4580b8a20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff46807a180 tx=0x7ff454005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:37:36.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.708+0000 7ff465ffb640 1 -- 172.21.15.67:0/3836844653 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7ff45c0e72b0 con 0x7ff468102aa0 2024-09-21T06:37:36.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.713+0000 7ff465ffb640 1 -- 172.21.15.67:0/3836844653 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff45c0ad770 con 0x7ff468102aa0 2024-09-21T06:37:36.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.944+0000 7ff46ed86640 1 -- 172.21.15.67:0/3836844653 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff46806bc50 con 0x7ff468102aa0 2024-09-21T06:37:36.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.945+0000 7ff465ffb640 1 -- 172.21.15.67:0/3836844653 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7ff45c009de0 con 0x7ff468102aa0 2024-09-21T06:37:36.947 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:37:36.948 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:37:36.948 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:37:36.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.948+0000 7ff46ed86640 1 -- 172.21.15.67:0/3836844653 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7ff4580b6510 msgr2=0x7ff4580b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:36.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.948+0000 7ff46ed86640 1 --2- 172.21.15.67:0/3836844653 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7ff4580b6510 0x7ff4580b8a20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff46807a180 tx=0x7ff454005f90 comp rx=0 tx=0).stop 2024-09-21T06:37:36.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.948+0000 7ff46ed86640 1 -- 172.21.15.67:0/3836844653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff468102aa0 msgr2=0x7ff46806c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:36.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.948+0000 7ff46ed86640 1 --2- 172.21.15.67:0/3836844653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff468102aa0 0x7ff46806c360 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff45c00d950 tx=0x7ff45c00de10 comp rx=0 tx=0).stop 2024-09-21T06:37:36.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.948+0000 7ff46ed86640 1 -- 172.21.15.67:0/3836844653 shutdown_connections 2024-09-21T06:37:36.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.948+0000 7ff46ed86640 1 --2- 172.21.15.67:0/3836844653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff468108d30 0x7ff468070770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:36.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.948+0000 7ff46ed86640 1 --2- 172.21.15.67:0/3836844653 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7ff4580b6510 0x7ff4580b8a20 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:36.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.948+0000 7ff46ed86640 1 --2- 172.21.15.67:0/3836844653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4681083e0 0x7ff468070180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:36.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.948+0000 7ff46ed86640 1 --2- 172.21.15.67:0/3836844653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff468102aa0 0x7ff46806c360 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:36.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.948+0000 7ff46ed86640 1 -- 172.21.15.67:0/3836844653 >> 172.21.15.67:0/3836844653 conn(0x7ff468079750 msgr2=0x7ff468106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:37:36.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.948+0000 7ff46ed86640 1 -- 172.21.15.67:0/3836844653 shutdown_connections 2024-09-21T06:37:36.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:36.949+0000 7ff46ed86640 1 -- 172.21.15.67:0/3836844653 wait complete. 2024-09-21T06:37:37.757 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db --set-id 6c8619e7 --release --await' 2024-09-21T06:37:37.758 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24671 quiesce db --set-id 6c8619e7 --release --await 2024-09-21T06:37:37.821 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:37 smithi067 ceph-mon[31232]: pgmap v391: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T06:37:37.822 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:37 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3836844653' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:37:37.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:37 smithi203 ceph-mon[33735]: pgmap v391: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T06:37:37.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:37 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3836844653' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:37:38.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:37 smithi100 ceph-mon[33401]: pgmap v391: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T06:37:38.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:37 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3836844653' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:37:38.138 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:37:38.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.758+0000 7f4d89498640 1 -- 172.21.15.67:0/828426486 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d84108cf0 msgr2=0x7f4d8410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:38.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.758+0000 7f4d89498640 1 --2- 172.21.15.67:0/828426486 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d84108cf0 0x7f4d8410f520 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c009ca0 tx=0x7f4d7c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.758+0000 7f4d89498640 1 -- 172.21.15.67:0/828426486 shutdown_connections 2024-09-21T06:37:38.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.758+0000 7f4d89498640 1 --2- 172.21.15.67:0/828426486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d841083a0 0x7f4d841087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.758+0000 7f4d89498640 1 --2- 172.21.15.67:0/828426486 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d84108cf0 0x7f4d8410f520 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.758+0000 7f4d89498640 1 --2- 172.21.15.67:0/828426486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d84102a60 0x7f4d84102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.758+0000 7f4d89498640 1 -- 172.21.15.67:0/828426486 >> 172.21.15.67:0/828426486 conn(0x7f4d84079730 msgr2=0x7f4d84079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:37:38.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.758+0000 7f4d89498640 1 -- 172.21.15.67:0/828426486 shutdown_connections 2024-09-21T06:37:38.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.758+0000 7f4d89498640 1 -- 172.21.15.67:0/828426486 wait complete. 2024-09-21T06:37:38.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.760+0000 7f4d89498640 1 Processor -- start 2024-09-21T06:37:38.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.760+0000 7f4d89498640 1 -- start start 2024-09-21T06:37:38.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.761+0000 7f4d89498640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d84102a60 0x7f4d84199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:38.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.761+0000 7f4d89498640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d841083a0 0x7f4d841995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:38.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.761+0000 7f4d89498640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d84108cf0 0x7f4d8419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:38.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.761+0000 7f4d827fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d841083a0 0x7f4d841995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:38.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.761+0000 7f4d827fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d841083a0 0x7f4d841995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52338/0 (socket says 172.21.15.67:52338) 2024-09-21T06:37:38.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.761+0000 7f4d827fc640 1 -- 172.21.15.67:0/671896012 learned_addr learned my addr 172.21.15.67:0/671896012 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:37:38.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.761+0000 7f4d82ffd640 1 --2- 172.21.15.67:0/671896012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d84102a60 0x7f4d84199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:38.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.761+0000 7f4d89498640 1 -- 172.21.15.67:0/671896012 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4d84112860 con 0x7f4d841083a0 2024-09-21T06:37:38.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.761+0000 7f4d89498640 1 -- 172.21.15.67:0/671896012 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4d841126e0 con 0x7f4d84108cf0 2024-09-21T06:37:38.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.761+0000 7f4d89498640 1 -- 172.21.15.67:0/671896012 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4d841129e0 con 0x7f4d84102a60 2024-09-21T06:37:38.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.761+0000 7f4d837fe640 1 --2- 172.21.15.67:0/671896012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d84108cf0 0x7f4d8419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:38.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.762+0000 7f4d827fc640 1 -- 172.21.15.67:0/671896012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d84102a60 msgr2=0x7f4d84199050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:38.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.762+0000 7f4d827fc640 1 --2- 172.21.15.67:0/671896012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d84102a60 0x7f4d84199050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.762+0000 7f4d827fc640 1 -- 172.21.15.67:0/671896012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d84108cf0 msgr2=0x7f4d8419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:38.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.762+0000 7f4d827fc640 1 --2- 172.21.15.67:0/671896012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d84108cf0 0x7f4d8419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.762+0000 7f4d827fc640 1 -- 172.21.15.67:0/671896012 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d8419efc0 con 0x7f4d841083a0 2024-09-21T06:37:38.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.762+0000 7f4d837fe640 1 --2- 172.21.15.67:0/671896012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d84108cf0 0x7f4d8419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:37:38.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.762+0000 7f4d827fc640 1 --2- 172.21.15.67:0/671896012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d841083a0 0x7f4d841995f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800e9c0 tx=0x7f4d7800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:37:38.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.762+0000 7f4d6bfff640 1 -- 172.21.15.67:0/671896012 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7800cd80 con 0x7f4d841083a0 2024-09-21T06:37:38.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.763+0000 7f4d6bfff640 1 -- 172.21.15.67:0/671896012 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4d780054b0 con 0x7f4d841083a0 2024-09-21T06:37:38.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.763+0000 7f4d89498640 1 -- 172.21.15.67:0/671896012 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d8419f2b0 con 0x7f4d841083a0 2024-09-21T06:37:38.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.763+0000 7f4d6bfff640 1 -- 172.21.15.67:0/671896012 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d78010430 con 0x7f4d841083a0 2024-09-21T06:37:38.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.763+0000 7f4d89498640 1 -- 172.21.15.67:0/671896012 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d8406ae60 con 0x7f4d841083a0 2024-09-21T06:37:38.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.766+0000 7f4d8fa0c740 1 Processor -- start 2024-09-21T06:37:38.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.766+0000 7f4d8fa0c740 1 -- start start 2024-09-21T06:37:38.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.767+0000 7f4d8fa0c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56119f99d520 0x56119fd47920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:38.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.767+0000 7f4d8fa0c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56119f974660 0x56119fd47e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:38.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.767+0000 7f4d8fa0c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56119f97c840 0x56119fd483a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:38.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.767+0000 7f4d8fa0c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56119fc2edc0 con 0x56119f97c840 2024-09-21T06:37:38.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.767+0000 7f4d8fa0c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56119fcbb1f0 con 0x56119f99d520 2024-09-21T06:37:38.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.767+0000 7f4d8fa0c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56119fcaa620 con 0x56119f974660 2024-09-21T06:37:38.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.767+0000 7f4d837fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56119f99d520 0x56119fd47920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:38.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.767+0000 7f4d837fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56119f99d520 0x56119fd47920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.67:41508/0 (socket says 172.21.15.67:41508) 2024-09-21T06:37:38.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.767+0000 7f4d837fe640 1 -- 172.21.15.67:0/2781735143 learned_addr learned my addr 172.21.15.67:0/2781735143 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:37:38.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.767+0000 7f4d82ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56119f974660 0x56119fd47e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:38.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.768+0000 7f4d837fe640 1 -- 172.21.15.67:0/2781735143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56119f974660 msgr2=0x56119fd47e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:37:38.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.768+0000 7f4d837fe640 1 --2- 172.21.15.67:0/2781735143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56119f974660 0x56119fd47e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.768+0000 7f4d837fe640 1 -- 172.21.15.67:0/2781735143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56119f97c840 msgr2=0x56119fd483a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:37:38.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.768+0000 7f4d837fe640 1 --2- 172.21.15.67:0/2781735143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56119f97c840 0x56119fd483a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.768+0000 7f4d837fe640 1 -- 172.21.15.67:0/2781735143 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56119fc72fd0 con 0x56119f99d520 2024-09-21T06:37:38.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.769+0000 7f4d837fe640 1 --2- 172.21.15.67:0/2781735143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56119f99d520 0x56119fd47920 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0094c0 tx=0x7f4d7c001140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1761ce324ec9cbbc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:37:38.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.769+0000 7f4d6bfff640 1 -- 172.21.15.67:0/671896012 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f4d78004f20 con 0x7f4d841083a0 2024-09-21T06:37:38.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.770+0000 7f4d697fa640 1 -- 172.21.15.67:0/2781735143 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c031650 con 0x56119f99d520 2024-09-21T06:37:38.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.770+0000 7f4d697fa640 1 -- 172.21.15.67:0/2781735143 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4d7c036070 con 0x56119f99d520 2024-09-21T06:37:38.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.770+0000 7f4d697fa640 1 -- 172.21.15.67:0/2781735143 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c004040 con 0x56119f99d520 2024-09-21T06:37:38.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.771+0000 7f4d6bfff640 1 --2- 172.21.15.67:0/671896012 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f4d6c0b6590 0x7f4d6c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:38.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.771+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/2781735143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56119f99d520 msgr2=0x56119fd47920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:38.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.771+0000 7f4d8fa0c740 1 --2- 172.21.15.67:0/2781735143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56119f99d520 0x56119fd47920 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0094c0 tx=0x7f4d7c001140 comp rx=0 tx=0).stop 2024-09-21T06:37:38.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.771+0000 7f4d82ffd640 1 --2- 172.21.15.67:0/671896012 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f4d6c0b6590 0x7f4d6c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:38.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.771+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/2781735143 shutdown_connections 2024-09-21T06:37:38.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.771+0000 7f4d8fa0c740 1 --2- 172.21.15.67:0/2781735143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56119f97c840 0x56119fd483a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.771+0000 7f4d8fa0c740 1 --2- 172.21.15.67:0/2781735143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56119f974660 0x56119fd47e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.771+0000 7f4d8fa0c740 1 --2- 172.21.15.67:0/2781735143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56119f99d520 0x56119fd47920 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.771+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/2781735143 >> 172.21.15.67:0/2781735143 conn(0x56119f991910 msgr2=0x56119fd45810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:37:38.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.772+0000 7f4d6bfff640 1 -- 172.21.15.67:0/671896012 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f4d780e7d00 con 0x7f4d841083a0 2024-09-21T06:37:38.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.772+0000 7f4d82ffd640 1 --2- 172.21.15.67:0/671896012 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f4d6c0b6590 0x7f4d6c0b8aa0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x56119fd465a0 tx=0x7f4d74005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:37:38.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.772+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/2781735143 shutdown_connections 2024-09-21T06:37:38.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.772+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/2781735143 wait complete. 2024-09-21T06:37:38.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.773+0000 7f4d8fa0c740 1 -- start start 2024-09-21T06:37:38.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.773+0000 7f4d8fa0c740 1 Processor -- start 2024-09-21T06:37:38.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.773+0000 7f4d8fa0c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56119f974660 0x56119fd4f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:38.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.773+0000 7f4d8fa0c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56119f97c840 0x56119fd4f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:38.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.773+0000 7f4d8fa0c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56119f99d520 0x56119fd59a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:38.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.773+0000 7f4d8fa0c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56119fc45300 con 0x56119f99d520 2024-09-21T06:37:38.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.773+0000 7f4d8fa0c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56119fcae430 con 0x56119f97c840 2024-09-21T06:37:38.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.773+0000 7f4d8fa0c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56119fb0bbb0 con 0x56119f974660 2024-09-21T06:37:38.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.773+0000 7f4d8fa0c740 10 client.0 _ll_register_callbacks cb 0x56119fc5f120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:37:38.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.774+0000 7f4d82ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56119f974660 0x56119fd4f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:38.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.774+0000 7f4d82ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56119f974660 0x56119fd4f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.67:56704/0 (socket says 172.21.15.67:56704) 2024-09-21T06:37:38.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.774+0000 7f4d82ffd640 1 -- 172.21.15.67:0/3170869535 learned_addr learned my addr 172.21.15.67:0/3170869535 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:37:38.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.774+0000 7f4d82ffd640 1 -- 172.21.15.67:0/3170869535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56119f97c840 msgr2=0x56119fd4f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:37:38.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.774+0000 7f4d82ffd640 1 --2- 172.21.15.67:0/3170869535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56119f97c840 0x56119fd4f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.774+0000 7f4d82ffd640 1 -- 172.21.15.67:0/3170869535 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56119f99d520 msgr2=0x56119fd59a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:37:38.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.774+0000 7f4d82ffd640 1 --2- 172.21.15.67:0/3170869535 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56119f99d520 0x56119fd59a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.774+0000 7f4d82ffd640 1 -- 172.21.15.67:0/3170869535 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56119fc34780 con 0x56119f974660 2024-09-21T06:37:38.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.775+0000 7f4d82ffd640 1 --2- 172.21.15.67:0/3170869535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56119f974660 0x56119fd4f290 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4d740045d0 tx=0x7f4d74004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb9cb0cc09c65d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:37:38.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.775+0000 7f4d66ff5640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T06:37:38.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.775+0000 7f4d66ff5640 1 -- 172.21.15.67:0/3170869535 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7402d060 con 0x56119f974660 2024-09-21T06:37:38.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.775+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/3170869535 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d7c036070 con 0x56119f974660 2024-09-21T06:37:38.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.775+0000 7f4d66ff5640 1 -- 172.21.15.67:0/3170869535 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f4d74018030 con 0x56119f974660 2024-09-21T06:37:38.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.775+0000 7f4d66ff5640 1 -- 172.21.15.67:0/3170869535 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4d74034050 con 0x56119f974660 2024-09-21T06:37:38.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.775+0000 7f4d8fa0c740 10 client.25180 fetch_fsmap learned FSMap version 21 2024-09-21T06:37:38.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.775+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/3170869535 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56119fa08b00 con 0x56119f974660 2024-09-21T06:37:38.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.776+0000 7f4d66ff5640 1 -- 172.21.15.67:0/3170869535 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4d7c036070 con 0x56119f974660 2024-09-21T06:37:38.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.777+0000 7f4d66ff5640 1 -- 172.21.15.67:0/3170869535 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f4d74028710 con 0x56119f974660 2024-09-21T06:37:38.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.777+0000 7f4d8fa0c740 10 client.25180 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:37:38.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.777+0000 7f4d8fa0c740 10 client.25180 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:37:38.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.777+0000 7f4d8fa0c740 20 client.25180 populate_metadata read hostname 'smithi067' 2024-09-21T06:37:38.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.777+0000 7f4d8fa0c740 1 --2- 172.21.15.67:0/3170869535 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x56119f95bea0 0x56119fd72380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:38.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.777+0000 7f4d8fa0c740 4 client.25180 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:37:38.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.777+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/3170869535 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56119fcbd140 con 0x56119f95bea0 2024-09-21T06:37:38.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.777+0000 7f4d827fc640 1 --2- 172.21.15.67:0/3170869535 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x56119f95bea0 0x56119fd72380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:38.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.778+0000 7f4d66ff5640 10 client.25180 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:37:38.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.778+0000 7f4d827fc640 1 --2- 172.21.15.67:0/3170869535 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x56119f95bea0 0x56119fd72380 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7842e3727dbb1a33 server_cookie=aeab0c1c04a6948 in_seq=0 out_seq=0 2024-09-21T06:37:38.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.779+0000 7f4d66ff5640 1 -- 172.21.15.67:0/3170869535 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56119fcbd140 con 0x56119f95bea0 2024-09-21T06:37:38.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.779+0000 7f4d66ff5640 10 client.25180 handle_command_reply: tid=0 2024-09-21T06:37:38.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.779+0000 7f4d8fa0c740 1 client.25180 shutdown 2024-09-21T06:37:38.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.779+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/3170869535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56119f974660 msgr2=0x56119fd4f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:38.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.779+0000 7f4d8fa0c740 1 --2- 172.21.15.67:0/3170869535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56119f974660 0x56119fd4f290 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4d740045d0 tx=0x7f4d74004730 comp rx=0 tx=0).stop 2024-09-21T06:37:38.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.779+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/3170869535 shutdown_connections 2024-09-21T06:37:38.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.779+0000 7f4d8fa0c740 1 --2- 172.21.15.67:0/3170869535 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x56119f95bea0 0x56119fd72380 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.779+0000 7f4d8fa0c740 1 --2- 172.21.15.67:0/3170869535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56119f97c840 0x56119fd4f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.779+0000 7f4d8fa0c740 1 --2- 172.21.15.67:0/3170869535 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56119f99d520 0x56119fd59a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.779+0000 7f4d8fa0c740 1 --2- 172.21.15.67:0/3170869535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56119f974660 0x56119fd4f290 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.779+0000 7f4d66ff5640 0 client.25180 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:37:38.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.779+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/3170869535 >> 172.21.15.67:0/3170869535 conn(0x56119f991910 msgr2=0x56119fd4aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:37:38.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.780+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/3170869535 shutdown_connections 2024-09-21T06:37:38.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.780+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/3170869535 wait complete. 2024-09-21T06:37:38.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.780+0000 7f4d8fa0c740 20 client.25180 trim_cache size 0 max 16384 2024-09-21T06:37:38.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.813+0000 7f4d8fa0c740 1 Processor -- start 2024-09-21T06:37:38.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.813+0000 7f4d8fa0c740 1 -- start start 2024-09-21T06:37:38.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.814+0000 7f4d8fa0c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56119f974660 0x56119fd416d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:38.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.814+0000 7f4d8fa0c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56119f97c840 0x56119fd43110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:38.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.814+0000 7f4d827fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56119f97c840 0x56119fd43110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:38.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.814+0000 7f4d827fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56119f97c840 0x56119fd43110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52346/0 (socket says 172.21.15.67:52346) 2024-09-21T06:37:38.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.815+0000 7f4d82ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56119f974660 0x56119fd416d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:38.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.815+0000 7f4d827fc640 1 -- 172.21.15.67:0/115138793 learned_addr learned my addr 172.21.15.67:0/115138793 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:37:38.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.815+0000 7f4d8fa0c740 1 --2- 172.21.15.67:0/115138793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56119f99d520 0x56119fd43650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:38.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.815+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/115138793 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4d841ac820 con 0x56119f97c840 2024-09-21T06:37:38.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.815+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/115138793 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4d841aeff0 con 0x56119f974660 2024-09-21T06:37:38.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.815+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/115138793 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4d841acde0 con 0x56119f99d520 2024-09-21T06:37:38.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.815+0000 7f4d837fe640 1 --2- 172.21.15.67:0/115138793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56119f99d520 0x56119fd43650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:38.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.815+0000 7f4d82ffd640 1 -- 172.21.15.67:0/115138793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56119f99d520 msgr2=0x56119fd43650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:37:38.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.815+0000 7f4d82ffd640 1 --2- 172.21.15.67:0/115138793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56119f99d520 0x56119fd43650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.815+0000 7f4d82ffd640 1 -- 172.21.15.67:0/115138793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56119f97c840 msgr2=0x56119fd43110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:37:38.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.816+0000 7f4d82ffd640 1 --2- 172.21.15.67:0/115138793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56119f97c840 0x56119fd43110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.816+0000 7f4d82ffd640 1 -- 172.21.15.67:0/115138793 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56119fa7f960 con 0x56119f974660 2024-09-21T06:37:38.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.816+0000 7f4d837fe640 1 --2- 172.21.15.67:0/115138793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56119f99d520 0x56119fd43650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:37:38.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.816+0000 7f4d827fc640 1 --2- 172.21.15.67:0/115138793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56119f97c840 0x56119fd43110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:37:38.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.816+0000 7f4d82ffd640 1 --2- 172.21.15.67:0/115138793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56119f974660 0x56119fd416d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4d74019040 tx=0x7f4d74003ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99270703bde17bcc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:37:38.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.816+0000 7f4d68ff9640 1 -- 172.21.15.67:0/115138793 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d740283e0 con 0x56119f974660 2024-09-21T06:37:38.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.816+0000 7f4d68ff9640 1 -- 172.21.15.67:0/115138793 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4d74020710 con 0x56119f974660 2024-09-21T06:37:38.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.816+0000 7f4d68ff9640 1 -- 172.21.15.67:0/115138793 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7401d6f0 con 0x56119f974660 2024-09-21T06:37:38.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.817+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/115138793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56119f974660 msgr2=0x56119fd416d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:38.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.817+0000 7f4d8fa0c740 1 --2- 172.21.15.67:0/115138793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56119f974660 0x56119fd416d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4d74019040 tx=0x7f4d74003ed0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.817+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/115138793 shutdown_connections 2024-09-21T06:37:38.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.817+0000 7f4d8fa0c740 1 --2- 172.21.15.67:0/115138793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56119f99d520 0x56119fd43650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.817+0000 7f4d8fa0c740 1 --2- 172.21.15.67:0/115138793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56119f97c840 0x56119fd43110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.818+0000 7f4d8fa0c740 1 --2- 172.21.15.67:0/115138793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56119f974660 0x56119fd416d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.818+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/115138793 >> 172.21.15.67:0/115138793 conn(0x56119f991910 msgr2=0x56119fd3c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:37:38.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.818+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/115138793 shutdown_connections 2024-09-21T06:37:38.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.818+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/115138793 wait complete. 2024-09-21T06:37:38.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.818+0000 7f4d8fa0c740 1 -- start start 2024-09-21T06:37:38.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.818+0000 7f4d8fa0c740 1 Processor -- start 2024-09-21T06:37:38.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.819+0000 7f4d8fa0c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56119f974660 0x56119fdd6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:38.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.819+0000 7f4d8fa0c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56119f97c840 0x56119fdd65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:38.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.819+0000 7f4d8fa0c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56119f99d520 0x56119fd80250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:38.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.819+0000 7f4d827fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56119f97c840 0x56119fdd65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:38.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.819+0000 7f4d837fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56119f99d520 0x56119fd80250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:38.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.819+0000 7f4d827fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56119f97c840 0x56119fdd65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.67:41536/0 (socket says 172.21.15.67:41536) 2024-09-21T06:37:38.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.819+0000 7f4d82ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56119f974660 0x56119fdd6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:38.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.819+0000 7f4d82ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56119f974660 0x56119fdd6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.67:56722/0 (socket says 172.21.15.67:56722) 2024-09-21T06:37:38.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.819+0000 7f4d837fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56119f99d520 0x56119fd80250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52348/0 (socket says 172.21.15.67:52348) 2024-09-21T06:37:38.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.819+0000 7f4d827fc640 1 -- 172.21.15.67:0/2728810903 learned_addr learned my addr 172.21.15.67:0/2728810903 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:37:38.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.819+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/2728810903 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4d841ad4b0 con 0x56119f99d520 2024-09-21T06:37:38.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.819+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/2728810903 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4d841ae920 con 0x56119f97c840 2024-09-21T06:37:38.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.819+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/2728810903 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4d841adb80 con 0x56119f974660 2024-09-21T06:37:38.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.820+0000 7f4d827fc640 1 -- 172.21.15.67:0/2728810903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56119f974660 msgr2=0x56119fdd6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:37:38.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.820+0000 7f4d827fc640 1 --2- 172.21.15.67:0/2728810903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56119f974660 0x56119fdd6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.820+0000 7f4d827fc640 1 -- 172.21.15.67:0/2728810903 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56119f99d520 msgr2=0x56119fd80250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:37:38.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.820+0000 7f4d827fc640 1 --2- 172.21.15.67:0/2728810903 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56119f99d520 0x56119fd80250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.820+0000 7f4d827fc640 1 -- 172.21.15.67:0/2728810903 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56119fd809c0 con 0x56119f97c840 2024-09-21T06:37:38.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.820+0000 7f4d8fa0c740 10 client.0 _ll_register_callbacks cb 0x7f4d74009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:37:38.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.820+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/2728810903 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d74020710 con 0x56119f97c840 2024-09-21T06:37:38.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.820+0000 7f4d827fc640 1 --2- 172.21.15.67:0/2728810903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56119f97c840 0x56119fdd65c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4d780ad9f0 tx=0x7f4d780ad290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9046ed4dfff5379 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:37:38.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.820+0000 7f4d66ff5640 10 client.25217 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-21T06:37:38.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.821+0000 7f4d66ff5640 1 -- 172.21.15.67:0/2728810903 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d780b2540 con 0x56119f97c840 2024-09-21T06:37:38.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.821+0000 7f4d66ff5640 1 -- 172.21.15.67:0/2728810903 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f4d780e54e0 con 0x56119f97c840 2024-09-21T06:37:38.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.821+0000 7f4d66ff5640 1 -- 172.21.15.67:0/2728810903 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4d780b7480 con 0x56119f97c840 2024-09-21T06:37:38.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.821+0000 7f4d8fa0c740 10 client.25217 fetch_fsmap learned FSMap version 21 2024-09-21T06:37:38.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.821+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/2728810903 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({fsmap=21}) -- 0x56119fddd940 con 0x56119f97c840 2024-09-21T06:37:38.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.821+0000 7f4d66ff5640 1 -- 172.21.15.67:0/2728810903 <== mon.2 v2:172.21.15.100:3300/0 4 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f4d780a9040 con 0x56119f97c840 2024-09-21T06:37:38.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.821+0000 7f4d8fa0c740 10 client.25217 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:37:38.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.821+0000 7f4d8fa0c740 10 client.25217 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:37:38.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.821+0000 7f4d8fa0c740 20 client.25217 populate_metadata read hostname 'smithi067' 2024-09-21T06:37:38.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.822+0000 7f4d8fa0c740 1 --2- 172.21.15.67:0/2728810903 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x56119f931250 0x56119fde0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:37:38.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.822+0000 7f4d8fa0c740 4 client.25217 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6c8619e7", "release": true, "await": true}] 2024-09-21T06:37:38.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.822+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/2728810903 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6c8619e7", "release": true, "await": true}) -- 0x56119fcb8bf0 con 0x56119f931250 2024-09-21T06:37:38.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.822+0000 7f4d82ffd640 1 --2- 172.21.15.67:0/2728810903 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x56119f931250 0x56119fde0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:37:38.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.822+0000 7f4d66ff5640 10 client.25217 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:37:38.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.822+0000 7f4d82ffd640 1 --2- 172.21.15.67:0/2728810903 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x56119f931250 0x56119fde0180 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=402f9dfdeb9dac36 server_cookie=529fdb47af6c5484 in_seq=0 out_seq=0 2024-09-21T06:37:38.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.824+0000 7f4d66ff5640 1 -- 172.21.15.67:0/2728810903 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x56119fcb8bf0 con 0x56119f931250 2024-09-21T06:37:38.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.824+0000 7f4d66ff5640 10 client.25217 handle_command_reply: tid=0 2024-09-21T06:37:38.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.824+0000 7f4d8fa0c740 1 client.25217 shutdown 2024-09-21T06:37:38.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.824+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/2728810903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56119f97c840 msgr2=0x56119fdd65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:38.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.824+0000 7f4d8fa0c740 1 --2- 172.21.15.67:0/2728810903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56119f97c840 0x56119fdd65c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4d780ad9f0 tx=0x7f4d780ad290 comp rx=0 tx=0).stop 2024-09-21T06:37:38.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.824+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/2728810903 shutdown_connections 2024-09-21T06:37:38.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.824+0000 7f4d8fa0c740 1 --2- 172.21.15.67:0/2728810903 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x56119f931250 0x56119fde0180 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.824+0000 7f4d8fa0c740 1 --2- 172.21.15.67:0/2728810903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56119f97c840 0x56119fdd65c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.824+0000 7f4d8fa0c740 1 --2- 172.21.15.67:0/2728810903 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56119f99d520 0x56119fd80250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.824+0000 7f4d8fa0c740 1 --2- 172.21.15.67:0/2728810903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56119f974660 0x56119fdd6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.824+0000 7f4d66ff5640 0 client.25217 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:37:38.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.824+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/2728810903 >> 172.21.15.67:0/2728810903 conn(0x56119f991910 msgr2=0x56119fd48950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:37:38.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.825+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/2728810903 shutdown_connections 2024-09-21T06:37:38.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.825+0000 7f4d8fa0c740 1 -- 172.21.15.67:0/2728810903 wait complete. 2024-09-21T06:37:38.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.825+0000 7f4d8fa0c740 20 client.25217 trim_cache size 0 max 16384 2024-09-21T06:37:38.828 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-21T06:37:38.828 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 21, 2024-09-21T06:37:38.828 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-09-21T06:37:38.829 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 4, 2024-09-21T06:37:38.829 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-21T06:37:38.829 INFO:teuthology.orchestra.run.smithi067.stdout: "6c8619e7": { 2024-09-21T06:37:38.829 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 4, 2024-09-21T06:37:38.829 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-21T06:37:38.829 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:37:38.829 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-21T06:37:38.829 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:37:38.829 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:37:38.829 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-21T06:37:38.829 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 134.5, 2024-09-21T06:37:38.829 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-21T06:37:38.829 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-21T06:37:38.829 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-21T06:37:38.830 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:37:38.830 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-21T06:37:38.830 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:37:38.830 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:37:38.830 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:37:38.830 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:37:38.830 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:37:38.830 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:37:38.830 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-21T06:37:38.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.826+0000 7f4d89498640 1 -- 172.21.15.67:0/671896012 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f4d6c0b6590 msgr2=0x7f4d6c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:38.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.826+0000 7f4d89498640 1 --2- 172.21.15.67:0/671896012 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f4d6c0b6590 0x7f4d6c0b8aa0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x56119fd465a0 tx=0x7f4d74005f90 comp rx=0 tx=0).stop 2024-09-21T06:37:38.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.826+0000 7f4d89498640 1 -- 172.21.15.67:0/671896012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d841083a0 msgr2=0x7f4d841995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:37:38.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.826+0000 7f4d89498640 1 --2- 172.21.15.67:0/671896012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d841083a0 0x7f4d841995f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800e9c0 tx=0x7f4d7800ee80 comp rx=0 tx=0).stop 2024-09-21T06:37:38.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.826+0000 7f4d89498640 1 -- 172.21.15.67:0/671896012 shutdown_connections 2024-09-21T06:37:38.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.826+0000 7f4d89498640 1 --2- 172.21.15.67:0/671896012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d84108cf0 0x7f4d8419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.826+0000 7f4d89498640 1 --2- 172.21.15.67:0/671896012 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f4d6c0b6590 0x7f4d6c0b8aa0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.826+0000 7f4d89498640 1 --2- 172.21.15.67:0/671896012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d841083a0 0x7f4d841995f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.826+0000 7f4d89498640 1 --2- 172.21.15.67:0/671896012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d84102a60 0x7f4d84199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:37:38.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.826+0000 7f4d89498640 1 -- 172.21.15.67:0/671896012 >> 172.21.15.67:0/671896012 conn(0x7f4d84079730 msgr2=0x7f4d84104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:37:38.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.826+0000 7f4d89498640 1 -- 172.21.15.67:0/671896012 shutdown_connections 2024-09-21T06:37:38.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:37:38.826+0000 7f4d89498640 1 -- 172.21.15.67:0/671896012 wait complete. 2024-09-21T06:37:39.675 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6c8619e7', total seconds elapsed: 18.3 2024-09-21T06:37:39.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:39 smithi203 ceph-mon[33735]: pgmap v392: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-21T06:37:40.040 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:39 smithi067 ceph-mon[31232]: pgmap v392: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-21T06:37:40.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:39 smithi100 ceph-mon[33401]: pgmap v392: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-21T06:37:41.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:41 smithi203 ceph-mon[33735]: pgmap v393: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:37:42.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:41 smithi067 ceph-mon[31232]: pgmap v393: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:37:42.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:41 smithi100 ceph-mon[33401]: pgmap v393: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:37:43.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:43 smithi203 ceph-mon[33735]: pgmap v394: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:37:44.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:43 smithi067 ceph-mon[31232]: pgmap v394: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:37:44.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:43 smithi100 ceph-mon[33401]: pgmap v394: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:37:45.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:45 smithi203 ceph-mon[33735]: pgmap v395: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:37:46.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:45 smithi067 ceph-mon[31232]: pgmap v395: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:37:46.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:45 smithi100 ceph-mon[33401]: pgmap v395: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:37:47.855 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:47 smithi100 ceph-mon[33401]: pgmap v396: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:37:47.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:47 smithi203 ceph-mon[33735]: pgmap v396: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:37:48.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:47 smithi067 ceph-mon[31232]: pgmap v396: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:37:49.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:49 smithi067 ceph-mon[31232]: pgmap v397: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:37:49.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:49 smithi203 ceph-mon[33735]: pgmap v397: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:37:50.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:49 smithi100 ceph-mon[33401]: pgmap v397: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:37:51.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:51 smithi203 ceph-mon[33735]: pgmap v398: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:37:52.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:51 smithi067 ceph-mon[31232]: pgmap v398: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:37:52.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:51 smithi100 ceph-mon[33401]: pgmap v398: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:37:53.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:53 smithi203 ceph-mon[33735]: pgmap v399: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:37:54.041 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:53 smithi067 ceph-mon[31232]: pgmap v399: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:37:54.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:53 smithi100 ceph-mon[33401]: pgmap v399: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:37:55.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:55 smithi203 ceph-mon[33735]: pgmap v400: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:37:56.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:55 smithi067 ceph-mon[31232]: pgmap v400: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:37:56.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:55 smithi100 ceph-mon[33401]: pgmap v400: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:37:57.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:57 smithi203 ceph-mon[33735]: pgmap v401: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:37:58.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:57 smithi067 ceph-mon[31232]: pgmap v401: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:37:58.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:57 smithi100 ceph-mon[33401]: pgmap v401: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:37:59.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:37:59 smithi203 ceph-mon[33735]: pgmap v402: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:38:00.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:37:59 smithi067 ceph-mon[31232]: pgmap v402: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:38:00.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:37:59 smithi100 ceph-mon[33401]: pgmap v402: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:38:01.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:01 smithi203 ceph-mon[33735]: pgmap v403: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:38:02.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:01 smithi067 ceph-mon[31232]: pgmap v403: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:38:02.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:01 smithi100 ceph-mon[33401]: pgmap v403: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:38:03.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:03 smithi203 ceph-mon[33735]: pgmap v404: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:38:04.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:03 smithi067 ceph-mon[31232]: pgmap v404: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:38:04.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:03 smithi100 ceph-mon[33401]: pgmap v404: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:38:05.316 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 93% (11876/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-21T06:38:05.907 INFO:tasks.workunit.client.0.smithi067.stderr:Note: switching to '1843221895adccfb3b692060251dd6a0183bc410'. 2024-09-21T06:38:05.907 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-09-21T06:38:05.907 INFO:tasks.workunit.client.0.smithi067.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-21T06:38:05.907 INFO:tasks.workunit.client.0.smithi067.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-21T06:38:05.908 INFO:tasks.workunit.client.0.smithi067.stderr:state without impacting any branches by switching back to a branch. 2024-09-21T06:38:05.908 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-09-21T06:38:05.908 INFO:tasks.workunit.client.0.smithi067.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-21T06:38:05.908 INFO:tasks.workunit.client.0.smithi067.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-21T06:38:05.908 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-09-21T06:38:05.908 INFO:tasks.workunit.client.0.smithi067.stderr: git switch -c 2024-09-21T06:38:05.908 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-09-21T06:38:05.908 INFO:tasks.workunit.client.0.smithi067.stderr:Or undo this operation with: 2024-09-21T06:38:05.908 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-09-21T06:38:05.908 INFO:tasks.workunit.client.0.smithi067.stderr: git switch - 2024-09-21T06:38:05.908 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-09-21T06:38:05.908 INFO:tasks.workunit.client.0.smithi067.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-21T06:38:05.908 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-09-21T06:38:05.908 INFO:tasks.workunit.client.0.smithi067.stderr:HEAD is now at 1843221895a Merge PR #56776 into wip-pdonnell-testing-20240920.212106-debug 2024-09-21T06:38:05.912 DEBUG:teuthology.orchestra.run.smithi067:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-21T06:38:05.937 INFO:tasks.workunit.client.0.smithi067.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-21T06:38:05.940 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T06:38:05.940 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-21T06:38:05.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:05 smithi203 ceph-mon[33735]: pgmap v405: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:38:06.041 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:05 smithi067 ceph-mon[31232]: pgmap v405: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:38:06.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:05 smithi100 ceph-mon[33401]: pgmap v405: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:38:06.605 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.0 2024-09-21T06:38:06.605 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:38:06.661 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-21T06:38:06.739 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-21T06:38:06.780 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T06:38:06.782 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T06:38:06.782 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-21T06:38:06.824 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T06:38:06.828 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-21T06:38:06.828 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-21T06:38:06.846 INFO:tasks.workunit:Running workunits matching suites/iogen.sh on client.0... 2024-09-21T06:38:06.848 INFO:tasks.workunit:Running workunit suites/iogen.sh... 2024-09-21T06:38:06.848 DEBUG:teuthology.orchestra.run.smithi067:workunit test suites/iogen.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1843221895adccfb3b692060251dd6a0183bc410 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iogen.sh 2024-09-21T06:38:06.881 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:38:06.913 INFO:tasks.workunit.client.0.smithi067.stdout:getting iogen 2024-09-21T06:38:06.914 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'getting iogen' 2024-09-21T06:38:06.914 INFO:tasks.workunit.client.0.smithi067.stderr:+ wget http://download.ceph.com/qa/iogen_3.1p0.tar 2024-09-21T06:38:07.069 INFO:tasks.workunit.client.0.smithi067.stderr:--2024-09-21 06:38:07-- http://download.ceph.com/qa/iogen_3.1p0.tar 2024-09-21T06:38:07.079 INFO:tasks.workunit.client.0.smithi067.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-09-21T06:38:07.100 INFO:tasks.workunit.client.0.smithi067.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-09-21T06:38:07.120 INFO:tasks.workunit.client.0.smithi067.stderr:HTTP request sent, awaiting response... 200 OK 2024-09-21T06:38:07.121 INFO:tasks.workunit.client.0.smithi067.stderr:Length: 8071 (7.9K) [application/octet-stream] 2024-09-21T06:38:07.122 INFO:tasks.workunit.client.0.smithi067.stderr:Saving to: ‘iogen_3.1p0.tar’ 2024-09-21T06:38:07.127 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-09-21T06:38:07.127 INFO:tasks.workunit.client.0.smithi067.stderr: 0K ....... 100% 1.28G=0s 2024-09-21T06:38:07.127 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-09-21T06:38:07.128 INFO:tasks.workunit.client.0.smithi067.stderr:2024-09-21 06:38:07 (1.28 GB/s) - ‘iogen_3.1p0.tar’ saved [8071/8071] 2024-09-21T06:38:07.128 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-09-21T06:38:07.128 INFO:tasks.workunit.client.0.smithi067.stderr:+ tar -xvzf iogen_3.1p0.tar 2024-09-21T06:38:07.169 INFO:tasks.workunit.client.0.smithi067.stdout:iogen_3.1p0 2024-09-21T06:38:07.171 INFO:tasks.workunit.client.0.smithi067.stdout:iogen_3.1p0/src 2024-09-21T06:38:07.172 INFO:tasks.workunit.client.0.smithi067.stdout:iogen_3.1p0/src/iogen.8 2024-09-21T06:38:07.175 INFO:tasks.workunit.client.0.smithi067.stdout:iogen_3.1p0/src/iogen.c 2024-09-21T06:38:07.178 INFO:tasks.workunit.client.0.smithi067.stdout:iogen_3.1p0/README 2024-09-21T06:38:07.180 INFO:tasks.workunit.client.0.smithi067.stdout:iogen_3.1p0/Makefile.bsd 2024-09-21T06:38:07.182 INFO:tasks.workunit.client.0.smithi067.stdout:iogen_3.1p0/Makefile 2024-09-21T06:38:07.185 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd iogen_3.1p0 2024-09-21T06:38:07.187 INFO:tasks.workunit.client.0.smithi067.stdout:making iogen 2024-09-21T06:38:07.187 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'making iogen' 2024-09-21T06:38:07.187 INFO:tasks.workunit.client.0.smithi067.stderr:+ make 2024-09-21T06:38:07.198 INFO:tasks.workunit.client.0.smithi067.stdout:gcc -D_FILE_OFFSET_BITS=64 -DVERSION=\"3.1p0\" -Dstrlcpy=strncpy -Darc4random=rand -O2 src/iogen.c -o iogen 2024-09-21T06:38:07.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.491+0000 7fe930561640 1 -- 172.21.15.67:0/4183928241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe928102ad0 msgr2=0x7fe928102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:07.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.491+0000 7fe930561640 1 --2- 172.21.15.67:0/4183928241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe928102ad0 0x7fe928102ee0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe924009900 tx=0x7fe92402a410 comp rx=0 tx=0).stop 2024-09-21T06:38:07.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.491+0000 7fe930561640 1 -- 172.21.15.67:0/4183928241 shutdown_connections 2024-09-21T06:38:07.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.491+0000 7fe930561640 1 --2- 172.21.15.67:0/4183928241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9281083c0 0x7fe9281087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:07.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.491+0000 7fe930561640 1 --2- 172.21.15.67:0/4183928241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe928108d10 0x7fe92810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:07.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.491+0000 7fe930561640 1 --2- 172.21.15.67:0/4183928241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe928102ad0 0x7fe928102ee0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:07.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.491+0000 7fe930561640 1 -- 172.21.15.67:0/4183928241 >> 172.21.15.67:0/4183928241 conn(0x7fe9280796d0 msgr2=0x7fe928079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:38:07.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.491+0000 7fe930561640 1 -- 172.21.15.67:0/4183928241 shutdown_connections 2024-09-21T06:38:07.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.491+0000 7fe930561640 1 -- 172.21.15.67:0/4183928241 wait complete. 2024-09-21T06:38:07.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.492+0000 7fe930561640 1 Processor -- start 2024-09-21T06:38:07.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.492+0000 7fe930561640 1 -- start start 2024-09-21T06:38:07.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.492+0000 7fe930561640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe928102ad0 0x7fe92819a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:07.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.492+0000 7fe930561640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9281083c0 0x7fe92819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:07.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.492+0000 7fe930561640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe928108d10 0x7fe92819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:07.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.492+0000 7fe930561640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe9281139a0 con 0x7fe928102ad0 2024-09-21T06:38:07.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.492+0000 7fe930561640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe928113820 con 0x7fe928108d10 2024-09-21T06:38:07.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.492+0000 7fe930561640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe928113b20 con 0x7fe9281083c0 2024-09-21T06:38:07.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.492+0000 7fe92e2fd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe928102ad0 0x7fe92819a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:07.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.492+0000 7fe92e2fd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe928102ad0 0x7fe92819a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55080/0 (socket says 172.21.15.67:55080) 2024-09-21T06:38:07.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.492+0000 7fe92e2fd640 1 -- 172.21.15.67:0/2823699761 learned_addr learned my addr 172.21.15.67:0/2823699761 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:38:07.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.492+0000 7fe92eafe640 1 --2- 172.21.15.67:0/2823699761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe928108d10 0x7fe92819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:07.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.492+0000 7fe92dafc640 1 --2- 172.21.15.67:0/2823699761 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9281083c0 0x7fe92819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:07.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.493+0000 7fe92e2fd640 1 -- 172.21.15.67:0/2823699761 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9281083c0 msgr2=0x7fe92819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:07.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.493+0000 7fe92e2fd640 1 --2- 172.21.15.67:0/2823699761 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9281083c0 0x7fe92819a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:07.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.493+0000 7fe92e2fd640 1 -- 172.21.15.67:0/2823699761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe928108d10 msgr2=0x7fe92819fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:07.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.493+0000 7fe92e2fd640 1 --2- 172.21.15.67:0/2823699761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe928108d10 0x7fe92819fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:07.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.493+0000 7fe92e2fd640 1 -- 172.21.15.67:0/2823699761 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9281a02c0 con 0x7fe928102ad0 2024-09-21T06:38:07.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.493+0000 7fe92e2fd640 1 --2- 172.21.15.67:0/2823699761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe928102ad0 0x7fe92819a410 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe924009230 tx=0x7fe924004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:38:07.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.493+0000 7fe91f7fe640 1 -- 172.21.15.67:0/2823699761 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92403f030 con 0x7fe928102ad0 2024-09-21T06:38:07.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.493+0000 7fe930561640 1 -- 172.21.15.67:0/2823699761 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9281a0550 con 0x7fe928102ad0 2024-09-21T06:38:07.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.493+0000 7fe91f7fe640 1 -- 172.21.15.67:0/2823699761 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe924003a80 con 0x7fe928102ad0 2024-09-21T06:38:07.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.493+0000 7fe91f7fe640 1 -- 172.21.15.67:0/2823699761 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924035070 con 0x7fe928102ad0 2024-09-21T06:38:07.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.493+0000 7fe930561640 1 -- 172.21.15.67:0/2823699761 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9281a0a30 con 0x7fe928102ad0 2024-09-21T06:38:07.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.494+0000 7fe930561640 1 -- 172.21.15.67:0/2823699761 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe928103d00 con 0x7fe928102ad0 2024-09-21T06:38:07.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.496+0000 7fe91f7fe640 1 -- 172.21.15.67:0/2823699761 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fe924042030 con 0x7fe928102ad0 2024-09-21T06:38:07.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.497+0000 7fe91f7fe640 1 --2- 172.21.15.67:0/2823699761 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe9140b6540 0x7fe9140b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:07.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.497+0000 7fe92dafc640 1 --2- 172.21.15.67:0/2823699761 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe9140b6540 0x7fe9140b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:07.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.497+0000 7fe91f7fe640 1 -- 172.21.15.67:0/2823699761 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fe924108130 con 0x7fe928102ad0 2024-09-21T06:38:07.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.497+0000 7fe92dafc640 1 --2- 172.21.15.67:0/2823699761 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe9140b6540 0x7fe9140b8a50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe92807a090 tx=0x7fe91800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:38:07.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.504+0000 7fe91f7fe640 1 -- 172.21.15.67:0/2823699761 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe9240d29d0 con 0x7fe928102ad0 2024-09-21T06:38:07.688 INFO:tasks.workunit.client.0.smithi067.stdout:nroff -mandoc -Tascii src/iogen.8 > iogen.cat8 2024-09-21T06:38:07.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.750+0000 7fe930561640 1 -- 172.21.15.67:0/2823699761 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe92819b540 con 0x7fe928102ad0 2024-09-21T06:38:07.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.751+0000 7fe91f7fe640 1 -- 172.21.15.67:0/2823699761 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7fe92819b540 con 0x7fe928102ad0 2024-09-21T06:38:07.753 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:38:07.753 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:38:07.753 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:38:07.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.753+0000 7fe930561640 1 -- 172.21.15.67:0/2823699761 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe9140b6540 msgr2=0x7fe9140b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:07.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.753+0000 7fe930561640 1 --2- 172.21.15.67:0/2823699761 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe9140b6540 0x7fe9140b8a50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe92807a090 tx=0x7fe91800b390 comp rx=0 tx=0).stop 2024-09-21T06:38:07.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.753+0000 7fe930561640 1 -- 172.21.15.67:0/2823699761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe928102ad0 msgr2=0x7fe92819a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:07.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.754+0000 7fe930561640 1 --2- 172.21.15.67:0/2823699761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe928102ad0 0x7fe92819a410 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe924009230 tx=0x7fe924004270 comp rx=0 tx=0).stop 2024-09-21T06:38:07.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.754+0000 7fe930561640 1 -- 172.21.15.67:0/2823699761 shutdown_connections 2024-09-21T06:38:07.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.754+0000 7fe930561640 1 --2- 172.21.15.67:0/2823699761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe928108d10 0x7fe92819fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:07.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.754+0000 7fe930561640 1 --2- 172.21.15.67:0/2823699761 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe9140b6540 0x7fe9140b8a50 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:07.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.754+0000 7fe930561640 1 --2- 172.21.15.67:0/2823699761 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9281083c0 0x7fe92819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:07.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.754+0000 7fe930561640 1 --2- 172.21.15.67:0/2823699761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe928102ad0 0x7fe92819a410 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:07.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.754+0000 7fe930561640 1 -- 172.21.15.67:0/2823699761 >> 172.21.15.67:0/2823699761 conn(0x7fe9280796d0 msgr2=0x7fe928106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:38:07.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.754+0000 7fe930561640 1 -- 172.21.15.67:0/2823699761 shutdown_connections 2024-09-21T06:38:07.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:07.754+0000 7fe930561640 1 -- 172.21.15.67:0/2823699761 wait complete. 2024-09-21T06:38:07.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:07 smithi067 ceph-mon[31232]: pgmap v406: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:38:07.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:07 smithi203 ceph-mon[33735]: pgmap v406: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:38:08.020 INFO:tasks.workunit.client.0.smithi067.stdout:running iogen 2024-09-21T06:38:08.021 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'running iogen' 2024-09-21T06:38:08.021 INFO:tasks.workunit.client.0.smithi067.stderr:+ ./iogen -n 5 -s 2g 2024-09-21T06:38:08.022 INFO:tasks.workunit.client.0.smithi067.stderr:child spawned 86609 2024-09-21T06:38:08.022 INFO:tasks.workunit.client.0.smithi067.stderr:child spawned 86610 2024-09-21T06:38:08.022 INFO:tasks.workunit.client.0.smithi067.stderr:child spawned 86611 2024-09-21T06:38:08.022 INFO:tasks.workunit.client.0.smithi067.stderr:child spawned 86612 2024-09-21T06:38:08.022 INFO:tasks.workunit.client.0.smithi067.stderr:child spawned 86614 2024-09-21T06:38:08.023 INFO:tasks.workunit.client.0.smithi067.stdout:sleep for 10 min 2024-09-21T06:38:08.023 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'sleep for 10 min' 2024-09-21T06:38:08.023 INFO:tasks.workunit.client.0.smithi067.stderr:+ sleep 600 2024-09-21T06:38:08.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:07 smithi100 ceph-mon[33401]: pgmap v406: 129 pgs: 129 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:38:08.719 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db / --timeout 90 --expiration 137.0 --await-for 45.0' 2024-09-21T06:38:08.720 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24671 quiesce db / --timeout 90 --expiration 137.0 --await-for 45.0 2024-09-21T06:38:08.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:08 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2823699761' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:38:09.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:08 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2823699761' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:38:09.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:08 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2823699761' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:38:09.141 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:38:09.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.777+0000 7f8e71150640 1 -- 172.21.15.67:0/3423987913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e6c108d40 msgr2=0x7f8e6c10f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:09.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.777+0000 7f8e71150640 1 --2- 172.21.15.67:0/3423987913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e6c108d40 0x7f8e6c10f570 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009be0 tx=0x7f8e6402a6c0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.777+0000 7f8e71150640 1 -- 172.21.15.67:0/3423987913 shutdown_connections 2024-09-21T06:38:09.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.777+0000 7f8e71150640 1 --2- 172.21.15.67:0/3423987913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e6c108d40 0x7f8e6c10f570 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.777+0000 7f8e71150640 1 --2- 172.21.15.67:0/3423987913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e6c1083f0 0x7f8e6c108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.777+0000 7f8e71150640 1 --2- 172.21.15.67:0/3423987913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e6c102ab0 0x7f8e6c102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.777+0000 7f8e71150640 1 -- 172.21.15.67:0/3423987913 >> 172.21.15.67:0/3423987913 conn(0x7f8e6c079870 msgr2=0x7f8e6c079c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:38:09.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.778+0000 7f8e71150640 1 -- 172.21.15.67:0/3423987913 shutdown_connections 2024-09-21T06:38:09.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.778+0000 7f8e71150640 1 -- 172.21.15.67:0/3423987913 wait complete. 2024-09-21T06:38:09.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.779+0000 7f8e71150640 1 Processor -- start 2024-09-21T06:38:09.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.779+0000 7f8e71150640 1 -- start start 2024-09-21T06:38:09.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.779+0000 7f8e71150640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e6c102ab0 0x7f8e6c19a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:09.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.779+0000 7f8e71150640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e6c1083f0 0x7f8e6c19a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:09.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.779+0000 7f8e71150640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e6c108d40 0x7f8e6c19fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:09.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.779+0000 7f8e71150640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c1128b0 con 0x7f8e6c108d40 2024-09-21T06:38:09.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.779+0000 7f8e71150640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c112730 con 0x7f8e6c1083f0 2024-09-21T06:38:09.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.779+0000 7f8e71150640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c112a30 con 0x7f8e6c102ab0 2024-09-21T06:38:09.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.780+0000 7f8e6b59e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e6c108d40 0x7f8e6c19fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:09.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.780+0000 7f8e6b59e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e6c108d40 0x7f8e6c19fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55110/0 (socket says 172.21.15.67:55110) 2024-09-21T06:38:09.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.780+0000 7f8e6b59e640 1 -- 172.21.15.67:0/3705673986 learned_addr learned my addr 172.21.15.67:0/3705673986 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:38:09.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.780+0000 7f8e6ad9d640 1 --2- 172.21.15.67:0/3705673986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e6c102ab0 0x7f8e6c19a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:09.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.780+0000 7f8e6a59c640 1 --2- 172.21.15.67:0/3705673986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e6c1083f0 0x7f8e6c19a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:09.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.780+0000 7f8e6b59e640 1 -- 172.21.15.67:0/3705673986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e6c102ab0 msgr2=0x7f8e6c19a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:09.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.780+0000 7f8e6b59e640 1 --2- 172.21.15.67:0/3705673986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e6c102ab0 0x7f8e6c19a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.780+0000 7f8e6b59e640 1 -- 172.21.15.67:0/3705673986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e6c1083f0 msgr2=0x7f8e6c19a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:09.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.780+0000 7f8e6b59e640 1 --2- 172.21.15.67:0/3705673986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e6c1083f0 0x7f8e6c19a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.780+0000 7f8e6b59e640 1 -- 172.21.15.67:0/3705673986 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e6c1a0300 con 0x7f8e6c108d40 2024-09-21T06:38:09.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.781+0000 7f8e6b59e640 1 --2- 172.21.15.67:0/3705673986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e6c108d40 0x7f8e6c19fbd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8e64004af0 tx=0x7f8e64030b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:38:09.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.781+0000 7f8e53fff640 1 -- 172.21.15.67:0/3705673986 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e64041020 con 0x7f8e6c108d40 2024-09-21T06:38:09.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.781+0000 7f8e53fff640 1 -- 172.21.15.67:0/3705673986 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e64037040 con 0x7f8e6c108d40 2024-09-21T06:38:09.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.781+0000 7f8e53fff640 1 -- 172.21.15.67:0/3705673986 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e64036070 con 0x7f8e6c108d40 2024-09-21T06:38:09.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.781+0000 7f8e71150640 1 -- 172.21.15.67:0/3705673986 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e6c1a0590 con 0x7f8e6c108d40 2024-09-21T06:38:09.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.781+0000 7f8e71150640 1 -- 172.21.15.67:0/3705673986 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e6c0ff510 con 0x7f8e6c108d40 2024-09-21T06:38:09.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.786+0000 7f8e53fff640 1 -- 172.21.15.67:0/3705673986 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f8e64044020 con 0x7f8e6c108d40 2024-09-21T06:38:09.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.786+0000 7f8e776c4740 1 Processor -- start 2024-09-21T06:38:09.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.786+0000 7f8e776c4740 1 -- start start 2024-09-21T06:38:09.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.786+0000 7f8e776c4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b56af8c840 0x55b56b333600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:09.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.786+0000 7f8e776c4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b56af63980 0x55b56b333b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:09.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.786+0000 7f8e776c4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b56af6bb60 0x55b56b334080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:09.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.786+0000 7f8e776c4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55b56b2a9b90 con 0x55b56af63980 2024-09-21T06:38:09.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.786+0000 7f8e776c4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55b56b298fc0 con 0x55b56af6bb60 2024-09-21T06:38:09.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.786+0000 7f8e776c4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b56b29cdd0 con 0x55b56af8c840 2024-09-21T06:38:09.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.787+0000 7f8e6a59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b56af8c840 0x55b56b333600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:09.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.787+0000 7f8e6a59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b56af8c840 0x55b56b333600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.67:58014/0 (socket says 172.21.15.67:58014) 2024-09-21T06:38:09.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.787+0000 7f8e6a59c640 1 -- 172.21.15.67:0/3363612321 learned_addr learned my addr 172.21.15.67:0/3363612321 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:38:09.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.787+0000 7f8e6a59c640 1 --2- 172.21.15.67:0/3363612321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b56af6bb60 0x55b56b334080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:09.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.787+0000 7f8e6ad9d640 1 --2- 172.21.15.67:0/3363612321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b56af63980 0x55b56b333b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:09.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.787+0000 7f8e53fff640 1 --2- 172.21.15.67:0/3705673986 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f8e540ba9a0 0x7f8e540bceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:09.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.788+0000 7f8e6a59c640 1 -- 172.21.15.67:0/3363612321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b56af6bb60 msgr2=0x55b56b334080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:38:09.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.788+0000 7f8e6a59c640 1 --2- 172.21.15.67:0/3363612321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b56af6bb60 0x55b56b334080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.788+0000 7f8e6a59c640 1 -- 172.21.15.67:0/3363612321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b56af63980 msgr2=0x55b56b333b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:38:09.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.788+0000 7f8e6a59c640 1 --2- 172.21.15.67:0/3363612321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b56af63980 0x55b56b333b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.788+0000 7f8e6a59c640 1 -- 172.21.15.67:0/3363612321 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b56b220060 con 0x55b56af8c840 2024-09-21T06:38:09.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.788+0000 7f8e53fff640 1 -- 172.21.15.67:0/3705673986 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f8e641081d0 con 0x7f8e6c108d40 2024-09-21T06:38:09.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.788+0000 7f8e6ad9d640 1 --2- 172.21.15.67:0/3363612321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b56af63980 0x55b56b333b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:38:09.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.788+0000 7f8e6a59c640 1 --2- 172.21.15.67:0/3363612321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b56af8c840 0x55b56b333600 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8e600126b0 tx=0x7f8e60009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89b5eec8e7ca39e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:38:09.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.788+0000 7f8e6b59e640 1 --2- 172.21.15.67:0/3705673986 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f8e540ba9a0 0x7f8e540bceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:09.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.788+0000 7f8e517fa640 1 -- 172.21.15.67:0/3363612321 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e60012040 con 0x55b56af8c840 2024-09-21T06:38:09.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.788+0000 7f8e517fa640 1 -- 172.21.15.67:0/3363612321 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e600121e0 con 0x55b56af8c840 2024-09-21T06:38:09.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.789+0000 7f8e517fa640 1 -- 172.21.15.67:0/3363612321 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e60012040 con 0x55b56af8c840 2024-09-21T06:38:09.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.789+0000 7f8e6b59e640 1 --2- 172.21.15.67:0/3705673986 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f8e540ba9a0 0x7f8e540bceb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8e64030800 tx=0x7f8e6402a6c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:38:09.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.789+0000 7f8e776c4740 1 -- 172.21.15.67:0/3363612321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b56af8c840 msgr2=0x55b56b333600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:09.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.789+0000 7f8e776c4740 1 --2- 172.21.15.67:0/3363612321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b56af8c840 0x55b56b333600 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8e600126b0 tx=0x7f8e60009f90 comp rx=0 tx=0).stop 2024-09-21T06:38:09.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.790+0000 7f8e776c4740 1 -- 172.21.15.67:0/3363612321 shutdown_connections 2024-09-21T06:38:09.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.790+0000 7f8e776c4740 1 --2- 172.21.15.67:0/3363612321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b56af6bb60 0x55b56b334080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.790+0000 7f8e776c4740 1 --2- 172.21.15.67:0/3363612321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b56af63980 0x55b56b333b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.790+0000 7f8e776c4740 1 --2- 172.21.15.67:0/3363612321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b56af8c840 0x55b56b333600 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.790+0000 7f8e776c4740 1 -- 172.21.15.67:0/3363612321 >> 172.21.15.67:0/3363612321 conn(0x55b56af80990 msgr2=0x55b56b3314f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:38:09.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.790+0000 7f8e776c4740 1 -- 172.21.15.67:0/3363612321 shutdown_connections 2024-09-21T06:38:09.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.790+0000 7f8e776c4740 1 -- 172.21.15.67:0/3363612321 wait complete. 2024-09-21T06:38:09.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.791+0000 7f8e776c4740 1 -- start start 2024-09-21T06:38:09.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.791+0000 7f8e776c4740 1 Processor -- start 2024-09-21T06:38:09.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.791+0000 7f8e776c4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b56af63980 0x55b56b33f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:09.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.791+0000 7f8e776c4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b56af6bb60 0x55b56b33f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:09.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.791+0000 7f8e776c4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b56af8c840 0x55b56b34b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:09.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.791+0000 7f8e776c4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55b56b0fa7a0 con 0x55b56af8c840 2024-09-21T06:38:09.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.791+0000 7f8e776c4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55b56b23e0a0 con 0x55b56af6bb60 2024-09-21T06:38:09.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.791+0000 7f8e776c4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b56b341e30 con 0x55b56af63980 2024-09-21T06:38:09.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.791+0000 7f8e6ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b56af6bb60 0x55b56b33f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:09.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.792+0000 7f8e6ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b56af6bb60 0x55b56b33f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.67:53840/0 (socket says 172.21.15.67:53840) 2024-09-21T06:38:09.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.791+0000 7f8e776c4740 10 client.0 _ll_register_callbacks cb 0x55b56b21b3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:38:09.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.792+0000 7f8e6ad9d640 1 -- 172.21.15.67:0/2607708076 learned_addr learned my addr 172.21.15.67:0/2607708076 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:38:09.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.792+0000 7f8e6a59c640 1 --2- 172.21.15.67:0/2607708076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b56af63980 0x55b56b33f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:09.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.792+0000 7f8e6a59c640 1 --2- 172.21.15.67:0/2607708076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b56af8c840 0x55b56b34b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:09.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.792+0000 7f8e6ad9d640 1 -- 172.21.15.67:0/2607708076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b56af63980 msgr2=0x55b56b33f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:38:09.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.792+0000 7f8e6ad9d640 1 --2- 172.21.15.67:0/2607708076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b56af63980 0x55b56b33f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.792+0000 7f8e6ad9d640 1 -- 172.21.15.67:0/2607708076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b56af8c840 msgr2=0x55b56b34b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:38:09.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.792+0000 7f8e6ad9d640 1 --2- 172.21.15.67:0/2607708076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b56af8c840 0x55b56b34b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.792+0000 7f8e6ad9d640 1 -- 172.21.15.67:0/2607708076 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b56b22c880 con 0x55b56af6bb60 2024-09-21T06:38:09.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.792+0000 7f8e6a59c640 1 --2- 172.21.15.67:0/2607708076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b56af63980 0x55b56b33f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:38:09.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.792+0000 7f8e6ad9d640 1 --2- 172.21.15.67:0/2607708076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b56af6bb60 0x55b56b33f6b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c013790 tx=0x7f8e5c013c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3882585308e2cb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:38:09.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.792+0000 7f8e4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-21T06:38:09.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.792+0000 7f8e4eff5640 1 -- 172.21.15.67:0/2607708076 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c020060 con 0x55b56af6bb60 2024-09-21T06:38:09.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.793+0000 7f8e776c4740 1 -- 172.21.15.67:0/2607708076 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e600121e0 con 0x55b56af6bb60 2024-09-21T06:38:09.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.793+0000 7f8e4eff5640 1 -- 172.21.15.67:0/2607708076 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f8e5c011dd0 con 0x55b56af6bb60 2024-09-21T06:38:09.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.793+0000 7f8e4eff5640 1 -- 172.21.15.67:0/2607708076 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8e5c027050 con 0x55b56af6bb60 2024-09-21T06:38:09.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.793+0000 7f8e776c4740 10 client.25235 fetch_fsmap learned FSMap version 21 2024-09-21T06:38:09.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.793+0000 7f8e776c4740 1 -- 172.21.15.67:0/2607708076 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b56b006b90 con 0x55b56af6bb60 2024-09-21T06:38:09.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.793+0000 7f8e4eff5640 1 -- 172.21.15.67:0/2607708076 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e600121e0 con 0x55b56af6bb60 2024-09-21T06:38:09.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.794+0000 7f8e4eff5640 1 -- 172.21.15.67:0/2607708076 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f8e5c01a030 con 0x55b56af6bb60 2024-09-21T06:38:09.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.794+0000 7f8e776c4740 10 client.25235 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:38:09.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.794+0000 7f8e776c4740 10 client.25235 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:38:09.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.794+0000 7f8e776c4740 20 client.25235 populate_metadata read hostname 'smithi067' 2024-09-21T06:38:09.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.794+0000 7f8e776c4740 1 --2- 172.21.15.67:0/2607708076 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55b56af4b1c0 0x55b56b359e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:09.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.794+0000 7f8e776c4740 4 client.25235 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:38:09.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.794+0000 7f8e776c4740 1 -- 172.21.15.67:0/2607708076 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b56b2abae0 con 0x55b56af4b1c0 2024-09-21T06:38:09.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.794+0000 7f8e6a59c640 1 --2- 172.21.15.67:0/2607708076 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55b56af4b1c0 0x55b56b359e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:09.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.795+0000 7f8e4eff5640 10 client.25235 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:38:09.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.795+0000 7f8e6a59c640 1 --2- 172.21.15.67:0/2607708076 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55b56af4b1c0 0x55b56b359e00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43e6608183fb762e server_cookie=c0cf37dd457f9325 in_seq=0 out_seq=0 2024-09-21T06:38:09.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.795+0000 7f8e4eff5640 1 -- 172.21.15.67:0/2607708076 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b56b2abae0 con 0x55b56af4b1c0 2024-09-21T06:38:09.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.795+0000 7f8e4eff5640 10 client.25235 handle_command_reply: tid=0 2024-09-21T06:38:09.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.796+0000 7f8e776c4740 1 client.25235 shutdown 2024-09-21T06:38:09.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.796+0000 7f8e776c4740 1 -- 172.21.15.67:0/2607708076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b56af6bb60 msgr2=0x55b56b33f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:09.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.796+0000 7f8e776c4740 1 --2- 172.21.15.67:0/2607708076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b56af6bb60 0x55b56b33f6b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c013790 tx=0x7f8e5c013c50 comp rx=0 tx=0).stop 2024-09-21T06:38:09.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.796+0000 7f8e776c4740 1 -- 172.21.15.67:0/2607708076 shutdown_connections 2024-09-21T06:38:09.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.796+0000 7f8e776c4740 1 --2- 172.21.15.67:0/2607708076 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55b56af4b1c0 0x55b56b359e00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.796+0000 7f8e776c4740 1 --2- 172.21.15.67:0/2607708076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b56af6bb60 0x55b56b33f6b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.796+0000 7f8e776c4740 1 --2- 172.21.15.67:0/2607708076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b56af8c840 0x55b56b34b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.796+0000 7f8e776c4740 1 --2- 172.21.15.67:0/2607708076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b56af63980 0x55b56b33f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.796+0000 7f8e776c4740 1 -- 172.21.15.67:0/2607708076 >> 172.21.15.67:0/2607708076 conn(0x55b56af80990 msgr2=0x55b56b334ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:38:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.796+0000 7f8e4eff5640 0 client.25235 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:38:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.796+0000 7f8e776c4740 1 -- 172.21.15.67:0/2607708076 shutdown_connections 2024-09-21T06:38:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.796+0000 7f8e776c4740 1 -- 172.21.15.67:0/2607708076 wait complete. 2024-09-21T06:38:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.796+0000 7f8e776c4740 20 client.25235 trim_cache size 0 max 16384 2024-09-21T06:38:09.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.828+0000 7f8e776c4740 1 Processor -- start 2024-09-21T06:38:09.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.828+0000 7f8e776c4740 1 -- start start 2024-09-21T06:38:09.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.828+0000 7f8e776c4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b56af63980 0x55b56b340e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:09.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.828+0000 7f8e776c4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b56af6bb60 0x55b56b338880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:09.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.828+0000 7f8e776c4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b56af8c840 0x55b56b330aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:09.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.828+0000 7f8e776c4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c1ac910 con 0x55b56af63980 2024-09-21T06:38:09.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.828+0000 7f8e776c4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c1af190 con 0x55b56af6bb60 2024-09-21T06:38:09.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.828+0000 7f8e776c4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c1acf80 con 0x55b56af8c840 2024-09-21T06:38:09.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.828+0000 7f8e6a59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b56af63980 0x55b56b340e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:09.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.828+0000 7f8e6ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b56af6bb60 0x55b56b338880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:09.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.828+0000 7f8e6a59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b56af63980 0x55b56b340e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55120/0 (socket says 172.21.15.67:55120) 2024-09-21T06:38:09.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.828+0000 7f8e6a59c640 1 -- 172.21.15.67:0/3532420883 learned_addr learned my addr 172.21.15.67:0/3532420883 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:38:09.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.829+0000 7f8e6a59c640 1 --2- 172.21.15.67:0/3532420883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b56af8c840 0x55b56b330aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:09.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.829+0000 7f8e6ad9d640 1 -- 172.21.15.67:0/3532420883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b56af8c840 msgr2=0x55b56b330aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:38:09.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.829+0000 7f8e6ad9d640 1 --2- 172.21.15.67:0/3532420883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b56af8c840 0x55b56b330aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.829+0000 7f8e6ad9d640 1 -- 172.21.15.67:0/3532420883 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b56af63980 msgr2=0x55b56b340e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:38:09.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.829+0000 7f8e6ad9d640 1 --2- 172.21.15.67:0/3532420883 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b56af63980 0x55b56b340e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.829+0000 7f8e6ad9d640 1 -- 172.21.15.67:0/3532420883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b56af4b3f0 con 0x55b56af6bb60 2024-09-21T06:38:09.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.829+0000 7f8e6ad9d640 1 --2- 172.21.15.67:0/3532420883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b56af6bb60 0x55b56b338880 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c00e6e0 tx=0x7f8e5c009dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e37f7eb000b9da3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:38:09.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.830+0000 7f8e50ff9640 1 -- 172.21.15.67:0/3532420883 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c000e60 con 0x55b56af6bb60 2024-09-21T06:38:09.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.830+0000 7f8e50ff9640 1 -- 172.21.15.67:0/3532420883 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e5c00e760 con 0x55b56af6bb60 2024-09-21T06:38:09.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.830+0000 7f8e50ff9640 1 -- 172.21.15.67:0/3532420883 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c00ee00 con 0x55b56af6bb60 2024-09-21T06:38:09.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.831+0000 7f8e776c4740 1 -- 172.21.15.67:0/3532420883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b56af6bb60 msgr2=0x55b56b338880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:09.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.831+0000 7f8e776c4740 1 --2- 172.21.15.67:0/3532420883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b56af6bb60 0x55b56b338880 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c00e6e0 tx=0x7f8e5c009dd0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.831+0000 7f8e776c4740 1 -- 172.21.15.67:0/3532420883 shutdown_connections 2024-09-21T06:38:09.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.831+0000 7f8e776c4740 1 --2- 172.21.15.67:0/3532420883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b56af6bb60 0x55b56b338880 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.831+0000 7f8e776c4740 1 --2- 172.21.15.67:0/3532420883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b56af8c840 0x55b56b330aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.831+0000 7f8e776c4740 1 --2- 172.21.15.67:0/3532420883 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b56af63980 0x55b56b340e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.832+0000 7f8e776c4740 1 -- 172.21.15.67:0/3532420883 >> 172.21.15.67:0/3532420883 conn(0x55b56af80990 msgr2=0x55b56b33a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:38:09.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.832+0000 7f8e776c4740 1 -- 172.21.15.67:0/3532420883 shutdown_connections 2024-09-21T06:38:09.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.832+0000 7f8e776c4740 1 -- 172.21.15.67:0/3532420883 wait complete. 2024-09-21T06:38:09.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.832+0000 7f8e776c4740 1 -- start start 2024-09-21T06:38:09.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.832+0000 7f8e776c4740 1 Processor -- start 2024-09-21T06:38:09.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.833+0000 7f8e776c4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b56af63980 0x55b56b347780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:09.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.833+0000 7f8e776c4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b56af6bb60 0x55b56b342040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:09.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.833+0000 7f8e776c4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b56af8c840 0x55b56b342580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:09.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.833+0000 7f8e776c4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c1ad650 con 0x55b56af8c840 2024-09-21T06:38:09.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.833+0000 7f8e776c4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c1aeac0 con 0x55b56af6bb60 2024-09-21T06:38:09.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.833+0000 7f8e776c4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c1add20 con 0x55b56af63980 2024-09-21T06:38:09.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.833+0000 7f8e776c4740 10 client.0 _ll_register_callbacks cb 0x7f8e5c009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:38:09.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.833+0000 7f8e6a59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b56af63980 0x55b56b347780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:09.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.833+0000 7f8e6a59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b56af63980 0x55b56b347780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.67:58042/0 (socket says 172.21.15.67:58042) 2024-09-21T06:38:09.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.833+0000 7f8e6a59c640 1 -- 172.21.15.67:0/201002201 learned_addr learned my addr 172.21.15.67:0/201002201 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:38:09.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.833+0000 7f8e6a59c640 1 --2- 172.21.15.67:0/201002201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b56af8c840 0x55b56b342580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:09.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.833+0000 7f8e6ad9d640 1 --2- 172.21.15.67:0/201002201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b56af6bb60 0x55b56b342040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:09.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.834+0000 7f8e6a59c640 1 -- 172.21.15.67:0/201002201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b56af6bb60 msgr2=0x55b56b342040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:38:09.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.834+0000 7f8e6a59c640 1 --2- 172.21.15.67:0/201002201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b56af6bb60 0x55b56b342040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.834+0000 7f8e6a59c640 1 -- 172.21.15.67:0/201002201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b56af8c840 msgr2=0x55b56b342580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:38:09.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.834+0000 7f8e6a59c640 1 --2- 172.21.15.67:0/201002201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b56af8c840 0x55b56b342580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.834+0000 7f8e6a59c640 1 -- 172.21.15.67:0/201002201 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b56b342c20 con 0x55b56af63980 2024-09-21T06:38:09.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.834+0000 7f8e6a59c640 1 --2- 172.21.15.67:0/201002201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b56af63980 0x55b56b347780 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x55b56b331fc0 tx=0x7f8e60021040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b98a578d6cd8068 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:38:09.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.834+0000 7f8e4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T06:38:09.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.834+0000 7f8e4eff5640 1 -- 172.21.15.67:0/201002201 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e60012040 con 0x55b56af63980 2024-09-21T06:38:09.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.834+0000 7f8e4eff5640 1 -- 172.21.15.67:0/201002201 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f8e6001b020 con 0x55b56af63980 2024-09-21T06:38:09.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.834+0000 7f8e776c4740 1 -- 172.21.15.67:0/201002201 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e5c00e760 con 0x55b56af63980 2024-09-21T06:38:09.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.835+0000 7f8e4eff5640 1 -- 172.21.15.67:0/201002201 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8e60011650 con 0x55b56af63980 2024-09-21T06:38:09.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.835+0000 7f8e776c4740 10 client.25207 fetch_fsmap learned FSMap version 21 2024-09-21T06:38:09.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.835+0000 7f8e776c4740 1 -- 172.21.15.67:0/201002201 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b56b3c5630 con 0x55b56af63980 2024-09-21T06:38:09.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.835+0000 7f8e4eff5640 1 -- 172.21.15.67:0/201002201 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e5c00e760 con 0x55b56af63980 2024-09-21T06:38:09.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.835+0000 7f8e4eff5640 1 -- 172.21.15.67:0/201002201 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f8e60033040 con 0x55b56af63980 2024-09-21T06:38:09.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.835+0000 7f8e776c4740 10 client.25207 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:38:09.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.835+0000 7f8e776c4740 10 client.25207 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:38:09.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.835+0000 7f8e776c4740 20 client.25207 populate_metadata read hostname 'smithi067' 2024-09-21T06:38:09.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.835+0000 7f8e776c4740 1 --2- 172.21.15.67:0/201002201 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55b56af20570 0x55b56b3c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:09.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.835+0000 7f8e776c4740 4 client.25207 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.0, "await_for": 45.0}] 2024-09-21T06:38:09.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.835+0000 7f8e776c4740 1 -- 172.21.15.67:0/201002201 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.0, "await_for": 45.0}) -- 0x55b56b2a7590 con 0x55b56af20570 2024-09-21T06:38:09.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.836+0000 7f8e6ad9d640 1 --2- 172.21.15.67:0/201002201 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55b56af20570 0x55b56b3c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:09.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.836+0000 7f8e6ad9d640 1 --2- 172.21.15.67:0/201002201 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55b56af20570 0x55b56b3c7c70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0bb9671b438e77e server_cookie=ad704d1699eb8b19 in_seq=0 out_seq=0 2024-09-21T06:38:09.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.836+0000 7f8e4eff5640 10 client.25207 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:38:09.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.871+0000 7f8e4eff5640 1 -- 172.21.15.67:0/201002201 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55b56b2a7590 con 0x55b56af20570 2024-09-21T06:38:09.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.871+0000 7f8e4eff5640 10 client.25207 handle_command_reply: tid=0 2024-09-21T06:38:09.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.871+0000 7f8e776c4740 1 client.25207 shutdown 2024-09-21T06:38:09.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:09 smithi067 ceph-mon[31232]: pgmap v407: 129 pgs: 129 active+clean; 679 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1.6 KiB/s wr, 1 op/s 2024-09-21T06:38:09.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.871+0000 7f8e776c4740 1 -- 172.21.15.67:0/201002201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b56af63980 msgr2=0x55b56b347780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:09.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.871+0000 7f8e776c4740 1 --2- 172.21.15.67:0/201002201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b56af63980 0x55b56b347780 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x55b56b331fc0 tx=0x7f8e60021040 comp rx=0 tx=0).stop 2024-09-21T06:38:09.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.872+0000 7f8e776c4740 1 -- 172.21.15.67:0/201002201 shutdown_connections 2024-09-21T06:38:09.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.872+0000 7f8e776c4740 1 --2- 172.21.15.67:0/201002201 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55b56af20570 0x55b56b3c7c70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.872+0000 7f8e776c4740 1 --2- 172.21.15.67:0/201002201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b56af6bb60 0x55b56b342040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.872+0000 7f8e776c4740 1 --2- 172.21.15.67:0/201002201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b56af8c840 0x55b56b342580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.872+0000 7f8e4eff5640 0 client.25207 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:38:09.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.872+0000 7f8e776c4740 1 --2- 172.21.15.67:0/201002201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b56af63980 0x55b56b347780 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.872+0000 7f8e776c4740 1 -- 172.21.15.67:0/201002201 >> 172.21.15.67:0/201002201 conn(0x55b56af80990 msgr2=0x55b56b336700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:38:09.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.872+0000 7f8e776c4740 1 -- 172.21.15.67:0/201002201 shutdown_connections 2024-09-21T06:38:09.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.872+0000 7f8e776c4740 1 -- 172.21.15.67:0/201002201 wait complete. 2024-09-21T06:38:09.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.872+0000 7f8e776c4740 20 client.25207 trim_cache size 0 max 16384 2024-09-21T06:38:09.875 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-21T06:38:09.875 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 21, 2024-09-21T06:38:09.875 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-09-21T06:38:09.875 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 6, 2024-09-21T06:38:09.875 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-21T06:38:09.875 INFO:teuthology.orchestra.run.smithi067.stdout: "4fe5f098": { 2024-09-21T06:38:09.875 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 6, 2024-09-21T06:38:09.875 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-21T06:38:09.875 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:38:09.875 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-21T06:38:09.875 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:38:09.875 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:38:09.875 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-21T06:38:09.876 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 137.0, 2024-09-21T06:38:09.876 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-21T06:38:09.876 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-21T06:38:09.876 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-21T06:38:09.876 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:38:09.876 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-21T06:38:09.876 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:38:09.876 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:38:09.876 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:38:09.876 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:38:09.876 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:38:09.876 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:38:09.876 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-21T06:38:09.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.873+0000 7f8e71150640 1 -- 172.21.15.67:0/3705673986 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f8e540ba9a0 msgr2=0x7f8e540bceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:09.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.873+0000 7f8e71150640 1 --2- 172.21.15.67:0/3705673986 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f8e540ba9a0 0x7f8e540bceb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8e64030800 tx=0x7f8e6402a6c0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.873+0000 7f8e71150640 1 -- 172.21.15.67:0/3705673986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e6c108d40 msgr2=0x7f8e6c19fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:09.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.873+0000 7f8e71150640 1 --2- 172.21.15.67:0/3705673986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e6c108d40 0x7f8e6c19fbd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8e64004af0 tx=0x7f8e64030b80 comp rx=0 tx=0).stop 2024-09-21T06:38:09.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.873+0000 7f8e71150640 1 -- 172.21.15.67:0/3705673986 shutdown_connections 2024-09-21T06:38:09.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.873+0000 7f8e71150640 1 --2- 172.21.15.67:0/3705673986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e6c1083f0 0x7f8e6c19a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.873+0000 7f8e71150640 1 --2- 172.21.15.67:0/3705673986 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f8e540ba9a0 0x7f8e540bceb0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.873+0000 7f8e71150640 1 --2- 172.21.15.67:0/3705673986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e6c108d40 0x7f8e6c19fbd0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.873+0000 7f8e71150640 1 --2- 172.21.15.67:0/3705673986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e6c102ab0 0x7f8e6c19a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:09.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.873+0000 7f8e71150640 1 -- 172.21.15.67:0/3705673986 >> 172.21.15.67:0/3705673986 conn(0x7f8e6c079870 msgr2=0x7f8e6c101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:38:09.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.873+0000 7f8e71150640 1 -- 172.21.15.67:0/3705673986 shutdown_connections 2024-09-21T06:38:09.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:09.873+0000 7f8e71150640 1 -- 172.21.15.67:0/3705673986 wait complete. 2024-09-21T06:38:09.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:09 smithi203 ceph-mon[33735]: pgmap v407: 129 pgs: 129 active+clean; 679 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1.6 KiB/s wr, 1 op/s 2024-09-21T06:38:10.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:09 smithi100 ceph-mon[33401]: pgmap v407: 129 pgs: 129 active+clean; 679 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1.6 KiB/s wr, 1 op/s 2024-09-21T06:38:10.586 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4fe5f098', quiesce took 4.0 seconds. Will release after: 13.0 2024-09-21T06:38:11.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:11 smithi203 ceph-mon[33735]: pgmap v408: 129 pgs: 129 active+clean; 707 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 4.3 KiB/s wr, 3 op/s 2024-09-21T06:38:12.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:11 smithi067 ceph-mon[31232]: pgmap v408: 129 pgs: 129 active+clean; 707 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 4.3 KiB/s wr, 3 op/s 2024-09-21T06:38:12.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:11 smithi100 ceph-mon[33401]: pgmap v408: 129 pgs: 129 active+clean; 707 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 4.3 KiB/s wr, 3 op/s 2024-09-21T06:38:13.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:13 smithi203 ceph-mon[33735]: pgmap v409: 129 pgs: 129 active+clean; 175 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 15 op/s 2024-09-21T06:38:14.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:13 smithi067 ceph-mon[31232]: pgmap v409: 129 pgs: 129 active+clean; 175 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 15 op/s 2024-09-21T06:38:14.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:13 smithi100 ceph-mon[33401]: pgmap v409: 129 pgs: 129 active+clean; 175 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 15 op/s 2024-09-21T06:38:15.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:15 smithi203 ceph-mon[33735]: pgmap v410: 129 pgs: 129 active+clean; 237 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 20 MiB/s wr, 17 op/s 2024-09-21T06:38:16.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:15 smithi067 ceph-mon[31232]: pgmap v410: 129 pgs: 129 active+clean; 237 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 20 MiB/s wr, 17 op/s 2024-09-21T06:38:16.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:15 smithi100 ceph-mon[33401]: pgmap v410: 129 pgs: 129 active+clean; 237 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 20 MiB/s wr, 17 op/s 2024-09-21T06:38:17.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:17 smithi203 ceph-mon[33735]: pgmap v411: 129 pgs: 129 active+clean; 238 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 20 MiB/s wr, 18 op/s 2024-09-21T06:38:18.039 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:17 smithi067 ceph-mon[31232]: pgmap v411: 129 pgs: 129 active+clean; 238 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 20 MiB/s wr, 18 op/s 2024-09-21T06:38:18.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:17 smithi100 ceph-mon[33401]: pgmap v411: 129 pgs: 129 active+clean; 238 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 20 MiB/s wr, 18 op/s 2024-09-21T06:38:19.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:19 smithi203 ceph-mon[33735]: pgmap v412: 129 pgs: 129 active+clean; 243 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 20 MiB/s wr, 18 op/s 2024-09-21T06:38:20.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:19 smithi067 ceph-mon[31232]: pgmap v412: 129 pgs: 129 active+clean; 243 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 20 MiB/s wr, 18 op/s 2024-09-21T06:38:20.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:19 smithi100 ceph-mon[33401]: pgmap v412: 129 pgs: 129 active+clean; 243 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 20 MiB/s wr, 18 op/s 2024-09-21T06:38:21.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:21 smithi203 ceph-mon[33735]: pgmap v413: 129 pgs: 129 active+clean; 243 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 20 MiB/s wr, 18 op/s 2024-09-21T06:38:22.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:21 smithi067 ceph-mon[31232]: pgmap v413: 129 pgs: 129 active+clean; 243 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 20 MiB/s wr, 18 op/s 2024-09-21T06:38:22.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:21 smithi100 ceph-mon[33401]: pgmap v413: 129 pgs: 129 active+clean; 243 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 20 MiB/s wr, 18 op/s 2024-09-21T06:38:23.587 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:38:23.862 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:38:23.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:23 smithi203 ceph-mon[33735]: pgmap v414: 129 pgs: 129 active+clean; 243 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 20 MiB/s wr, 18 op/s 2024-09-21T06:38:23.984 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:23 smithi067 ceph-mon[31232]: pgmap v414: 129 pgs: 129 active+clean; 243 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 20 MiB/s wr, 18 op/s 2024-09-21T06:38:24.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:23 smithi100 ceph-mon[33401]: pgmap v414: 129 pgs: 129 active+clean; 243 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 20 MiB/s wr, 18 op/s 2024-09-21T06:38:24.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.414+0000 7f73cc2ce640 1 -- 172.21.15.67:0/4246350929 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73c4103bb0 msgr2=0x7f73c4103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:24.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.414+0000 7f73cc2ce640 1 --2- 172.21.15.67:0/4246350929 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73c4103bb0 0x7f73c4103fc0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009900 tx=0x7f73c002a410 comp rx=0 tx=0).stop 2024-09-21T06:38:24.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.414+0000 7f73cc2ce640 1 -- 172.21.15.67:0/4246350929 shutdown_connections 2024-09-21T06:38:24.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.414+0000 7f73cc2ce640 1 --2- 172.21.15.67:0/4246350929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73c41094f0 0x7f73c4109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:24.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.414+0000 7f73cc2ce640 1 --2- 172.21.15.67:0/4246350929 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73c4109e40 0x7f73c4110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:24.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.414+0000 7f73cc2ce640 1 --2- 172.21.15.67:0/4246350929 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73c4103bb0 0x7f73c4103fc0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:24.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.414+0000 7f73cc2ce640 1 -- 172.21.15.67:0/4246350929 >> 172.21.15.67:0/4246350929 conn(0x7f73c40ff600 msgr2=0x7f73c4101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:38:24.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.414+0000 7f73cc2ce640 1 -- 172.21.15.67:0/4246350929 shutdown_connections 2024-09-21T06:38:24.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.414+0000 7f73cc2ce640 1 -- 172.21.15.67:0/4246350929 wait complete. 2024-09-21T06:38:24.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.414+0000 7f73cc2ce640 1 Processor -- start 2024-09-21T06:38:24.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.415+0000 7f73cc2ce640 1 -- start start 2024-09-21T06:38:24.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.415+0000 7f73cc2ce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73c4103bb0 0x7f73c419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:24.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.415+0000 7f73cc2ce640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73c41094f0 0x7f73c419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:24.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.415+0000 7f73cc2ce640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73c4109e40 0x7f73c419fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:24.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.415+0000 7f73cc2ce640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f73c41139b0 con 0x7f73c41094f0 2024-09-21T06:38:24.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.415+0000 7f73cc2ce640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f73c4113830 con 0x7f73c4103bb0 2024-09-21T06:38:24.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.415+0000 7f73cc2ce640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f73c4113b30 con 0x7f73c4109e40 2024-09-21T06:38:24.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.415+0000 7f73c9869640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73c41094f0 0x7f73c419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:24.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.415+0000 7f73c9869640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73c41094f0 0x7f73c419a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53304/0 (socket says 172.21.15.67:53304) 2024-09-21T06:38:24.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.415+0000 7f73c9869640 1 -- 172.21.15.67:0/1129885286 learned_addr learned my addr 172.21.15.67:0/1129885286 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:38:24.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.415+0000 7f73ca06a640 1 --2- 172.21.15.67:0/1129885286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73c4103bb0 0x7f73c419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:24.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.415+0000 7f73ca86b640 1 --2- 172.21.15.67:0/1129885286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73c4109e40 0x7f73c419fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:24.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.415+0000 7f73c9869640 1 -- 172.21.15.67:0/1129885286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73c4109e40 msgr2=0x7f73c419fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:24.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.415+0000 7f73c9869640 1 --2- 172.21.15.67:0/1129885286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73c4109e40 0x7f73c419fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:24.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.415+0000 7f73c9869640 1 -- 172.21.15.67:0/1129885286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73c4103bb0 msgr2=0x7f73c419a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:24.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.415+0000 7f73c9869640 1 --2- 172.21.15.67:0/1129885286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73c4103bb0 0x7f73c419a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:24.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.415+0000 7f73c9869640 1 -- 172.21.15.67:0/1129885286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73c41a0370 con 0x7f73c41094f0 2024-09-21T06:38:24.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.415+0000 7f73c9869640 1 --2- 172.21.15.67:0/1129885286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73c41094f0 0x7f73c419a9a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f73b400d740 tx=0x7f73b400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:38:24.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.416+0000 7f73bb7fe640 1 -- 172.21.15.67:0/1129885286 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b4017070 con 0x7f73c41094f0 2024-09-21T06:38:24.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.416+0000 7f73cc2ce640 1 -- 172.21.15.67:0/1129885286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73c41a0660 con 0x7f73c41094f0 2024-09-21T06:38:24.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.416+0000 7f73bb7fe640 1 -- 172.21.15.67:0/1129885286 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f73b400de50 con 0x7f73c41094f0 2024-09-21T06:38:24.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.416+0000 7f73bb7fe640 1 -- 172.21.15.67:0/1129885286 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b400f620 con 0x7f73c41094f0 2024-09-21T06:38:24.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.416+0000 7f73cc2ce640 1 -- 172.21.15.67:0/1129885286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73c41a6dc0 con 0x7f73c41094f0 2024-09-21T06:38:24.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.417+0000 7f73cc2ce640 1 -- 172.21.15.67:0/1129885286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73c4103fc0 con 0x7f73c41094f0 2024-09-21T06:38:24.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.418+0000 7f73bb7fe640 1 -- 172.21.15.67:0/1129885286 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f73b401a070 con 0x7f73c41094f0 2024-09-21T06:38:24.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.419+0000 7f73bb7fe640 1 --2- 172.21.15.67:0/1129885286 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f73b00b6510 0x7f73b00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:24.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.419+0000 7f73ca06a640 1 --2- 172.21.15.67:0/1129885286 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f73b00b6510 0x7f73b00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:24.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.419+0000 7f73bb7fe640 1 -- 172.21.15.67:0/1129885286 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f73b40e7100 con 0x7f73c41094f0 2024-09-21T06:38:24.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.420+0000 7f73ca06a640 1 --2- 172.21.15.67:0/1129885286 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f73b00b6510 0x7f73b00b8a20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f73c00057c0 tx=0x7f73c00058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:38:24.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.423+0000 7f73bb7fe640 1 -- 172.21.15.67:0/1129885286 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f73b40ad5c0 con 0x7f73c41094f0 2024-09-21T06:38:24.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.651+0000 7f73cc2ce640 1 -- 172.21.15.67:0/1129885286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73c402e640 con 0x7f73c41094f0 2024-09-21T06:38:24.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.652+0000 7f73bb7fe640 1 -- 172.21.15.67:0/1129885286 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7f73b40b19a0 con 0x7f73c41094f0 2024-09-21T06:38:24.655 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:38:24.655 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:38:24.655 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:38:24.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.656+0000 7f73cc2ce640 1 -- 172.21.15.67:0/1129885286 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f73b00b6510 msgr2=0x7f73b00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:24.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.656+0000 7f73cc2ce640 1 --2- 172.21.15.67:0/1129885286 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f73b00b6510 0x7f73b00b8a20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f73c00057c0 tx=0x7f73c00058d0 comp rx=0 tx=0).stop 2024-09-21T06:38:24.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.656+0000 7f73cc2ce640 1 -- 172.21.15.67:0/1129885286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73c41094f0 msgr2=0x7f73c419a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:24.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.656+0000 7f73cc2ce640 1 --2- 172.21.15.67:0/1129885286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73c41094f0 0x7f73c419a9a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f73b400d740 tx=0x7f73b400dc00 comp rx=0 tx=0).stop 2024-09-21T06:38:24.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.656+0000 7f73cc2ce640 1 -- 172.21.15.67:0/1129885286 shutdown_connections 2024-09-21T06:38:24.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.656+0000 7f73cc2ce640 1 --2- 172.21.15.67:0/1129885286 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f73b00b6510 0x7f73b00b8a20 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:24.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.656+0000 7f73cc2ce640 1 --2- 172.21.15.67:0/1129885286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73c4109e40 0x7f73c419fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:24.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.656+0000 7f73cc2ce640 1 --2- 172.21.15.67:0/1129885286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73c41094f0 0x7f73c419a9a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:24.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.656+0000 7f73cc2ce640 1 --2- 172.21.15.67:0/1129885286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73c4103bb0 0x7f73c419a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:24.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.656+0000 7f73cc2ce640 1 -- 172.21.15.67:0/1129885286 >> 172.21.15.67:0/1129885286 conn(0x7f73c40ff600 msgr2=0x7f73c4101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:38:24.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.656+0000 7f73cc2ce640 1 -- 172.21.15.67:0/1129885286 shutdown_connections 2024-09-21T06:38:24.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:24.656+0000 7f73cc2ce640 1 -- 172.21.15.67:0/1129885286 wait complete. 2024-09-21T06:38:25.412 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db --set-id 4fe5f098 --release --await' 2024-09-21T06:38:25.412 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24671 quiesce db --set-id 4fe5f098 --release --await 2024-09-21T06:38:25.674 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:38:25.929 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:25 smithi067 ceph-mon[31232]: pgmap v415: 129 pgs: 129 active+clean; 243 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5.7 MiB/s wr, 5 op/s 2024-09-21T06:38:25.929 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:25 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1129885286' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:38:25.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:25 smithi203 ceph-mon[33735]: pgmap v415: 129 pgs: 129 active+clean; 243 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5.7 MiB/s wr, 5 op/s 2024-09-21T06:38:25.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:25 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1129885286' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:38:26.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:25 smithi100 ceph-mon[33401]: pgmap v415: 129 pgs: 129 active+clean; 243 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5.7 MiB/s wr, 5 op/s 2024-09-21T06:38:26.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:25 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1129885286' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:38:26.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.168+0000 7f0035b41640 1 -- 172.21.15.67:0/2230747050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0030102a60 msgr2=0x7f0030102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:26.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.168+0000 7f0035b41640 1 --2- 172.21.15.67:0/2230747050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0030102a60 0x7f0030102e70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0020009900 tx=0x7f002002a410 comp rx=0 tx=0).stop 2024-09-21T06:38:26.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.168+0000 7f0035b41640 1 -- 172.21.15.67:0/2230747050 shutdown_connections 2024-09-21T06:38:26.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.168+0000 7f0035b41640 1 --2- 172.21.15.67:0/2230747050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00301083a0 0x7f00301087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.168+0000 7f0035b41640 1 --2- 172.21.15.67:0/2230747050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0030108cf0 0x7f003010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.168+0000 7f0035b41640 1 --2- 172.21.15.67:0/2230747050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0030102a60 0x7f0030102e70 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.168+0000 7f0035b41640 1 -- 172.21.15.67:0/2230747050 >> 172.21.15.67:0/2230747050 conn(0x7f0030079730 msgr2=0x7f0030079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:38:26.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.168+0000 7f0035b41640 1 -- 172.21.15.67:0/2230747050 shutdown_connections 2024-09-21T06:38:26.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.168+0000 7f0035b41640 1 -- 172.21.15.67:0/2230747050 wait complete. 2024-09-21T06:38:26.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.169+0000 7f0035b41640 1 Processor -- start 2024-09-21T06:38:26.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.169+0000 7f0035b41640 1 -- start start 2024-09-21T06:38:26.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.170+0000 7f0035b41640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0030102a60 0x7f003019a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:26.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.170+0000 7f0035b41640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00301083a0 0x7f003019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:26.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.170+0000 7f0035b41640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0030108cf0 0x7f003019fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:26.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.170+0000 7f0035b41640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0030113930 con 0x7f00301083a0 2024-09-21T06:38:26.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.170+0000 7f0035b41640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f00301137b0 con 0x7f0030102a60 2024-09-21T06:38:26.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.170+0000 7f0035b41640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0030113ab0 con 0x7f0030108cf0 2024-09-21T06:38:26.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.170+0000 7f002effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00301083a0 0x7f003019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:26.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.170+0000 7f002effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00301083a0 0x7f003019a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53326/0 (socket says 172.21.15.67:53326) 2024-09-21T06:38:26.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.170+0000 7f002effd640 1 -- 172.21.15.67:0/2494587104 learned_addr learned my addr 172.21.15.67:0/2494587104 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:38:26.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.170+0000 7f002ffff640 1 --2- 172.21.15.67:0/2494587104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0030108cf0 0x7f003019fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:26.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.170+0000 7f002f7fe640 1 --2- 172.21.15.67:0/2494587104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0030102a60 0x7f003019a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:26.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.170+0000 7f002effd640 1 -- 172.21.15.67:0/2494587104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0030108cf0 msgr2=0x7f003019fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:26.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.170+0000 7f002effd640 1 --2- 172.21.15.67:0/2494587104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0030108cf0 0x7f003019fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.170+0000 7f002effd640 1 -- 172.21.15.67:0/2494587104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0030102a60 msgr2=0x7f003019a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:26.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.170+0000 7f002effd640 1 --2- 172.21.15.67:0/2494587104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0030102a60 0x7f003019a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.170+0000 7f002effd640 1 -- 172.21.15.67:0/2494587104 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00301a01d0 con 0x7f00301083a0 2024-09-21T06:38:26.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.170+0000 7f002f7fe640 1 --2- 172.21.15.67:0/2494587104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0030102a60 0x7f003019a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:38:26.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.170+0000 7f002effd640 1 --2- 172.21.15.67:0/2494587104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00301083a0 0x7f003019a8c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f002400c970 tx=0x7f002400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:38:26.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.171+0000 7f002cff9640 1 -- 172.21.15.67:0/2494587104 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024014070 con 0x7f00301083a0 2024-09-21T06:38:26.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.171+0000 7f0035b41640 1 -- 172.21.15.67:0/2494587104 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00301a04c0 con 0x7f00301083a0 2024-09-21T06:38:26.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.171+0000 7f002cff9640 1 -- 172.21.15.67:0/2494587104 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0024004440 con 0x7f00301083a0 2024-09-21T06:38:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.171+0000 7f002cff9640 1 -- 172.21.15.67:0/2494587104 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024003d40 con 0x7f00301083a0 2024-09-21T06:38:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.171+0000 7f0035b41640 1 -- 172.21.15.67:0/2494587104 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00301a6ce0 con 0x7f00301083a0 2024-09-21T06:38:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.173+0000 7f003c0b5740 1 Processor -- start 2024-09-21T06:38:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.173+0000 7f003c0b5740 1 -- start start 2024-09-21T06:38:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.173+0000 7f002cff9640 1 -- 172.21.15.67:0/2494587104 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0024017070 con 0x7f00301083a0 2024-09-21T06:38:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.173+0000 7f003c0b5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca4057520 0x556ca43fd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.174+0000 7f003c0b5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556ca402e660 0x556ca43fda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.174+0000 7f003c0b5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ca4036840 0x556ca43fdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.174+0000 7f003c0b5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556ca42e8dc0 con 0x556ca402e660 2024-09-21T06:38:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.174+0000 7f003c0b5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556ca43751f0 con 0x556ca4036840 2024-09-21T06:38:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.174+0000 7f003c0b5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556ca4364620 con 0x556ca4057520 2024-09-21T06:38:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.174+0000 7f002f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556ca402e660 0x556ca43fda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.174+0000 7f002f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556ca402e660 0x556ca43fda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53342/0 (socket says 172.21.15.67:53342) 2024-09-21T06:38:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.174+0000 7f002f7fe640 1 -- 172.21.15.67:0/2401519228 learned_addr learned my addr 172.21.15.67:0/2401519228 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:38:26.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.174+0000 7f002ffff640 1 --2- 172.21.15.67:0/2401519228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca4057520 0x556ca43fd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:26.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.174+0000 7f002effd640 1 --2- 172.21.15.67:0/2401519228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ca4036840 0x556ca43fdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:26.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.174+0000 7f002f7fe640 1 -- 172.21.15.67:0/2401519228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca4057520 msgr2=0x556ca43fd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:38:26.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.174+0000 7f002f7fe640 1 --2- 172.21.15.67:0/2401519228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca4057520 0x556ca43fd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.174+0000 7f002f7fe640 1 -- 172.21.15.67:0/2401519228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ca4036840 msgr2=0x556ca43fdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:38:26.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.174+0000 7f002f7fe640 1 --2- 172.21.15.67:0/2401519228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ca4036840 0x556ca43fdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.174+0000 7f002f7fe640 1 -- 172.21.15.67:0/2401519228 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ca432cfd0 con 0x556ca402e660 2024-09-21T06:38:26.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.174+0000 7f002ffff640 1 --2- 172.21.15.67:0/2401519228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca4057520 0x556ca43fd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:38:26.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.174+0000 7f002f7fe640 1 --2- 172.21.15.67:0/2401519228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556ca402e660 0x556ca43fda50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0020000c00 tx=0x7f0020032dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6df3f829a0bf296 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:38:26.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.174+0000 7f002cff9640 1 --2- 172.21.15.67:0/2494587104 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f00180b6540 0x7f00180b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:26.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.175+0000 7f0015ffb640 1 -- 172.21.15.67:0/2401519228 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002003f070 con 0x556ca402e660 2024-09-21T06:38:26.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.175+0000 7f002ffff640 1 --2- 172.21.15.67:0/2494587104 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f00180b6540 0x7f00180b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:26.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.175+0000 7f0015ffb640 1 -- 172.21.15.67:0/2401519228 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00200044c0 con 0x556ca402e660 2024-09-21T06:38:26.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.175+0000 7f002cff9640 1 -- 172.21.15.67:0/2494587104 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f00240e72a0 con 0x7f00301083a0 2024-09-21T06:38:26.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.175+0000 7f0015ffb640 1 -- 172.21.15.67:0/2401519228 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0020035070 con 0x556ca402e660 2024-09-21T06:38:26.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.176+0000 7f003c0b5740 1 -- 172.21.15.67:0/2401519228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556ca402e660 msgr2=0x556ca43fda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:26.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.176+0000 7f003c0b5740 1 --2- 172.21.15.67:0/2401519228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556ca402e660 0x556ca43fda50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0020000c00 tx=0x7f0020032dc0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.176+0000 7f002ffff640 1 --2- 172.21.15.67:0/2494587104 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f00180b6540 0x7f00180b8a50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0028005fd0 tx=0x7f0028009580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:38:26.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.176+0000 7f003c0b5740 1 -- 172.21.15.67:0/2401519228 shutdown_connections 2024-09-21T06:38:26.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.176+0000 7f003c0b5740 1 --2- 172.21.15.67:0/2401519228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ca4036840 0x556ca43fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.176+0000 7f003c0b5740 1 --2- 172.21.15.67:0/2401519228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556ca402e660 0x556ca43fda50 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.176+0000 7f003c0b5740 1 --2- 172.21.15.67:0/2401519228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca4057520 0x556ca43fd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.176+0000 7f003c0b5740 1 -- 172.21.15.67:0/2401519228 >> 172.21.15.67:0/2401519228 conn(0x556ca404b910 msgr2=0x556ca43fb400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:38:26.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.176+0000 7f003c0b5740 1 -- 172.21.15.67:0/2401519228 shutdown_connections 2024-09-21T06:38:26.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.176+0000 7f003c0b5740 1 -- 172.21.15.67:0/2401519228 wait complete. 2024-09-21T06:38:26.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.176+0000 7f003c0b5740 1 -- start start 2024-09-21T06:38:26.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.176+0000 7f003c0b5740 1 Processor -- start 2024-09-21T06:38:26.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.177+0000 7f003c0b5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ca402e660 0x556ca4408b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:26.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.177+0000 7f003c0b5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556ca4036840 0x556ca4409090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:26.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.177+0000 7f003c0b5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca4057520 0x556ca44095f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:26.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.177+0000 7f003c0b5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556ca42ff300 con 0x556ca4036840 2024-09-21T06:38:26.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.177+0000 7f003c0b5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556ca4368430 con 0x556ca402e660 2024-09-21T06:38:26.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.177+0000 7f003c0b5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556ca41c5bb0 con 0x556ca4057520 2024-09-21T06:38:26.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.177+0000 7f003c0b5740 10 client.0 _ll_register_callbacks cb 0x556ca4319120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:38:26.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.177+0000 7f002effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556ca4036840 0x556ca4409090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:26.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.177+0000 7f002effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556ca4036840 0x556ca4409090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53350/0 (socket says 172.21.15.67:53350) 2024-09-21T06:38:26.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.177+0000 7f002effd640 1 -- 172.21.15.67:0/1115181429 learned_addr learned my addr 172.21.15.67:0/1115181429 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:38:26.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.177+0000 7f002f7fe640 1 --2- 172.21.15.67:0/1115181429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ca402e660 0x556ca4408b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:26.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.177+0000 7f002ffff640 1 --2- 172.21.15.67:0/1115181429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca4057520 0x556ca44095f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:26.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.177+0000 7f002effd640 1 -- 172.21.15.67:0/1115181429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca4057520 msgr2=0x556ca44095f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:38:26.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.177+0000 7f002effd640 1 --2- 172.21.15.67:0/1115181429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca4057520 0x556ca44095f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.177+0000 7f002effd640 1 -- 172.21.15.67:0/1115181429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ca402e660 msgr2=0x556ca4408b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:38:26.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.177+0000 7f002effd640 1 --2- 172.21.15.67:0/1115181429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ca402e660 0x556ca4408b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.177+0000 7f002effd640 1 -- 172.21.15.67:0/1115181429 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ca42ee780 con 0x556ca4036840 2024-09-21T06:38:26.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.178+0000 7f002effd640 1 --2- 172.21.15.67:0/1115181429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556ca4036840 0x556ca4409090 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f00240a9630 tx=0x7f00240057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6407b7dadd8fb27 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:38:26.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.178+0000 7f000effd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:38:26.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.178+0000 7f000effd640 1 -- 172.21.15.67:0/1115181429 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00240ac9b0 con 0x556ca4036840 2024-09-21T06:38:26.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.178+0000 7f000effd640 1 -- 172.21.15.67:0/1115181429 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f00240abd30 con 0x556ca4036840 2024-09-21T06:38:26.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.178+0000 7f003c0b5740 1 -- 172.21.15.67:0/1115181429 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556ca4377140 con 0x556ca4036840 2024-09-21T06:38:26.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.178+0000 7f000effd640 1 -- 172.21.15.67:0/1115181429 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f00240ac7d0 con 0x556ca4036840 2024-09-21T06:38:26.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.178+0000 7f003c0b5740 10 client.15522 fetch_fsmap learned FSMap version 21 2024-09-21T06:38:26.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.178+0000 7f003c0b5740 1 -- 172.21.15.67:0/1115181429 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556ca40c2b00 con 0x556ca4036840 2024-09-21T06:38:26.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.178+0000 7f000effd640 1 -- 172.21.15.67:0/1115181429 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x556ca4377140 con 0x556ca4036840 2024-09-21T06:38:26.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.179+0000 7f000effd640 1 -- 172.21.15.67:0/1115181429 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f00240abf30 con 0x556ca4036840 2024-09-21T06:38:26.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.179+0000 7f003c0b5740 10 client.15522 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:38:26.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.179+0000 7f003c0b5740 10 client.15522 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:38:26.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.179+0000 7f003c0b5740 20 client.15522 populate_metadata read hostname 'smithi067' 2024-09-21T06:38:26.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.179+0000 7f003c0b5740 1 --2- 172.21.15.67:0/1115181429 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x556ca4015ea0 0x556ca441ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:26.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.179+0000 7f003c0b5740 4 client.15522 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:38:26.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.179+0000 7f003c0b5740 1 -- 172.21.15.67:0/1115181429 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ca4372bf0 con 0x556ca4015ea0 2024-09-21T06:38:26.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.179+0000 7f002f7fe640 1 --2- 172.21.15.67:0/1115181429 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x556ca4015ea0 0x556ca441ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:26.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.180+0000 7f000effd640 10 client.15522 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:38:26.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.180+0000 7f002f7fe640 1 --2- 172.21.15.67:0/1115181429 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x556ca4015ea0 0x556ca441ae80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6fc8e48a1d30358 server_cookie=c8fdc407f4ae20f8 in_seq=0 out_seq=0 2024-09-21T06:38:26.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.180+0000 7f000effd640 1 -- 172.21.15.67:0/1115181429 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556ca4372bf0 con 0x556ca4015ea0 2024-09-21T06:38:26.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.180+0000 7f000effd640 10 client.15522 handle_command_reply: tid=0 2024-09-21T06:38:26.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.180+0000 7f003c0b5740 1 client.15522 shutdown 2024-09-21T06:38:26.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.180+0000 7f003c0b5740 1 -- 172.21.15.67:0/1115181429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556ca4036840 msgr2=0x556ca4409090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:26.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.180+0000 7f003c0b5740 1 --2- 172.21.15.67:0/1115181429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556ca4036840 0x556ca4409090 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f00240a9630 tx=0x7f00240057d0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.181+0000 7f003c0b5740 1 -- 172.21.15.67:0/1115181429 shutdown_connections 2024-09-21T06:38:26.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.181+0000 7f003c0b5740 1 --2- 172.21.15.67:0/1115181429 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x556ca4015ea0 0x556ca441ae80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.181+0000 7f003c0b5740 1 --2- 172.21.15.67:0/1115181429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca4057520 0x556ca44095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.181+0000 7f003c0b5740 1 --2- 172.21.15.67:0/1115181429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556ca4036840 0x556ca4409090 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.181+0000 7f003c0b5740 1 --2- 172.21.15.67:0/1115181429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ca402e660 0x556ca4408b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.181+0000 7f003c0b5740 1 -- 172.21.15.67:0/1115181429 >> 172.21.15.67:0/1115181429 conn(0x556ca404b910 msgr2=0x556ca44006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:38:26.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.181+0000 7f000effd640 0 client.15522 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:38:26.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.181+0000 7f003c0b5740 1 -- 172.21.15.67:0/1115181429 shutdown_connections 2024-09-21T06:38:26.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.181+0000 7f003c0b5740 1 -- 172.21.15.67:0/1115181429 wait complete. 2024-09-21T06:38:26.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.181+0000 7f003c0b5740 20 client.15522 trim_cache size 0 max 16384 2024-09-21T06:38:26.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.198+0000 7f003c0b5740 1 Processor -- start 2024-09-21T06:38:26.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.198+0000 7f003c0b5740 1 -- start start 2024-09-21T06:38:26.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.198+0000 7f003c0b5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ca402e660 0x556ca4405d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:26.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.198+0000 7f003c0b5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca4036840 0x556ca4406280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:26.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.198+0000 7f003c0b5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556ca4057520 0x556ca4420640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:26.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.198+0000 7f003c0b5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f00301aeab0 con 0x556ca4057520 2024-09-21T06:38:26.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.198+0000 7f003c0b5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f00301b1330 con 0x556ca402e660 2024-09-21T06:38:26.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.198+0000 7f003c0b5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f00301af120 con 0x556ca4036840 2024-09-21T06:38:26.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.198+0000 7f002ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556ca4057520 0x556ca4420640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:26.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.199+0000 7f002ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556ca4057520 0x556ca4420640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53352/0 (socket says 172.21.15.67:53352) 2024-09-21T06:38:26.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.199+0000 7f002ffff640 1 -- 172.21.15.67:0/1452182813 learned_addr learned my addr 172.21.15.67:0/1452182813 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:38:26.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.199+0000 7f002f7fe640 1 --2- 172.21.15.67:0/1452182813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ca402e660 0x556ca4405d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:26.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.199+0000 7f002effd640 1 --2- 172.21.15.67:0/1452182813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca4036840 0x556ca4406280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:26.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.199+0000 7f002ffff640 1 -- 172.21.15.67:0/1452182813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca4036840 msgr2=0x556ca4406280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:38:26.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.199+0000 7f002ffff640 1 --2- 172.21.15.67:0/1452182813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca4036840 0x556ca4406280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.199+0000 7f002ffff640 1 -- 172.21.15.67:0/1452182813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ca402e660 msgr2=0x556ca4405d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:38:26.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.199+0000 7f002ffff640 1 --2- 172.21.15.67:0/1452182813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ca402e660 0x556ca4405d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.199+0000 7f002ffff640 1 -- 172.21.15.67:0/1452182813 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ca4139960 con 0x556ca4057520 2024-09-21T06:38:26.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.199+0000 7f002ffff640 1 --2- 172.21.15.67:0/1452182813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556ca4057520 0x556ca4420640 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f00280128e0 tx=0x7f0028014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac820b5e6985ccc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:38:26.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.199+0000 7f00157fa640 1 -- 172.21.15.67:0/1452182813 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0028020bb0 con 0x556ca4057520 2024-09-21T06:38:26.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.199+0000 7f00157fa640 1 -- 172.21.15.67:0/1452182813 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0028020d50 con 0x556ca4057520 2024-09-21T06:38:26.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.199+0000 7f00157fa640 1 -- 172.21.15.67:0/1452182813 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00280275f0 con 0x556ca4057520 2024-09-21T06:38:26.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.200+0000 7f003c0b5740 1 -- 172.21.15.67:0/1452182813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556ca4057520 msgr2=0x556ca4420640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:26.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.200+0000 7f003c0b5740 1 --2- 172.21.15.67:0/1452182813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556ca4057520 0x556ca4420640 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f00280128e0 tx=0x7f0028014000 comp rx=0 tx=0).stop 2024-09-21T06:38:26.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.200+0000 7f003c0b5740 1 -- 172.21.15.67:0/1452182813 shutdown_connections 2024-09-21T06:38:26.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.200+0000 7f003c0b5740 1 --2- 172.21.15.67:0/1452182813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556ca4057520 0x556ca4420640 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.200+0000 7f003c0b5740 1 --2- 172.21.15.67:0/1452182813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca4036840 0x556ca4406280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.200+0000 7f003c0b5740 1 --2- 172.21.15.67:0/1452182813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ca402e660 0x556ca4405d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.200+0000 7f003c0b5740 1 -- 172.21.15.67:0/1452182813 >> 172.21.15.67:0/1452182813 conn(0x556ca404b910 msgr2=0x556ca440a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:38:26.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.200+0000 7f003c0b5740 1 -- 172.21.15.67:0/1452182813 shutdown_connections 2024-09-21T06:38:26.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.200+0000 7f003c0b5740 1 -- 172.21.15.67:0/1452182813 wait complete. 2024-09-21T06:38:26.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.201+0000 7f003c0b5740 1 -- start start 2024-09-21T06:38:26.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.201+0000 7f003c0b5740 1 Processor -- start 2024-09-21T06:38:26.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.201+0000 7f003c0b5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ca402e660 0x556ca440f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:26.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.201+0000 7f003c0b5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca4036840 0x556ca440f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:26.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.201+0000 7f003c0b5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556ca4057520 0x556ca4488c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:26.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.201+0000 7f003c0b5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f00301af7f0 con 0x556ca4057520 2024-09-21T06:38:26.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.201+0000 7f003c0b5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f00301b0c60 con 0x556ca402e660 2024-09-21T06:38:26.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.201+0000 7f003c0b5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f00301afec0 con 0x556ca4036840 2024-09-21T06:38:26.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.201+0000 7f003c0b5740 10 client.0 _ll_register_callbacks cb 0x7f002400c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:38:26.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.201+0000 7f002ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556ca4057520 0x556ca4488c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:26.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.201+0000 7f002ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556ca4057520 0x556ca4488c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53354/0 (socket says 172.21.15.67:53354) 2024-09-21T06:38:26.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.201+0000 7f002ffff640 1 -- 172.21.15.67:0/3799591685 learned_addr learned my addr 172.21.15.67:0/3799591685 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:38:26.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.201+0000 7f002f7fe640 1 --2- 172.21.15.67:0/3799591685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ca402e660 0x556ca440f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:26.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.201+0000 7f002effd640 1 --2- 172.21.15.67:0/3799591685 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca4036840 0x556ca440f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:26.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.202+0000 7f002ffff640 1 -- 172.21.15.67:0/3799591685 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca4036840 msgr2=0x556ca440f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:38:26.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.202+0000 7f002ffff640 1 --2- 172.21.15.67:0/3799591685 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca4036840 0x556ca440f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.202+0000 7f002ffff640 1 -- 172.21.15.67:0/3799591685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ca402e660 msgr2=0x556ca440f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:38:26.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.202+0000 7f002ffff640 1 --2- 172.21.15.67:0/3799591685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ca402e660 0x556ca440f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.202+0000 7f002ffff640 1 -- 172.21.15.67:0/3799591685 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ca44891a0 con 0x556ca4057520 2024-09-21T06:38:26.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.202+0000 7f002ffff640 1 --2- 172.21.15.67:0/3799591685 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556ca4057520 0x556ca4488c60 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f002801bda0 tx=0x7f0028020be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e56c0783991633fa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:38:26.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.202+0000 7f000effd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:38:26.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.202+0000 7f000effd640 1 -- 172.21.15.67:0/3799591685 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0028037020 con 0x556ca4057520 2024-09-21T06:38:26.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.202+0000 7f000effd640 1 -- 172.21.15.67:0/3799591685 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f002801b3a0 con 0x556ca4057520 2024-09-21T06:38:26.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.202+0000 7f003c0b5740 1 -- 172.21.15.67:0/3799591685 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0028020d50 con 0x556ca4057520 2024-09-21T06:38:26.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.202+0000 7f000effd640 1 -- 172.21.15.67:0/3799591685 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0028042050 con 0x556ca4057520 2024-09-21T06:38:26.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.202+0000 7f003c0b5740 10 client.15528 fetch_fsmap learned FSMap version 21 2024-09-21T06:38:26.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.202+0000 7f003c0b5740 1 -- 172.21.15.67:0/3799591685 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556ca448a330 con 0x556ca4057520 2024-09-21T06:38:26.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.203+0000 7f000effd640 1 -- 172.21.15.67:0/3799591685 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0028020d50 con 0x556ca4057520 2024-09-21T06:38:26.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.203+0000 7f000effd640 1 -- 172.21.15.67:0/3799591685 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f0028027070 con 0x556ca4057520 2024-09-21T06:38:26.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.203+0000 7f003c0b5740 10 client.15528 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:38:26.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.203+0000 7f003c0b5740 10 client.15528 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:38:26.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.203+0000 7f003c0b5740 20 client.15528 populate_metadata read hostname 'smithi067' 2024-09-21T06:38:26.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.203+0000 7f003c0b5740 1 --2- 172.21.15.67:0/3799591685 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x556ca3feb250 0x556ca448ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:26.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.203+0000 7f003c0b5740 4 client.15528 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4fe5f098", "release": true, "await": true}] 2024-09-21T06:38:26.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.203+0000 7f003c0b5740 1 -- 172.21.15.67:0/3799591685 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4fe5f098", "release": true, "await": true}) -- 0x556ca43fb670 con 0x556ca3feb250 2024-09-21T06:38:26.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.203+0000 7f002f7fe640 1 --2- 172.21.15.67:0/3799591685 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x556ca3feb250 0x556ca448ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:26.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.204+0000 7f002f7fe640 1 --2- 172.21.15.67:0/3799591685 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x556ca3feb250 0x556ca448ca30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c42420222ee60982 server_cookie=3bfda846246f78b7 in_seq=0 out_seq=0 2024-09-21T06:38:26.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.204+0000 7f000effd640 10 client.15528 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:38:26.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.207+0000 7f000effd640 1 -- 172.21.15.67:0/3799591685 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x556ca43fb670 con 0x556ca3feb250 2024-09-21T06:38:26.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.207+0000 7f000effd640 10 client.15528 handle_command_reply: tid=0 2024-09-21T06:38:26.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.207+0000 7f003c0b5740 1 client.15528 shutdown 2024-09-21T06:38:26.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.207+0000 7f003c0b5740 1 -- 172.21.15.67:0/3799591685 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556ca4057520 msgr2=0x556ca4488c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:26.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.207+0000 7f003c0b5740 1 --2- 172.21.15.67:0/3799591685 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556ca4057520 0x556ca4488c60 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f002801bda0 tx=0x7f0028020be0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.208+0000 7f003c0b5740 1 -- 172.21.15.67:0/3799591685 shutdown_connections 2024-09-21T06:38:26.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.208+0000 7f003c0b5740 1 --2- 172.21.15.67:0/3799591685 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x556ca3feb250 0x556ca448ca30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.208+0000 7f003c0b5740 1 --2- 172.21.15.67:0/3799591685 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556ca4057520 0x556ca4488c60 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.208+0000 7f003c0b5740 1 --2- 172.21.15.67:0/3799591685 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca4036840 0x556ca440f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.208+0000 7f003c0b5740 1 --2- 172.21.15.67:0/3799591685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ca402e660 0x556ca440f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.208+0000 7f003c0b5740 1 -- 172.21.15.67:0/3799591685 >> 172.21.15.67:0/3799591685 conn(0x556ca404b910 msgr2=0x556ca4413d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:38:26.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.208+0000 7f000effd640 0 client.15528 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:38:26.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.208+0000 7f003c0b5740 1 -- 172.21.15.67:0/3799591685 shutdown_connections 2024-09-21T06:38:26.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.208+0000 7f003c0b5740 1 -- 172.21.15.67:0/3799591685 wait complete. 2024-09-21T06:38:26.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.208+0000 7f003c0b5740 20 client.15528 trim_cache size 0 max 16384 2024-09-21T06:38:26.212 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-21T06:38:26.212 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 21, 2024-09-21T06:38:26.212 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-09-21T06:38:26.212 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 8, 2024-09-21T06:38:26.212 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-21T06:38:26.212 INFO:teuthology.orchestra.run.smithi067.stdout: "4fe5f098": { 2024-09-21T06:38:26.212 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 8, 2024-09-21T06:38:26.212 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-21T06:38:26.212 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:38:26.212 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-21T06:38:26.213 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:38:26.213 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:38:26.213 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-21T06:38:26.213 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 137.0, 2024-09-21T06:38:26.213 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-21T06:38:26.213 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-21T06:38:26.213 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-21T06:38:26.213 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:38:26.213 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-21T06:38:26.213 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:38:26.213 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:38:26.213 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:38:26.213 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:38:26.213 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:38:26.214 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:38:26.214 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-21T06:38:26.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.208+0000 7f0035b41640 1 -- 172.21.15.67:0/2494587104 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f00180b6540 msgr2=0x7f00180b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:26.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.209+0000 7f0035b41640 1 --2- 172.21.15.67:0/2494587104 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f00180b6540 0x7f00180b8a50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0028005fd0 tx=0x7f0028009580 comp rx=0 tx=0).stop 2024-09-21T06:38:26.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.209+0000 7f0035b41640 1 -- 172.21.15.67:0/2494587104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00301083a0 msgr2=0x7f003019a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:26.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.209+0000 7f0035b41640 1 --2- 172.21.15.67:0/2494587104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00301083a0 0x7f003019a8c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f002400c970 tx=0x7f002400ce30 comp rx=0 tx=0).stop 2024-09-21T06:38:26.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.209+0000 7f0035b41640 1 -- 172.21.15.67:0/2494587104 shutdown_connections 2024-09-21T06:38:26.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.209+0000 7f0035b41640 1 --2- 172.21.15.67:0/2494587104 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f00180b6540 0x7f00180b8a50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.209+0000 7f0035b41640 1 --2- 172.21.15.67:0/2494587104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0030108cf0 0x7f003019fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.209+0000 7f0035b41640 1 --2- 172.21.15.67:0/2494587104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00301083a0 0x7f003019a8c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.209+0000 7f0035b41640 1 --2- 172.21.15.67:0/2494587104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0030102a60 0x7f003019a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:26.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.209+0000 7f0035b41640 1 -- 172.21.15.67:0/2494587104 >> 172.21.15.67:0/2494587104 conn(0x7f0030079730 msgr2=0x7f0030101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:38:26.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.209+0000 7f0035b41640 1 -- 172.21.15.67:0/2494587104 shutdown_connections 2024-09-21T06:38:26.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:26.209+0000 7f0035b41640 1 -- 172.21.15.67:0/2494587104 wait complete. 2024-09-21T06:38:26.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:26 smithi067 ceph-mon[31232]: pgmap v416: 129 pgs: 129 active+clean; 243 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 545 KiB/s wr, 3 op/s 2024-09-21T06:38:26.938 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4fe5f098', total seconds elapsed: 20.3 2024-09-21T06:38:26.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:26 smithi203 ceph-mon[33735]: pgmap v416: 129 pgs: 129 active+clean; 243 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 545 KiB/s wr, 3 op/s 2024-09-21T06:38:27.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:26 smithi100 ceph-mon[33401]: pgmap v416: 129 pgs: 129 active+clean; 243 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 545 KiB/s wr, 3 op/s 2024-09-21T06:38:29.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:28 smithi203 ceph-mon[33735]: pgmap v417: 129 pgs: 129 active+clean; 293 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 4.6 MiB/s wr, 4 op/s 2024-09-21T06:38:29.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:28 smithi067 ceph-mon[31232]: pgmap v417: 129 pgs: 129 active+clean; 293 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 4.6 MiB/s wr, 4 op/s 2024-09-21T06:38:29.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:28 smithi100 ceph-mon[33401]: pgmap v417: 129 pgs: 129 active+clean; 293 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 4.6 MiB/s wr, 4 op/s 2024-09-21T06:38:31.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:30 smithi203 ceph-mon[33735]: pgmap v418: 129 pgs: 129 active+clean; 332 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 7.4 MiB/s wr, 5 op/s 2024-09-21T06:38:31.291 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:30 smithi067 ceph-mon[31232]: pgmap v418: 129 pgs: 129 active+clean; 332 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 7.4 MiB/s wr, 5 op/s 2024-09-21T06:38:31.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:30 smithi100 ceph-mon[33401]: pgmap v418: 129 pgs: 129 active+clean; 332 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 7.4 MiB/s wr, 5 op/s 2024-09-21T06:38:33.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:32 smithi203 ceph-mon[33735]: pgmap v419: 129 pgs: 129 active+clean; 886 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 54 MiB/s wr, 31 op/s 2024-09-21T06:38:33.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:32 smithi067 ceph-mon[31232]: pgmap v419: 129 pgs: 129 active+clean; 886 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 54 MiB/s wr, 31 op/s 2024-09-21T06:38:33.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:32 smithi100 ceph-mon[33401]: pgmap v419: 129 pgs: 129 active+clean; 886 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 54 MiB/s wr, 31 op/s 2024-09-21T06:38:35.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:34 smithi203 ceph-mon[33735]: pgmap v420: 129 pgs: 129 active+clean; 1.0 GiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 69 MiB/s wr, 38 op/s 2024-09-21T06:38:35.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:34 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:38:35.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:34 smithi067 ceph-mon[31232]: pgmap v420: 129 pgs: 129 active+clean; 1.0 GiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 69 MiB/s wr, 38 op/s 2024-09-21T06:38:35.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:34 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:38:35.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:34 smithi100 ceph-mon[33401]: pgmap v420: 129 pgs: 129 active+clean; 1.0 GiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 69 MiB/s wr, 38 op/s 2024-09-21T06:38:35.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:34 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:38:36.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:35 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:38:36.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:35 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:38:36.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:35 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:38:36.294 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:38:36.294 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:38:36.294 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:38:36.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:35 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:38:36.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:35 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:38:36.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:35 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:38:37.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:37 smithi203 ceph-mon[33735]: pgmap v421: 129 pgs: 129 active+clean; 1.2 GiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 81 MiB/s wr, 45 op/s 2024-09-21T06:38:37.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:37 smithi067 ceph-mon[31232]: pgmap v421: 129 pgs: 129 active+clean; 1.2 GiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 81 MiB/s wr, 45 op/s 2024-09-21T06:38:37.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:37 smithi100 ceph-mon[33401]: pgmap v421: 129 pgs: 129 active+clean; 1.2 GiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 81 MiB/s wr, 45 op/s 2024-09-21T06:38:39.507 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:39 smithi067 ceph-mon[31232]: pgmap v422: 129 pgs: 129 active+clean; 1.7 GiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 125 MiB/s wr, 67 op/s 2024-09-21T06:38:39.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:39 smithi203 ceph-mon[33735]: pgmap v422: 129 pgs: 129 active+clean; 1.7 GiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 125 MiB/s wr, 67 op/s 2024-09-21T06:38:39.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:39 smithi100 ceph-mon[33401]: pgmap v422: 129 pgs: 129 active+clean; 1.7 GiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 125 MiB/s wr, 67 op/s 2024-09-21T06:38:41.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:41 smithi203 ceph-mon[33735]: pgmap v423: 129 pgs: 129 active+clean; 1.8 GiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 127 MiB/s wr, 68 op/s 2024-09-21T06:38:41.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:41 smithi067 ceph-mon[31232]: pgmap v423: 129 pgs: 129 active+clean; 1.8 GiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 127 MiB/s wr, 68 op/s 2024-09-21T06:38:41.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:41 smithi100 ceph-mon[33401]: pgmap v423: 129 pgs: 129 active+clean; 1.8 GiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 127 MiB/s wr, 68 op/s 2024-09-21T06:38:43.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:43 smithi203 ceph-mon[33735]: pgmap v424: 129 pgs: 129 active+clean; 2.3 GiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 168 MiB/s wr, 90 op/s 2024-09-21T06:38:43.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:43 smithi067 ceph-mon[31232]: pgmap v424: 129 pgs: 129 active+clean; 2.3 GiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 168 MiB/s wr, 90 op/s 2024-09-21T06:38:43.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:43 smithi100 ceph-mon[33401]: pgmap v424: 129 pgs: 129 active+clean; 2.3 GiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 168 MiB/s wr, 90 op/s 2024-09-21T06:38:45.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:45 smithi203 ceph-mon[33735]: pgmap v425: 129 pgs: 129 active+clean; 2.5 GiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 142 MiB/s wr, 74 op/s 2024-09-21T06:38:45.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:45 smithi067 ceph-mon[31232]: pgmap v425: 129 pgs: 129 active+clean; 2.5 GiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 142 MiB/s wr, 74 op/s 2024-09-21T06:38:45.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:45 smithi100 ceph-mon[33401]: pgmap v425: 129 pgs: 129 active+clean; 2.5 GiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 142 MiB/s wr, 74 op/s 2024-09-21T06:38:47.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:47 smithi203 ceph-mon[33735]: pgmap v426: 129 pgs: 129 active+clean; 2.6 GiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 137 MiB/s wr, 73 op/s 2024-09-21T06:38:47.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:47 smithi067 ceph-mon[31232]: pgmap v426: 129 pgs: 129 active+clean; 2.6 GiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 137 MiB/s wr, 73 op/s 2024-09-21T06:38:47.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:47 smithi100 ceph-mon[33401]: pgmap v426: 129 pgs: 129 active+clean; 2.6 GiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 137 MiB/s wr, 73 op/s 2024-09-21T06:38:49.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:49 smithi203 ceph-mon[33735]: pgmap v427: 129 pgs: 129 active+clean; 3.2 GiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 173 MiB/s wr, 90 op/s 2024-09-21T06:38:49.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:49 smithi067 ceph-mon[31232]: pgmap v427: 129 pgs: 129 active+clean; 3.2 GiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 173 MiB/s wr, 90 op/s 2024-09-21T06:38:49.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:49 smithi100 ceph-mon[33401]: pgmap v427: 129 pgs: 129 active+clean; 3.2 GiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 173 MiB/s wr, 90 op/s 2024-09-21T06:38:51.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:51 smithi203 ceph-mon[33735]: pgmap v428: 129 pgs: 129 active+clean; 3.3 GiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 137 MiB/s wr, 71 op/s 2024-09-21T06:38:51.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:51 smithi067 ceph-mon[31232]: pgmap v428: 129 pgs: 129 active+clean; 3.3 GiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 137 MiB/s wr, 71 op/s 2024-09-21T06:38:51.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:51 smithi100 ceph-mon[33401]: pgmap v428: 129 pgs: 129 active+clean; 3.3 GiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 137 MiB/s wr, 71 op/s 2024-09-21T06:38:53.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:53 smithi203 ceph-mon[33735]: pgmap v429: 129 pgs: 129 active+clean; 3.8 GiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 171 MiB/s wr, 90 op/s 2024-09-21T06:38:53.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:53 smithi067 ceph-mon[31232]: pgmap v429: 129 pgs: 129 active+clean; 3.8 GiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 171 MiB/s wr, 90 op/s 2024-09-21T06:38:53.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:53 smithi100 ceph-mon[33401]: pgmap v429: 129 pgs: 129 active+clean; 3.8 GiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 171 MiB/s wr, 90 op/s 2024-09-21T06:38:55.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:55 smithi203 ceph-mon[33735]: pgmap v430: 129 pgs: 129 active+clean; 4.0 GiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 144 MiB/s wr, 74 op/s 2024-09-21T06:38:55.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:55 smithi067 ceph-mon[31232]: pgmap v430: 129 pgs: 129 active+clean; 4.0 GiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 144 MiB/s wr, 74 op/s 2024-09-21T06:38:55.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:55 smithi100 ceph-mon[33401]: pgmap v430: 129 pgs: 129 active+clean; 4.0 GiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 144 MiB/s wr, 74 op/s 2024-09-21T06:38:57.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:57 smithi203 ceph-mon[33735]: pgmap v431: 129 pgs: 129 active+clean; 4.2 GiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 140 MiB/s wr, 74 op/s 2024-09-21T06:38:57.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:57 smithi067 ceph-mon[31232]: pgmap v431: 129 pgs: 129 active+clean; 4.2 GiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 140 MiB/s wr, 74 op/s 2024-09-21T06:38:57.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:57 smithi100 ceph-mon[33401]: pgmap v431: 129 pgs: 129 active+clean; 4.2 GiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 140 MiB/s wr, 74 op/s 2024-09-21T06:38:58.510 INFO:tasks.quiescer.fs.[cephfs]:Total duration (55.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-21T06:38:58.511 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.8 2024-09-21T06:38:58.511 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:38:58.898 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:38:59.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.476+0000 7f5bc864d640 1 -- 172.21.15.67:0/2888420500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5bc0105190 msgr2=0x7f5bc01055a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:59.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.476+0000 7f5bc864d640 1 --2- 172.21.15.67:0/2888420500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5bc0105190 0x7f5bc01055a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0009860 tx=0x7f5bb002a340 comp rx=0 tx=0).stop 2024-09-21T06:38:59.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.476+0000 7f5bc864d640 1 -- 172.21.15.67:0/2888420500 shutdown_connections 2024-09-21T06:38:59.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.476+0000 7f5bc864d640 1 --2- 172.21.15.67:0/2888420500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bc0105ae0 0x7f5bc010f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:59.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.476+0000 7f5bc864d640 1 --2- 172.21.15.67:0/2888420500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5bc0105190 0x7f5bc01055a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:59.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.476+0000 7f5bc864d640 1 --2- 172.21.15.67:0/2888420500 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bc01031c0 0x7f5bc01035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:59.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.476+0000 7f5bc864d640 1 -- 172.21.15.67:0/2888420500 >> 172.21.15.67:0/2888420500 conn(0x7f5bc0077c60 msgr2=0x7f5bc00780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:38:59.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.477+0000 7f5bc864d640 1 -- 172.21.15.67:0/2888420500 shutdown_connections 2024-09-21T06:38:59.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.477+0000 7f5bc864d640 1 -- 172.21.15.67:0/2888420500 wait complete. 2024-09-21T06:38:59.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.477+0000 7f5bc864d640 1 Processor -- start 2024-09-21T06:38:59.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.478+0000 7f5bc864d640 1 -- start start 2024-09-21T06:38:59.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.478+0000 7f5bc864d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5bc01031c0 0x7f5bc0199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:59.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.478+0000 7f5bc864d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bc0105190 0x7f5bc0199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:59.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.478+0000 7f5bc864d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bc0105ae0 0x7f5bc019eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:59.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.478+0000 7f5bc864d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc0112830 con 0x7f5bc01031c0 2024-09-21T06:38:59.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.478+0000 7f5bc864d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc01126b0 con 0x7f5bc0105ae0 2024-09-21T06:38:59.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.478+0000 7f5bc864d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc01129b0 con 0x7f5bc0105190 2024-09-21T06:38:59.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.478+0000 7f5bc63e9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5bc01031c0 0x7f5bc0199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:59.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.478+0000 7f5bc6bea640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bc0105ae0 0x7f5bc019eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:59.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.478+0000 7f5bc63e9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5bc01031c0 0x7f5bc0199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49704/0 (socket says 172.21.15.67:49704) 2024-09-21T06:38:59.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.478+0000 7f5bc6bea640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bc0105ae0 0x7f5bc019eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.67:51976/0 (socket says 172.21.15.67:51976) 2024-09-21T06:38:59.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.479+0000 7f5bc63e9640 1 -- 172.21.15.67:0/436280939 learned_addr learned my addr 172.21.15.67:0/436280939 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:38:59.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.479+0000 7f5bc5be8640 1 --2- 172.21.15.67:0/436280939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bc0105190 0x7f5bc0199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:59.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.479+0000 7f5bc5be8640 1 -- 172.21.15.67:0/436280939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bc0105ae0 msgr2=0x7f5bc019eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:59.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.479+0000 7f5bc5be8640 1 --2- 172.21.15.67:0/436280939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bc0105ae0 0x7f5bc019eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:59.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.479+0000 7f5bc5be8640 1 -- 172.21.15.67:0/436280939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5bc01031c0 msgr2=0x7f5bc0199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:59.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.479+0000 7f5bc5be8640 1 --2- 172.21.15.67:0/436280939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5bc01031c0 0x7f5bc0199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:59.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.479+0000 7f5bc5be8640 1 -- 172.21.15.67:0/436280939 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc019f1c0 con 0x7f5bc0105190 2024-09-21T06:38:59.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.479+0000 7f5bc63e9640 1 --2- 172.21.15.67:0/436280939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5bc01031c0 0x7f5bc0199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:38:59.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.479+0000 7f5bc6bea640 1 --2- 172.21.15.67:0/436280939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bc0105ae0 0x7f5bc019eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:38:59.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.480+0000 7f5bc5be8640 1 --2- 172.21.15.67:0/436280939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bc0105190 0x7f5bc0199820 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5bb002a860 tx=0x7f5bb0030d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:38:59.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.480+0000 7f5bb77fe640 1 -- 172.21.15.67:0/436280939 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0040020 con 0x7f5bc0105190 2024-09-21T06:38:59.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.480+0000 7f5bc864d640 1 -- 172.21.15.67:0/436280939 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bc019f450 con 0x7f5bc0105190 2024-09-21T06:38:59.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.480+0000 7f5bc864d640 1 -- 172.21.15.67:0/436280939 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bc0101e20 con 0x7f5bc0105190 2024-09-21T06:38:59.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.480+0000 7f5bb77fe640 1 -- 172.21.15.67:0/436280939 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bb002adc0 con 0x7f5bc0105190 2024-09-21T06:38:59.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.480+0000 7f5bb77fe640 1 -- 172.21.15.67:0/436280939 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0037040 con 0x7f5bc0105190 2024-09-21T06:38:59.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.481+0000 7f5bc864d640 1 -- 172.21.15.67:0/436280939 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ba40052c0 con 0x7f5bc0105190 2024-09-21T06:38:59.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.484+0000 7f5bb77fe640 1 -- 172.21.15.67:0/436280939 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f5bb0043020 con 0x7f5bc0105190 2024-09-21T06:38:59.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.485+0000 7f5bb77fe640 1 --2- 172.21.15.67:0/436280939 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5bac0b67c0 0x7f5bac0b8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:38:59.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.486+0000 7f5bb77fe640 1 -- 172.21.15.67:0/436280939 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f5bb0004240 con 0x7f5bc0105190 2024-09-21T06:38:59.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.486+0000 7f5bc63e9640 1 --2- 172.21.15.67:0/436280939 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5bac0b67c0 0x7f5bac0b8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:38:59.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.486+0000 7f5bc63e9640 1 --2- 172.21.15.67:0/436280939 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5bac0b67c0 0x7f5bac0b8cd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0104e0 tx=0x7f5bbc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:38:59.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.487+0000 7f5bb77fe640 1 -- 172.21.15.67:0/436280939 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5bb004a020 con 0x7f5bc0105190 2024-09-21T06:38:59.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:38:59 smithi203 ceph-mon[33735]: pgmap v432: 129 pgs: 129 active+clean; 4.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 174 MiB/s wr, 90 op/s 2024-09-21T06:38:59.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:38:59 smithi067 ceph-mon[31232]: pgmap v432: 129 pgs: 129 active+clean; 4.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 174 MiB/s wr, 90 op/s 2024-09-21T06:38:59.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:38:59 smithi100 ceph-mon[33401]: pgmap v432: 129 pgs: 129 active+clean; 4.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 174 MiB/s wr, 90 op/s 2024-09-21T06:38:59.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.875+0000 7f5bc864d640 1 -- 172.21.15.67:0/436280939 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ba40055b0 con 0x7f5bc0105190 2024-09-21T06:38:59.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.876+0000 7f5bb77fe640 1 -- 172.21.15.67:0/436280939 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7f5bb00cda60 con 0x7f5bc0105190 2024-09-21T06:38:59.879 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:38:59.879 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:38:59.879 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:38:59.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.880+0000 7f5bc864d640 1 -- 172.21.15.67:0/436280939 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5bac0b67c0 msgr2=0x7f5bac0b8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:59.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.880+0000 7f5bc864d640 1 --2- 172.21.15.67:0/436280939 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5bac0b67c0 0x7f5bac0b8cd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0104e0 tx=0x7f5bbc00b410 comp rx=0 tx=0).stop 2024-09-21T06:38:59.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.880+0000 7f5bc864d640 1 -- 172.21.15.67:0/436280939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bc0105190 msgr2=0x7f5bc0199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:38:59.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.880+0000 7f5bc864d640 1 --2- 172.21.15.67:0/436280939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bc0105190 0x7f5bc0199820 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5bb002a860 tx=0x7f5bb0030d60 comp rx=0 tx=0).stop 2024-09-21T06:38:59.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.880+0000 7f5bc864d640 1 -- 172.21.15.67:0/436280939 shutdown_connections 2024-09-21T06:38:59.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.880+0000 7f5bc864d640 1 --2- 172.21.15.67:0/436280939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bc0105ae0 0x7f5bc019eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:59.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.880+0000 7f5bc864d640 1 --2- 172.21.15.67:0/436280939 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5bac0b67c0 0x7f5bac0b8cd0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:59.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.880+0000 7f5bc864d640 1 --2- 172.21.15.67:0/436280939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bc0105190 0x7f5bc0199820 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:59.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.880+0000 7f5bc864d640 1 --2- 172.21.15.67:0/436280939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5bc01031c0 0x7f5bc0199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:38:59.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.880+0000 7f5bc864d640 1 -- 172.21.15.67:0/436280939 >> 172.21.15.67:0/436280939 conn(0x7f5bc0077c60 msgr2=0x7f5bc0106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:38:59.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.880+0000 7f5bc864d640 1 -- 172.21.15.67:0/436280939 shutdown_connections 2024-09-21T06:38:59.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:38:59.880+0000 7f5bc864d640 1 -- 172.21.15.67:0/436280939 wait complete. 2024-09-21T06:39:00.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:00 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/436280939' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:39:00.682 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0' 2024-09-21T06:39:00.682 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24671 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0 2024-09-21T06:39:00.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:00 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/436280939' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:39:00.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:00 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/436280939' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:39:01.078 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:39:01.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:01 smithi067 ceph-mon[31232]: pgmap v433: 129 pgs: 129 active+clean; 4.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 134 MiB/s wr, 69 op/s 2024-09-21T06:39:01.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:01 smithi203 ceph-mon[33735]: pgmap v433: 129 pgs: 129 active+clean; 4.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 134 MiB/s wr, 69 op/s 2024-09-21T06:39:01.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:01 smithi100 ceph-mon[33401]: pgmap v433: 129 pgs: 129 active+clean; 4.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 134 MiB/s wr, 69 op/s 2024-09-21T06:39:02.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.996+0000 7fc14492c640 1 -- 172.21.15.67:0/1298127700 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1401083e0 msgr2=0x7fc1401087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:02.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.996+0000 7fc14492c640 1 --2- 172.21.15.67:0/1298127700 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1401083e0 0x7fc1401087f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc134009860 tx=0x7fc13402a340 comp rx=0 tx=0).stop 2024-09-21T06:39:02.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.996+0000 7fc14492c640 1 -- 172.21.15.67:0/1298127700 shutdown_connections 2024-09-21T06:39:02.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.996+0000 7fc14492c640 1 --2- 172.21.15.67:0/1298127700 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc140108d30 0x7fc14010f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.996+0000 7fc14492c640 1 --2- 172.21.15.67:0/1298127700 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1401083e0 0x7fc1401087f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.996+0000 7fc14492c640 1 --2- 172.21.15.67:0/1298127700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc140102af0 0x7fc140102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.996+0000 7fc14492c640 1 -- 172.21.15.67:0/1298127700 >> 172.21.15.67:0/1298127700 conn(0x7fc140079830 msgr2=0x7fc140079c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:39:02.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.996+0000 7fc14492c640 1 -- 172.21.15.67:0/1298127700 shutdown_connections 2024-09-21T06:39:02.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.996+0000 7fc14492c640 1 -- 172.21.15.67:0/1298127700 wait complete. 2024-09-21T06:39:02.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.997+0000 7fc14492c640 1 Processor -- start 2024-09-21T06:39:02.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.997+0000 7fc14492c640 1 -- start start 2024-09-21T06:39:02.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.998+0000 7fc14492c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc140102af0 0x7fc14019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:02.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.998+0000 7fc14492c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1401083e0 0x7fc14019a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:02.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.998+0000 7fc14492c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc140108d30 0x7fc14019fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:02.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.998+0000 7fc14492c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc140113990 con 0x7fc1401083e0 2024-09-21T06:39:02.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.998+0000 7fc14492c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc140113810 con 0x7fc140102af0 2024-09-21T06:39:02.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.998+0000 7fc14492c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc140113b10 con 0x7fc140108d30 2024-09-21T06:39:02.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.998+0000 7fc13e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc140102af0 0x7fc14019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:02.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.998+0000 7fc13dd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1401083e0 0x7fc14019a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:02.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.998+0000 7fc13dd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1401083e0 0x7fc14019a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49722/0 (socket says 172.21.15.67:49722) 2024-09-21T06:39:02.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.998+0000 7fc13dd9b640 1 -- 172.21.15.67:0/1593393076 learned_addr learned my addr 172.21.15.67:0/1593393076 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:39:02.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.998+0000 7fc13e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc140102af0 0x7fc14019a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.67:51992/0 (socket says 172.21.15.67:51992) 2024-09-21T06:39:02.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.998+0000 7fc13ed9d640 1 --2- 172.21.15.67:0/1593393076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc140108d30 0x7fc14019fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:02.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.999+0000 7fc13dd9b640 1 -- 172.21.15.67:0/1593393076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc140108d30 msgr2=0x7fc14019fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:02.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.999+0000 7fc13dd9b640 1 --2- 172.21.15.67:0/1593393076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc140108d30 0x7fc14019fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.999+0000 7fc13dd9b640 1 -- 172.21.15.67:0/1593393076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc140102af0 msgr2=0x7fc14019a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:02.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.999+0000 7fc13dd9b640 1 --2- 172.21.15.67:0/1593393076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc140102af0 0x7fc14019a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.999+0000 7fc13dd9b640 1 -- 172.21.15.67:0/1593393076 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1401a0350 con 0x7fc1401083e0 2024-09-21T06:39:02.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.999+0000 7fc13e59c640 1 --2- 172.21.15.67:0/1593393076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc140102af0 0x7fc14019a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:39:02.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.999+0000 7fc13dd9b640 1 --2- 172.21.15.67:0/1593393076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1401083e0 0x7fc14019a980 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc134005940 tx=0x7fc13402ae40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:39:02.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.999+0000 7fc12f7fe640 1 -- 172.21.15.67:0/1593393076 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc134040020 con 0x7fc1401083e0 2024-09-21T06:39:02.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.999+0000 7fc14492c640 1 -- 172.21.15.67:0/1593393076 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1401a05e0 con 0x7fc1401083e0 2024-09-21T06:39:02.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.999+0000 7fc14492c640 1 -- 172.21.15.67:0/1593393076 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1401a6da0 con 0x7fc1401083e0 2024-09-21T06:39:02.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.999+0000 7fc12f7fe640 1 -- 172.21.15.67:0/1593393076 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc1340044d0 con 0x7fc1401083e0 2024-09-21T06:39:02.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:01.999+0000 7fc12f7fe640 1 -- 172.21.15.67:0/1593393076 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc134037040 con 0x7fc1401083e0 2024-09-21T06:39:02.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.003+0000 7fc14aea0740 1 Processor -- start 2024-09-21T06:39:02.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.003+0000 7fc14aea0740 1 -- start start 2024-09-21T06:39:02.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.004+0000 7fc14aea0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563454119840 0x5634544c0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:02.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.004+0000 7fc14aea0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634540f0980 0x5634544c0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:02.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.004+0000 7fc14aea0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634540f8b60 0x5634544c1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:02.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.004+0000 7fc14aea0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x563454436b90 con 0x5634540f0980 2024-09-21T06:39:02.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.004+0000 7fc14aea0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x563454425fc0 con 0x5634540f8b60 2024-09-21T06:39:02.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.004+0000 7fc14aea0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563454429dd0 con 0x563454119840 2024-09-21T06:39:02.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.004+0000 7fc13ed9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563454119840 0x5634544c0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:02.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.004+0000 7fc13ed9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563454119840 0x5634544c0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.67:57368/0 (socket says 172.21.15.67:57368) 2024-09-21T06:39:02.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.004+0000 7fc13ed9d640 1 -- 172.21.15.67:0/1301657015 learned_addr learned my addr 172.21.15.67:0/1301657015 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:39:02.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.005+0000 7fc13e59c640 1 --2- 172.21.15.67:0/1301657015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634540f0980 0x5634544c0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:02.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.005+0000 7fc13ed9d640 1 -- 172.21.15.67:0/1301657015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634540f8b60 msgr2=0x5634544c1080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:39:02.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.005+0000 7fc13ed9d640 1 --2- 172.21.15.67:0/1301657015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634540f8b60 0x5634544c1080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.005+0000 7fc13ed9d640 1 -- 172.21.15.67:0/1301657015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634540f0980 msgr2=0x5634544c0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:39:02.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.005+0000 7fc13ed9d640 1 --2- 172.21.15.67:0/1301657015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634540f0980 0x5634544c0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.005+0000 7fc13ed9d640 1 -- 172.21.15.67:0/1301657015 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634543ad060 con 0x563454119840 2024-09-21T06:39:02.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.005+0000 7fc12f7fe640 1 -- 172.21.15.67:0/1593393076 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fc134043020 con 0x7fc1401083e0 2024-09-21T06:39:02.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.005+0000 7fc13ed9d640 1 --2- 172.21.15.67:0/1301657015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563454119840 0x5634544c0600 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc138009f40 tx=0x7fc13800f890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c68c29ae1ddb7272 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:39:02.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.005+0000 7fc12cff9640 1 -- 172.21.15.67:0/1301657015 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1380168e0 con 0x563454119840 2024-09-21T06:39:02.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.005+0000 7fc12cff9640 1 -- 172.21.15.67:0/1301657015 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc13801a070 con 0x563454119840 2024-09-21T06:39:02.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.005+0000 7fc12cff9640 1 -- 172.21.15.67:0/1301657015 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13800d3e0 con 0x563454119840 2024-09-21T06:39:02.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.006+0000 7fc12f7fe640 1 --2- 172.21.15.67:0/1593393076 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc1280b6590 0x7fc1280b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:02.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.006+0000 7fc13e59c640 1 --2- 172.21.15.67:0/1593393076 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc1280b6590 0x7fc1280b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:02.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.006+0000 7fc14aea0740 1 -- 172.21.15.67:0/1301657015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563454119840 msgr2=0x5634544c0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:02.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.006+0000 7fc14aea0740 1 --2- 172.21.15.67:0/1301657015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563454119840 0x5634544c0600 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc138009f40 tx=0x7fc13800f890 comp rx=0 tx=0).stop 2024-09-21T06:39:02.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.006+0000 7fc12f7fe640 1 -- 172.21.15.67:0/1593393076 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fc1341075f0 con 0x7fc1401083e0 2024-09-21T06:39:02.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.007+0000 7fc14aea0740 1 -- 172.21.15.67:0/1301657015 shutdown_connections 2024-09-21T06:39:02.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.007+0000 7fc14aea0740 1 --2- 172.21.15.67:0/1301657015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634540f8b60 0x5634544c1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.007+0000 7fc14aea0740 1 --2- 172.21.15.67:0/1301657015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634540f0980 0x5634544c0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.007+0000 7fc14aea0740 1 --2- 172.21.15.67:0/1301657015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563454119840 0x5634544c0600 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.007+0000 7fc14aea0740 1 -- 172.21.15.67:0/1301657015 >> 172.21.15.67:0/1301657015 conn(0x56345410d990 msgr2=0x5634544be4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:39:02.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.007+0000 7fc14aea0740 1 -- 172.21.15.67:0/1301657015 shutdown_connections 2024-09-21T06:39:02.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.007+0000 7fc14aea0740 1 -- 172.21.15.67:0/1301657015 wait complete. 2024-09-21T06:39:02.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.007+0000 7fc13e59c640 1 --2- 172.21.15.67:0/1593393076 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc1280b6590 0x7fc1280b8aa0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc1300095f0 tx=0x7fc130005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:39:02.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.007+0000 7fc14aea0740 1 -- start start 2024-09-21T06:39:02.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.007+0000 7fc14aea0740 1 Processor -- start 2024-09-21T06:39:02.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.008+0000 7fc14aea0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634540f0980 0x5634544d5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:02.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.008+0000 7fc14aea0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634540f8b60 0x5634544d5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:02.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.008+0000 7fc14aea0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563454119840 0x5634544da740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:02.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.008+0000 7fc14aea0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5634542877a0 con 0x5634540f8b60 2024-09-21T06:39:02.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.008+0000 7fc14aea0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5634543cb0a0 con 0x5634540f0980 2024-09-21T06:39:02.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.008+0000 7fc14aea0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5634544cede0 con 0x563454119840 2024-09-21T06:39:02.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.008+0000 7fc14aea0740 10 client.0 _ll_register_callbacks cb 0x5634543a83c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:39:02.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.008+0000 7fc13e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634540f0980 0x5634544d5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:02.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.008+0000 7fc13dd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634540f8b60 0x5634544d5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:02.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.008+0000 7fc13dd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634540f8b60 0x5634544d5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49732/0 (socket says 172.21.15.67:49732) 2024-09-21T06:39:02.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.008+0000 7fc13dd9b640 1 -- 172.21.15.67:0/522101487 learned_addr learned my addr 172.21.15.67:0/522101487 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:39:02.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.008+0000 7fc13dd9b640 1 -- 172.21.15.67:0/522101487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563454119840 msgr2=0x5634544da740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:39:02.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.008+0000 7fc13dd9b640 1 --2- 172.21.15.67:0/522101487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563454119840 0x5634544da740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.009+0000 7fc13dd9b640 1 -- 172.21.15.67:0/522101487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634540f0980 msgr2=0x5634544d5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:39:02.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.009+0000 7fc13dd9b640 1 --2- 172.21.15.67:0/522101487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634540f0980 0x5634544d5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.009+0000 7fc13dd9b640 1 -- 172.21.15.67:0/522101487 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634543b9880 con 0x5634540f8b60 2024-09-21T06:39:02.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.009+0000 7fc13e59c640 1 --2- 172.21.15.67:0/522101487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634540f0980 0x5634544d5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:39:02.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.009+0000 7fc13dd9b640 1 --2- 172.21.15.67:0/522101487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634540f8b60 0x5634544d5720 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc1340cd2e0 tx=0x7fc1340305c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ae182f78b51043 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:39:02.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.009+0000 7fc125ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:39:02.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.009+0000 7fc125ffb640 1 -- 172.21.15.67:0/522101487 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1340b4040 con 0x5634540f8b60 2024-09-21T06:39:02.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.009+0000 7fc14aea0740 1 -- 172.21.15.67:0/522101487 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc13801a070 con 0x5634540f8b60 2024-09-21T06:39:02.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.009+0000 7fc125ffb640 1 -- 172.21.15.67:0/522101487 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fc134004040 con 0x5634540f8b60 2024-09-21T06:39:02.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.010+0000 7fc125ffb640 1 -- 172.21.15.67:0/522101487 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc1340b4040 con 0x5634540f8b60 2024-09-21T06:39:02.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.010+0000 7fc14aea0740 10 client.15543 fetch_fsmap learned FSMap version 21 2024-09-21T06:39:02.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.010+0000 7fc14aea0740 1 -- 172.21.15.67:0/522101487 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563454193b90 con 0x5634540f8b60 2024-09-21T06:39:02.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.010+0000 7fc125ffb640 1 -- 172.21.15.67:0/522101487 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc13801a070 con 0x5634540f8b60 2024-09-21T06:39:02.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.010+0000 7fc125ffb640 1 -- 172.21.15.67:0/522101487 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc1340c8ca0 con 0x5634540f8b60 2024-09-21T06:39:02.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.010+0000 7fc14aea0740 10 client.15543 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:39:02.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.010+0000 7fc14aea0740 10 client.15543 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:39:02.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.010+0000 7fc14aea0740 20 client.15543 populate_metadata read hostname 'smithi067' 2024-09-21T06:39:02.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.010+0000 7fc14aea0740 1 --2- 172.21.15.67:0/522101487 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x5634540d81c0 0x5634544e69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:02.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.010+0000 7fc14aea0740 4 client.15543 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:39:02.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.010+0000 7fc14aea0740 1 -- 172.21.15.67:0/522101487 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563454438ae0 con 0x5634540d81c0 2024-09-21T06:39:02.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.011+0000 7fc13e59c640 1 --2- 172.21.15.67:0/522101487 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x5634540d81c0 0x5634544e69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:02.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.011+0000 7fc125ffb640 10 client.15543 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:39:02.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.011+0000 7fc13e59c640 1 --2- 172.21.15.67:0/522101487 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x5634540d81c0 0x5634544e69d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96fe1c4fadf3858d server_cookie=8f848275cb1276de in_seq=0 out_seq=0 2024-09-21T06:39:02.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.012+0000 7fc125ffb640 1 -- 172.21.15.67:0/522101487 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563454438ae0 con 0x5634540d81c0 2024-09-21T06:39:02.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.012+0000 7fc125ffb640 10 client.15543 handle_command_reply: tid=0 2024-09-21T06:39:02.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.012+0000 7fc14aea0740 1 client.15543 shutdown 2024-09-21T06:39:02.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.012+0000 7fc14aea0740 1 -- 172.21.15.67:0/522101487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634540f8b60 msgr2=0x5634544d5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:02.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.012+0000 7fc14aea0740 1 --2- 172.21.15.67:0/522101487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634540f8b60 0x5634544d5720 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc1340cd2e0 tx=0x7fc1340305c0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.012+0000 7fc14aea0740 1 -- 172.21.15.67:0/522101487 shutdown_connections 2024-09-21T06:39:02.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.012+0000 7fc14aea0740 1 --2- 172.21.15.67:0/522101487 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x5634540d81c0 0x5634544e69d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.012+0000 7fc14aea0740 1 --2- 172.21.15.67:0/522101487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563454119840 0x5634544da740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.012+0000 7fc14aea0740 1 --2- 172.21.15.67:0/522101487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634540f8b60 0x5634544d5720 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.012+0000 7fc14aea0740 1 --2- 172.21.15.67:0/522101487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634540f0980 0x5634544d5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.012+0000 7fc14aea0740 1 -- 172.21.15.67:0/522101487 >> 172.21.15.67:0/522101487 conn(0x56345410d990 msgr2=0x5634544c3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:39:02.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.012+0000 7fc125ffb640 0 client.15543 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:39:02.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.013+0000 7fc14aea0740 1 -- 172.21.15.67:0/522101487 shutdown_connections 2024-09-21T06:39:02.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.013+0000 7fc14aea0740 1 -- 172.21.15.67:0/522101487 wait complete. 2024-09-21T06:39:02.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.013+0000 7fc14aea0740 20 client.15543 trim_cache size 0 max 16384 2024-09-21T06:39:02.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.040+0000 7fc14aea0740 1 Processor -- start 2024-09-21T06:39:02.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.041+0000 7fc14aea0740 1 -- start start 2024-09-21T06:39:02.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.041+0000 7fc14aea0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634540f0980 0x5634544be350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:02.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.041+0000 7fc14aea0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563454119840 0x5634544d2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:02.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.041+0000 7fc13e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634540f0980 0x5634544be350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:02.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.041+0000 7fc13dd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563454119840 0x5634544d2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:02.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.041+0000 7fc13e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634540f0980 0x5634544be350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.67:52016/0 (socket says 172.21.15.67:52016) 2024-09-21T06:39:02.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.041+0000 7fc13e59c640 1 -- 172.21.15.67:0/1127674462 learned_addr learned my addr 172.21.15.67:0/1127674462 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:39:02.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.041+0000 7fc14aea0740 1 --2- 172.21.15.67:0/1127674462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5634540ad570 0x5634544d2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:02.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.041+0000 7fc14aea0740 1 -- 172.21.15.67:0/1127674462 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc1401aca80 con 0x563454119840 2024-09-21T06:39:02.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.041+0000 7fc14aea0740 1 -- 172.21.15.67:0/1127674462 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc1401af360 con 0x5634540f0980 2024-09-21T06:39:02.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.041+0000 7fc14aea0740 1 -- 172.21.15.67:0/1127674462 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc1401ad150 con 0x5634540ad570 2024-09-21T06:39:02.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.042+0000 7fc13ed9d640 1 --2- 172.21.15.67:0/1127674462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5634540ad570 0x5634544d2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:02.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.042+0000 7fc13dd9b640 1 -- 172.21.15.67:0/1127674462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5634540ad570 msgr2=0x5634544d2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:39:02.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.042+0000 7fc13dd9b640 1 --2- 172.21.15.67:0/1127674462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5634540ad570 0x5634544d2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.042+0000 7fc13dd9b640 1 -- 172.21.15.67:0/1127674462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634540f0980 msgr2=0x5634544be350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:39:02.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.042+0000 7fc13dd9b640 1 --2- 172.21.15.67:0/1127674462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634540f0980 0x5634544be350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.042+0000 7fc13dd9b640 1 -- 172.21.15.67:0/1127674462 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634544c2e90 con 0x563454119840 2024-09-21T06:39:02.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.042+0000 7fc13dd9b640 1 --2- 172.21.15.67:0/1127674462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563454119840 0x5634544d2130 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc1340f4020 tx=0x7fc134030550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3cacbe49f0d1d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:39:02.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.042+0000 7fc127fff640 1 -- 172.21.15.67:0/1127674462 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1340b4040 con 0x563454119840 2024-09-21T06:39:02.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.042+0000 7fc127fff640 1 -- 172.21.15.67:0/1127674462 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc134106070 con 0x563454119840 2024-09-21T06:39:02.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.042+0000 7fc127fff640 1 -- 172.21.15.67:0/1127674462 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1340cce60 con 0x563454119840 2024-09-21T06:39:02.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.043+0000 7fc14aea0740 1 -- 172.21.15.67:0/1127674462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563454119840 msgr2=0x5634544d2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:02.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.043+0000 7fc14aea0740 1 --2- 172.21.15.67:0/1127674462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563454119840 0x5634544d2130 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc1340f4020 tx=0x7fc134030550 comp rx=0 tx=0).stop 2024-09-21T06:39:02.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.043+0000 7fc14aea0740 1 -- 172.21.15.67:0/1127674462 shutdown_connections 2024-09-21T06:39:02.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.043+0000 7fc14aea0740 1 --2- 172.21.15.67:0/1127674462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5634540ad570 0x5634544d2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.043+0000 7fc14aea0740 1 --2- 172.21.15.67:0/1127674462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563454119840 0x5634544d2130 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.043+0000 7fc14aea0740 1 --2- 172.21.15.67:0/1127674462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634540f0980 0x5634544be350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.043+0000 7fc14aea0740 1 -- 172.21.15.67:0/1127674462 >> 172.21.15.67:0/1127674462 conn(0x56345410d990 msgr2=0x5634544bdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:39:02.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.044+0000 7fc14aea0740 1 -- 172.21.15.67:0/1127674462 shutdown_connections 2024-09-21T06:39:02.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.044+0000 7fc14aea0740 1 -- 172.21.15.67:0/1127674462 wait complete. 2024-09-21T06:39:02.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.044+0000 7fc14aea0740 1 -- start start 2024-09-21T06:39:02.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.044+0000 7fc14aea0740 1 Processor -- start 2024-09-21T06:39:02.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.045+0000 7fc14aea0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634540ad570 0x5634544dbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:02.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.045+0000 7fc14aea0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5634540f0980 0x5634544dc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:02.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.045+0000 7fc14aea0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563454119840 0x563454551390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:02.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.045+0000 7fc13e59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634540ad570 0x5634544dbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:02.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.045+0000 7fc14aea0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc1401ad820 con 0x5634540ad570 2024-09-21T06:39:02.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.045+0000 7fc14aea0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc1401aec90 con 0x563454119840 2024-09-21T06:39:02.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.045+0000 7fc14aea0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc1401adef0 con 0x5634540f0980 2024-09-21T06:39:02.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.045+0000 7fc13e59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634540ad570 0x5634544dbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49750/0 (socket says 172.21.15.67:49750) 2024-09-21T06:39:02.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.045+0000 7fc13e59c640 1 -- 172.21.15.67:0/1459022104 learned_addr learned my addr 172.21.15.67:0/1459022104 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:39:02.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.045+0000 7fc14aea0740 10 client.0 _ll_register_callbacks cb 0x7fc134005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:39:02.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.045+0000 7fc13ed9d640 1 --2- 172.21.15.67:0/1459022104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563454119840 0x563454551390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:02.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.045+0000 7fc13dd9b640 1 --2- 172.21.15.67:0/1459022104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5634540f0980 0x5634544dc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:02.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.045+0000 7fc13e59c640 1 -- 172.21.15.67:0/1459022104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5634540f0980 msgr2=0x5634544dc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:39:02.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.045+0000 7fc13e59c640 1 --2- 172.21.15.67:0/1459022104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5634540f0980 0x5634544dc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.045+0000 7fc13e59c640 1 -- 172.21.15.67:0/1459022104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563454119840 msgr2=0x563454551390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:39:02.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.045+0000 7fc13e59c640 1 --2- 172.21.15.67:0/1459022104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563454119840 0x563454551390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.045+0000 7fc13e59c640 1 -- 172.21.15.67:0/1459022104 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634544c4710 con 0x5634540ad570 2024-09-21T06:39:02.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.045+0000 7fc13dd9b640 1 --2- 172.21.15.67:0/1459022104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5634540f0980 0x5634544dc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:39:02.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.046+0000 7fc13e59c640 1 --2- 172.21.15.67:0/1459022104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634540ad570 0x5634544dbd30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc130011590 tx=0x7fc1300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4754154ca7caf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:39:02.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.046+0000 7fc125ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:39:02.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.046+0000 7fc125ffb640 1 -- 172.21.15.67:0/1459022104 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc130036060 con 0x5634540ad570 2024-09-21T06:39:02.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.046+0000 7fc14aea0740 1 -- 172.21.15.67:0/1459022104 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc134106070 con 0x5634540ad570 2024-09-21T06:39:02.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.046+0000 7fc125ffb640 1 -- 172.21.15.67:0/1459022104 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fc130021900 con 0x5634540ad570 2024-09-21T06:39:02.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.046+0000 7fc125ffb640 1 -- 172.21.15.67:0/1459022104 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc130021b00 con 0x5634540ad570 2024-09-21T06:39:02.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.046+0000 7fc14aea0740 10 client.15549 fetch_fsmap learned FSMap version 21 2024-09-21T06:39:02.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.046+0000 7fc14aea0740 1 -- 172.21.15.67:0/1459022104 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563454552930 con 0x5634540ad570 2024-09-21T06:39:02.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.047+0000 7fc125ffb640 1 -- 172.21.15.67:0/1459022104 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc134106070 con 0x5634540ad570 2024-09-21T06:39:02.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.047+0000 7fc125ffb640 1 -- 172.21.15.67:0/1459022104 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc130020420 con 0x5634540ad570 2024-09-21T06:39:02.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.047+0000 7fc14aea0740 10 client.15549 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:39:02.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.047+0000 7fc14aea0740 10 client.15549 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:39:02.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.047+0000 7fc14aea0740 20 client.15549 populate_metadata read hostname 'smithi067' 2024-09-21T06:39:02.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.047+0000 7fc14aea0740 1 --2- 172.21.15.67:0/1459022104 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x563454553130 0x563454555550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:02.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.047+0000 7fc14aea0740 4 client.15549 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}] 2024-09-21T06:39:02.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.047+0000 7fc14aea0740 1 -- 172.21.15.67:0/1459022104 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}) -- 0x563454434590 con 0x563454553130 2024-09-21T06:39:02.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.047+0000 7fc13dd9b640 1 --2- 172.21.15.67:0/1459022104 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x563454553130 0x563454555550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:02.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.048+0000 7fc13dd9b640 1 --2- 172.21.15.67:0/1459022104 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x563454553130 0x563454555550 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7355dbeb465cd320 server_cookie=b0480e13d40b0fcb in_seq=0 out_seq=0 2024-09-21T06:39:02.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.048+0000 7fc125ffb640 10 client.15549 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:39:02.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.085+0000 7fc125ffb640 1 -- 172.21.15.67:0/1459022104 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563454434590 con 0x563454553130 2024-09-21T06:39:02.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.085+0000 7fc125ffb640 10 client.15549 handle_command_reply: tid=0 2024-09-21T06:39:02.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.085+0000 7fc14aea0740 1 client.15549 shutdown 2024-09-21T06:39:02.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.085+0000 7fc14aea0740 1 -- 172.21.15.67:0/1459022104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634540ad570 msgr2=0x5634544dbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:02.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.085+0000 7fc14aea0740 1 --2- 172.21.15.67:0/1459022104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634540ad570 0x5634544dbd30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc130011590 tx=0x7fc1300046c0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.086+0000 7fc14aea0740 1 -- 172.21.15.67:0/1459022104 shutdown_connections 2024-09-21T06:39:02.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.086+0000 7fc14aea0740 1 --2- 172.21.15.67:0/1459022104 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x563454553130 0x563454555550 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.086+0000 7fc14aea0740 1 --2- 172.21.15.67:0/1459022104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563454119840 0x563454551390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.086+0000 7fc14aea0740 1 --2- 172.21.15.67:0/1459022104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5634540f0980 0x5634544dc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.086+0000 7fc14aea0740 1 --2- 172.21.15.67:0/1459022104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634540ad570 0x5634544dbd30 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.086+0000 7fc14aea0740 1 -- 172.21.15.67:0/1459022104 >> 172.21.15.67:0/1459022104 conn(0x56345410d990 msgr2=0x5634544bdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:39:02.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.086+0000 7fc125ffb640 0 client.15549 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:39:02.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.086+0000 7fc14aea0740 1 -- 172.21.15.67:0/1459022104 shutdown_connections 2024-09-21T06:39:02.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.086+0000 7fc14aea0740 1 -- 172.21.15.67:0/1459022104 wait complete. 2024-09-21T06:39:02.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.086+0000 7fc14aea0740 20 client.15549 trim_cache size 0 max 16384 2024-09-21T06:39:02.090 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-21T06:39:02.090 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 21, 2024-09-21T06:39:02.090 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-09-21T06:39:02.090 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 10, 2024-09-21T06:39:02.090 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-21T06:39:02.090 INFO:teuthology.orchestra.run.smithi067.stdout: "aa36f322": { 2024-09-21T06:39:02.090 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 10, 2024-09-21T06:39:02.091 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-21T06:39:02.091 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:39:02.091 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-21T06:39:02.091 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:39:02.091 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:39:02.091 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-21T06:39:02.091 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 147.8, 2024-09-21T06:39:02.091 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-21T06:39:02.091 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-21T06:39:02.091 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-21T06:39:02.091 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:39:02.091 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-21T06:39:02.092 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:39:02.092 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:39:02.092 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:39:02.092 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:39:02.092 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:39:02.092 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:39:02.092 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-21T06:39:02.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.087+0000 7fc14492c640 1 -- 172.21.15.67:0/1593393076 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc1280b6590 msgr2=0x7fc1280b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:02.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.087+0000 7fc14492c640 1 --2- 172.21.15.67:0/1593393076 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc1280b6590 0x7fc1280b8aa0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc1300095f0 tx=0x7fc130005f90 comp rx=0 tx=0).stop 2024-09-21T06:39:02.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.087+0000 7fc14492c640 1 -- 172.21.15.67:0/1593393076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1401083e0 msgr2=0x7fc14019a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:02.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.087+0000 7fc14492c640 1 --2- 172.21.15.67:0/1593393076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1401083e0 0x7fc14019a980 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc134005940 tx=0x7fc13402ae40 comp rx=0 tx=0).stop 2024-09-21T06:39:02.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.087+0000 7fc14492c640 1 -- 172.21.15.67:0/1593393076 shutdown_connections 2024-09-21T06:39:02.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.087+0000 7fc14492c640 1 --2- 172.21.15.67:0/1593393076 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc1280b6590 0x7fc1280b8aa0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.087+0000 7fc14492c640 1 --2- 172.21.15.67:0/1593393076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc140108d30 0x7fc14019fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.087+0000 7fc14492c640 1 --2- 172.21.15.67:0/1593393076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1401083e0 0x7fc14019a980 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.087+0000 7fc14492c640 1 --2- 172.21.15.67:0/1593393076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc140102af0 0x7fc14019a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:02.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.087+0000 7fc14492c640 1 -- 172.21.15.67:0/1593393076 >> 172.21.15.67:0/1593393076 conn(0x7fc140079830 msgr2=0x7fc1401064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:39:02.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.087+0000 7fc14492c640 1 -- 172.21.15.67:0/1593393076 shutdown_connections 2024-09-21T06:39:02.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:02.087+0000 7fc14492c640 1 -- 172.21.15.67:0/1593393076 wait complete. 2024-09-21T06:39:02.879 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'aa36f322', quiesce took 4.4 seconds. Will release after: 23.4 2024-09-21T06:39:03.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:03 smithi203 ceph-mon[33735]: pgmap v434: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 169 MiB/s wr, 88 op/s 2024-09-21T06:39:03.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:03 smithi067 ceph-mon[31232]: pgmap v434: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 169 MiB/s wr, 88 op/s 2024-09-21T06:39:03.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:03 smithi100 ceph-mon[33401]: pgmap v434: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 169 MiB/s wr, 88 op/s 2024-09-21T06:39:05.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:05 smithi203 ceph-mon[33735]: pgmap v435: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 142 MiB/s wr, 74 op/s 2024-09-21T06:39:05.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:05 smithi067 ceph-mon[31232]: pgmap v435: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 142 MiB/s wr, 74 op/s 2024-09-21T06:39:05.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:05 smithi100 ceph-mon[33401]: pgmap v435: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 142 MiB/s wr, 74 op/s 2024-09-21T06:39:07.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:07 smithi203 ceph-mon[33735]: pgmap v436: 129 pgs: 129 active+clean; 5.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 129 MiB/s wr, 69 op/s 2024-09-21T06:39:07.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:07 smithi067 ceph-mon[31232]: pgmap v436: 129 pgs: 129 active+clean; 5.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 129 MiB/s wr, 69 op/s 2024-09-21T06:39:07.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:07 smithi100 ceph-mon[33401]: pgmap v436: 129 pgs: 129 active+clean; 5.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 129 MiB/s wr, 69 op/s 2024-09-21T06:39:09.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:09 smithi203 ceph-mon[33735]: pgmap v437: 129 pgs: 129 active+clean; 5.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 118 MiB/s wr, 62 op/s 2024-09-21T06:39:09.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:09 smithi067 ceph-mon[31232]: pgmap v437: 129 pgs: 129 active+clean; 5.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 118 MiB/s wr, 62 op/s 2024-09-21T06:39:09.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:09 smithi100 ceph-mon[33401]: pgmap v437: 129 pgs: 129 active+clean; 5.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 118 MiB/s wr, 62 op/s 2024-09-21T06:39:11.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:11 smithi203 ceph-mon[33735]: pgmap v438: 129 pgs: 129 active+clean; 5.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 73 MiB/s wr, 39 op/s 2024-09-21T06:39:11.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:11 smithi067 ceph-mon[31232]: pgmap v438: 129 pgs: 129 active+clean; 5.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 73 MiB/s wr, 39 op/s 2024-09-21T06:39:11.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:11 smithi100 ceph-mon[33401]: pgmap v438: 129 pgs: 129 active+clean; 5.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 73 MiB/s wr, 39 op/s 2024-09-21T06:39:13.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:13 smithi203 ceph-mon[33735]: pgmap v439: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 66 MiB/s wr, 36 op/s 2024-09-21T06:39:13.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:13 smithi067 ceph-mon[31232]: pgmap v439: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 66 MiB/s wr, 36 op/s 2024-09-21T06:39:13.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:13 smithi100 ceph-mon[33401]: pgmap v439: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 66 MiB/s wr, 36 op/s 2024-09-21T06:39:15.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:15 smithi203 ceph-mon[33735]: pgmap v440: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-21T06:39:15.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:15 smithi067 ceph-mon[31232]: pgmap v440: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-21T06:39:15.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:15 smithi100 ceph-mon[33401]: pgmap v440: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-21T06:39:17.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:17 smithi203 ceph-mon[33735]: pgmap v441: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9.0 MiB/s wr, 8 op/s 2024-09-21T06:39:17.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:17 smithi067 ceph-mon[31232]: pgmap v441: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9.0 MiB/s wr, 8 op/s 2024-09-21T06:39:17.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:17 smithi100 ceph-mon[33401]: pgmap v441: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9.0 MiB/s wr, 8 op/s 2024-09-21T06:39:19.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:19 smithi203 ceph-mon[33735]: pgmap v442: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4.3 MiB/s wr, 4 op/s 2024-09-21T06:39:19.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:19 smithi067 ceph-mon[31232]: pgmap v442: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4.3 MiB/s wr, 4 op/s 2024-09-21T06:39:19.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:19 smithi100 ceph-mon[33401]: pgmap v442: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4.3 MiB/s wr, 4 op/s 2024-09-21T06:39:21.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:21 smithi203 ceph-mon[33735]: pgmap v443: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 353 KiB/s wr, 2 op/s 2024-09-21T06:39:21.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:21 smithi067 ceph-mon[31232]: pgmap v443: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 353 KiB/s wr, 2 op/s 2024-09-21T06:39:21.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:21 smithi100 ceph-mon[33401]: pgmap v443: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 353 KiB/s wr, 2 op/s 2024-09-21T06:39:23.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:23 smithi203 ceph-mon[33735]: pgmap v444: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 353 KiB/s wr, 3 op/s 2024-09-21T06:39:23.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:23 smithi067 ceph-mon[31232]: pgmap v444: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 353 KiB/s wr, 3 op/s 2024-09-21T06:39:23.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:23 smithi100 ceph-mon[33401]: pgmap v444: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 353 KiB/s wr, 3 op/s 2024-09-21T06:39:25.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:25 smithi203 ceph-mon[33735]: pgmap v445: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 938 B/s wr, 1 op/s 2024-09-21T06:39:25.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:25 smithi067 ceph-mon[31232]: pgmap v445: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 938 B/s wr, 1 op/s 2024-09-21T06:39:25.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:25 smithi100 ceph-mon[33401]: pgmap v445: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 938 B/s wr, 1 op/s 2024-09-21T06:39:26.280 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:39:26.555 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:39:27.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.547+0000 7fd611145640 1 -- 172.21.15.67:0/65752844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd60c1094a0 msgr2=0x7fd60c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:27.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.547+0000 7fd611145640 1 --2- 172.21.15.67:0/65752844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd60c1094a0 0x7fd60c1098b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd600009900 tx=0x7fd60002a410 comp rx=0 tx=0).stop 2024-09-21T06:39:27.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.547+0000 7fd611145640 1 -- 172.21.15.67:0/65752844 shutdown_connections 2024-09-21T06:39:27.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.547+0000 7fd611145640 1 --2- 172.21.15.67:0/65752844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd60c109df0 0x7fd60c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:27.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.547+0000 7fd611145640 1 --2- 172.21.15.67:0/65752844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd60c1094a0 0x7fd60c1098b0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:27.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.547+0000 7fd611145640 1 --2- 172.21.15.67:0/65752844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd60c103b60 0x7fd60c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:27.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.547+0000 7fd611145640 1 -- 172.21.15.67:0/65752844 >> 172.21.15.67:0/65752844 conn(0x7fd60c07b850 msgr2=0x7fd60c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:39:27.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.547+0000 7fd611145640 1 -- 172.21.15.67:0/65752844 shutdown_connections 2024-09-21T06:39:27.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.547+0000 7fd611145640 1 -- 172.21.15.67:0/65752844 wait complete. 2024-09-21T06:39:27.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.548+0000 7fd611145640 1 Processor -- start 2024-09-21T06:39:27.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.548+0000 7fd611145640 1 -- start start 2024-09-21T06:39:27.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.548+0000 7fd611145640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd60c103b60 0x7fd60c19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:27.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.548+0000 7fd611145640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd60c1094a0 0x7fd60c19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:27.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.548+0000 7fd611145640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd60c109df0 0x7fd60c19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:27.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.548+0000 7fd611145640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c113960 con 0x7fd60c103b60 2024-09-21T06:39:27.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.548+0000 7fd611145640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c1137e0 con 0x7fd60c109df0 2024-09-21T06:39:27.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.548+0000 7fd611145640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c113ae0 con 0x7fd60c1094a0 2024-09-21T06:39:27.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.548+0000 7fd60ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd60c103b60 0x7fd60c19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:27.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.548+0000 7fd60ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd60c103b60 0x7fd60c19a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34040/0 (socket says 172.21.15.67:34040) 2024-09-21T06:39:27.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.548+0000 7fd60ad9d640 1 -- 172.21.15.67:0/161945745 learned_addr learned my addr 172.21.15.67:0/161945745 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:39:27.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.548+0000 7fd60a59c640 1 --2- 172.21.15.67:0/161945745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd60c1094a0 0x7fd60c19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:27.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.548+0000 7fd60b59e640 1 --2- 172.21.15.67:0/161945745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd60c109df0 0x7fd60c19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:27.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.549+0000 7fd60ad9d640 1 -- 172.21.15.67:0/161945745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd60c1094a0 msgr2=0x7fd60c19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:27.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.549+0000 7fd60ad9d640 1 --2- 172.21.15.67:0/161945745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd60c1094a0 0x7fd60c19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:27.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.549+0000 7fd60ad9d640 1 -- 172.21.15.67:0/161945745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd60c109df0 msgr2=0x7fd60c19fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:27.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.549+0000 7fd60ad9d640 1 --2- 172.21.15.67:0/161945745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd60c109df0 0x7fd60c19fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:27.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.549+0000 7fd60ad9d640 1 -- 172.21.15.67:0/161945745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd60c1a0380 con 0x7fd60c103b60 2024-09-21T06:39:27.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.549+0000 7fd60ad9d640 1 --2- 172.21.15.67:0/161945745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd60c103b60 0x7fd60c19a410 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc00e9f0 tx=0x7fd5fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:39:27.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.549+0000 7fd5f3fff640 1 -- 172.21.15.67:0/161945745 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc00cd80 con 0x7fd60c103b60 2024-09-21T06:39:27.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.549+0000 7fd611145640 1 -- 172.21.15.67:0/161945745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd60c1a0670 con 0x7fd60c103b60 2024-09-21T06:39:27.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.549+0000 7fd5f3fff640 1 -- 172.21.15.67:0/161945745 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5fc0108c0 con 0x7fd60c103b60 2024-09-21T06:39:27.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.549+0000 7fd5f3fff640 1 -- 172.21.15.67:0/161945745 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc0187c0 con 0x7fd60c103b60 2024-09-21T06:39:27.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.549+0000 7fd611145640 1 -- 172.21.15.67:0/161945745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd60c1a6dd0 con 0x7fd60c103b60 2024-09-21T06:39:27.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.550+0000 7fd611145640 1 -- 172.21.15.67:0/161945745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd60c103f70 con 0x7fd60c103b60 2024-09-21T06:39:27.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.552+0000 7fd5f3fff640 1 -- 172.21.15.67:0/161945745 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fd5fc010430 con 0x7fd60c103b60 2024-09-21T06:39:27.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.553+0000 7fd5f3fff640 1 --2- 172.21.15.67:0/161945745 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd5f40b6510 0x7fd5f40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:27.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.553+0000 7fd60a59c640 1 --2- 172.21.15.67:0/161945745 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd5f40b6510 0x7fd5f40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:27.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.553+0000 7fd5f3fff640 1 -- 172.21.15.67:0/161945745 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fd5fc0e7ec0 con 0x7fd60c103b60 2024-09-21T06:39:27.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.554+0000 7fd60a59c640 1 --2- 172.21.15.67:0/161945745 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd5f40b6510 0x7fd5f40b8a20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd600005940 tx=0x7fd6000058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:39:27.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.556+0000 7fd5f3fff640 1 -- 172.21.15.67:0/161945745 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd5fc0b2760 con 0x7fd60c103b60 2024-09-21T06:39:27.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:27 smithi203 ceph-mon[33735]: pgmap v446: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-21T06:39:27.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.785+0000 7fd611145640 1 -- 172.21.15.67:0/161945745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd60c19b540 con 0x7fd60c103b60 2024-09-21T06:39:27.787 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:27 smithi067 ceph-mon[31232]: pgmap v446: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-21T06:39:27.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.786+0000 7fd5f3fff640 1 -- 172.21.15.67:0/161945745 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7fd5fc0b76a0 con 0x7fd60c103b60 2024-09-21T06:39:27.788 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:39:27.788 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:39:27.788 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:39:27.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.788+0000 7fd611145640 1 -- 172.21.15.67:0/161945745 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd5f40b6510 msgr2=0x7fd5f40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:27.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.788+0000 7fd611145640 1 --2- 172.21.15.67:0/161945745 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd5f40b6510 0x7fd5f40b8a20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd600005940 tx=0x7fd6000058d0 comp rx=0 tx=0).stop 2024-09-21T06:39:27.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.788+0000 7fd611145640 1 -- 172.21.15.67:0/161945745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd60c103b60 msgr2=0x7fd60c19a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:27.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.788+0000 7fd611145640 1 --2- 172.21.15.67:0/161945745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd60c103b60 0x7fd60c19a410 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc00e9f0 tx=0x7fd5fc00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:39:27.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.788+0000 7fd611145640 1 -- 172.21.15.67:0/161945745 shutdown_connections 2024-09-21T06:39:27.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.788+0000 7fd611145640 1 --2- 172.21.15.67:0/161945745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd60c109df0 0x7fd60c19fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:27.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.788+0000 7fd611145640 1 --2- 172.21.15.67:0/161945745 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd5f40b6510 0x7fd5f40b8a20 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:27.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.788+0000 7fd611145640 1 --2- 172.21.15.67:0/161945745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd60c1094a0 0x7fd60c19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:27.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.788+0000 7fd611145640 1 --2- 172.21.15.67:0/161945745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd60c103b60 0x7fd60c19a410 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:27.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.788+0000 7fd611145640 1 -- 172.21.15.67:0/161945745 >> 172.21.15.67:0/161945745 conn(0x7fd60c07b850 msgr2=0x7fd60c1055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:39:27.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.789+0000 7fd611145640 1 -- 172.21.15.67:0/161945745 shutdown_connections 2024-09-21T06:39:27.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:27.789+0000 7fd611145640 1 -- 172.21.15.67:0/161945745 wait complete. 2024-09-21T06:39:27.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:27 smithi100 ceph-mon[33401]: pgmap v446: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-21T06:39:28.557 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db --set-id aa36f322 --release --await' 2024-09-21T06:39:28.557 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24671 quiesce db --set-id aa36f322 --release --await 2024-09-21T06:39:28.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:28 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/161945745' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:39:28.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:28 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/161945745' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:39:28.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:28 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/161945745' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:39:28.815 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:39:29.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.296+0000 7ff199c4e640 1 -- 172.21.15.67:0/1216626706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff194108420 msgr2=0x7ff194108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:29.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.296+0000 7ff199c4e640 1 --2- 172.21.15.67:0/1216626706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff194108420 0x7ff194108830 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff188009900 tx=0x7ff18802a3f0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.296+0000 7ff199c4e640 1 -- 172.21.15.67:0/1216626706 shutdown_connections 2024-09-21T06:39:29.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.296+0000 7ff199c4e640 1 --2- 172.21.15.67:0/1216626706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff194108d70 0x7ff19410f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.296+0000 7ff199c4e640 1 --2- 172.21.15.67:0/1216626706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff194108420 0x7ff194108830 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.296+0000 7ff199c4e640 1 --2- 172.21.15.67:0/1216626706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff194102ae0 0x7ff194102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.296+0000 7ff199c4e640 1 -- 172.21.15.67:0/1216626706 >> 172.21.15.67:0/1216626706 conn(0x7ff194079770 msgr2=0x7ff194079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:39:29.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.296+0000 7ff199c4e640 1 -- 172.21.15.67:0/1216626706 shutdown_connections 2024-09-21T06:39:29.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.296+0000 7ff199c4e640 1 -- 172.21.15.67:0/1216626706 wait complete. 2024-09-21T06:39:29.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.297+0000 7ff199c4e640 1 Processor -- start 2024-09-21T06:39:29.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.297+0000 7ff199c4e640 1 -- start start 2024-09-21T06:39:29.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.298+0000 7ff199c4e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff194102ae0 0x7ff19410ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:29.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.298+0000 7ff199c4e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff194108420 0x7ff194110440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:29.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.298+0000 7ff199c4e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff194108d70 0x7ff194110980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:29.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.298+0000 7ff199c4e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff1941139d0 con 0x7ff194102ae0 2024-09-21T06:39:29.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.298+0000 7ff199c4e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff194113850 con 0x7ff194108d70 2024-09-21T06:39:29.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.298+0000 7ff199c4e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff194113b50 con 0x7ff194108420 2024-09-21T06:39:29.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.298+0000 7ff1937fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff194102ae0 0x7ff19410ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:29.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.298+0000 7ff1937fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff194102ae0 0x7ff19410ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34056/0 (socket says 172.21.15.67:34056) 2024-09-21T06:39:29.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.298+0000 7ff1937fe640 1 -- 172.21.15.67:0/1860909310 learned_addr learned my addr 172.21.15.67:0/1860909310 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:39:29.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.298+0000 7ff192ffd640 1 --2- 172.21.15.67:0/1860909310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff194108420 0x7ff194110440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:29.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.298+0000 7ff193fff640 1 --2- 172.21.15.67:0/1860909310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff194108d70 0x7ff194110980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:29.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.298+0000 7ff1937fe640 1 -- 172.21.15.67:0/1860909310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff194108420 msgr2=0x7ff194110440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:29.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.298+0000 7ff1937fe640 1 --2- 172.21.15.67:0/1860909310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff194108420 0x7ff194110440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.298+0000 7ff1937fe640 1 -- 172.21.15.67:0/1860909310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff194108d70 msgr2=0x7ff194110980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:29.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.298+0000 7ff1937fe640 1 --2- 172.21.15.67:0/1860909310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff194108d70 0x7ff194110980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.298+0000 7ff1937fe640 1 -- 172.21.15.67:0/1860909310 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1941a8650 con 0x7ff194102ae0 2024-09-21T06:39:29.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.298+0000 7ff192ffd640 1 --2- 172.21.15.67:0/1860909310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff194108420 0x7ff194110440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:39:29.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.298+0000 7ff1937fe640 1 --2- 172.21.15.67:0/1860909310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff194102ae0 0x7ff19410ff00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff18400c920 tx=0x7ff18400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:39:29.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.299+0000 7ff190ff9640 1 -- 172.21.15.67:0/1860909310 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff184007c70 con 0x7ff194102ae0 2024-09-21T06:39:29.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.299+0000 7ff199c4e640 1 -- 172.21.15.67:0/1860909310 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1941a8940 con 0x7ff194102ae0 2024-09-21T06:39:29.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.299+0000 7ff190ff9640 1 -- 172.21.15.67:0/1860909310 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff184007e10 con 0x7ff194102ae0 2024-09-21T06:39:29.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.299+0000 7ff190ff9640 1 -- 172.21.15.67:0/1860909310 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff184018710 con 0x7ff194102ae0 2024-09-21T06:39:29.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.299+0000 7ff199c4e640 1 -- 172.21.15.67:0/1860909310 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1941a8e50 con 0x7ff194102ae0 2024-09-21T06:39:29.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.301+0000 7ff1a01c2740 1 Processor -- start 2024-09-21T06:39:29.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.301+0000 7ff1a01c2740 1 -- start start 2024-09-21T06:39:29.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.301+0000 7ff190ff9640 1 -- 172.21.15.67:0/1860909310 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7ff18400f450 con 0x7ff194102ae0 2024-09-21T06:39:29.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.302+0000 7ff1a01c2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56026b3dc520 0x56026b782510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:29.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.302+0000 7ff1a01c2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b3b3660 0x56026b782a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:29.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.302+0000 7ff1a01c2740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56026b3bb840 0x56026b782f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:29.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.302+0000 7ff1a01c2740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56026b66ddc0 con 0x56026b3bb840 2024-09-21T06:39:29.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.302+0000 7ff1a01c2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56026b6fa1f0 con 0x56026b3b3660 2024-09-21T06:39:29.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.302+0000 7ff1a01c2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56026b6e9620 con 0x56026b3dc520 2024-09-21T06:39:29.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.302+0000 7ff192ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56026b3bb840 0x56026b782f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:29.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.302+0000 7ff192ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56026b3bb840 0x56026b782f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34060/0 (socket says 172.21.15.67:34060) 2024-09-21T06:39:29.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.302+0000 7ff192ffd640 1 -- 172.21.15.67:0/2851697272 learned_addr learned my addr 172.21.15.67:0/2851697272 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:39:29.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.302+0000 7ff1937fe640 1 --2- 172.21.15.67:0/2851697272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b3b3660 0x56026b782a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:29.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.302+0000 7ff193fff640 1 --2- 172.21.15.67:0/2851697272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56026b3dc520 0x56026b782510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:29.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.302+0000 7ff192ffd640 1 -- 172.21.15.67:0/2851697272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56026b3dc520 msgr2=0x56026b782510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:39:29.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.302+0000 7ff192ffd640 1 --2- 172.21.15.67:0/2851697272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56026b3dc520 0x56026b782510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.302+0000 7ff192ffd640 1 -- 172.21.15.67:0/2851697272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b3b3660 msgr2=0x56026b782a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:39:29.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.302+0000 7ff192ffd640 1 --2- 172.21.15.67:0/2851697272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b3b3660 0x56026b782a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.302+0000 7ff192ffd640 1 -- 172.21.15.67:0/2851697272 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56026b6b1fd0 con 0x56026b3bb840 2024-09-21T06:39:29.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.302+0000 7ff1937fe640 1 --2- 172.21.15.67:0/2851697272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b3b3660 0x56026b782a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:39:29.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.302+0000 7ff190ff9640 1 --2- 172.21.15.67:0/1860909310 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7ff17c0b6510 0x7ff17c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:29.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.303+0000 7ff192ffd640 1 --2- 172.21.15.67:0/2851697272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56026b3bb840 0x56026b782f90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff19406b9c0 tx=0x7ff188032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcaaf4b204a5a6d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:39:29.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.303+0000 7ff193fff640 1 --2- 172.21.15.67:0/1860909310 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7ff17c0b6510 0x7ff17c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:29.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.303+0000 7ff190ff9640 1 -- 172.21.15.67:0/1860909310 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7ff1840e7970 con 0x7ff194102ae0 2024-09-21T06:39:29.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.303+0000 7ff179ffb640 1 -- 172.21.15.67:0/2851697272 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18803f070 con 0x56026b3bb840 2024-09-21T06:39:29.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.303+0000 7ff179ffb640 1 -- 172.21.15.67:0/2851697272 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1880044c0 con 0x56026b3bb840 2024-09-21T06:39:29.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.303+0000 7ff179ffb640 1 -- 172.21.15.67:0/2851697272 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff188035070 con 0x56026b3bb840 2024-09-21T06:39:29.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.304+0000 7ff1a01c2740 1 -- 172.21.15.67:0/2851697272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56026b3bb840 msgr2=0x56026b782f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:29.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.304+0000 7ff193fff640 1 --2- 172.21.15.67:0/1860909310 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7ff17c0b6510 0x7ff17c0b8a20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff18c005fd0 tx=0x7ff18c009580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:39:29.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.304+0000 7ff1a01c2740 1 --2- 172.21.15.67:0/2851697272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56026b3bb840 0x56026b782f90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff19406b9c0 tx=0x7ff188032dd0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.304+0000 7ff1a01c2740 1 -- 172.21.15.67:0/2851697272 shutdown_connections 2024-09-21T06:39:29.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.304+0000 7ff1a01c2740 1 --2- 172.21.15.67:0/2851697272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b3b3660 0x56026b782a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.304+0000 7ff1a01c2740 1 --2- 172.21.15.67:0/2851697272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56026b3bb840 0x56026b782f90 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.304+0000 7ff1a01c2740 1 --2- 172.21.15.67:0/2851697272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56026b3dc520 0x56026b782510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.304+0000 7ff1a01c2740 1 -- 172.21.15.67:0/2851697272 >> 172.21.15.67:0/2851697272 conn(0x56026b3d0910 msgr2=0x56026b780400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:39:29.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.304+0000 7ff1a01c2740 1 -- 172.21.15.67:0/2851697272 shutdown_connections 2024-09-21T06:39:29.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.304+0000 7ff1a01c2740 1 -- 172.21.15.67:0/2851697272 wait complete. 2024-09-21T06:39:29.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.304+0000 7ff1a01c2740 1 -- start start 2024-09-21T06:39:29.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.305+0000 7ff1a01c2740 1 Processor -- start 2024-09-21T06:39:29.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.305+0000 7ff1a01c2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b3b3660 0x56026b78dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:29.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.305+0000 7ff1a01c2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56026b3bb840 0x56026b78e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:29.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.305+0000 7ff1a01c2740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56026b3dc520 0x56026b78e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:29.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.305+0000 7ff1a01c2740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56026b684300 con 0x56026b3dc520 2024-09-21T06:39:29.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.305+0000 7ff1a01c2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56026b6ed430 con 0x56026b3b3660 2024-09-21T06:39:29.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.305+0000 7ff1a01c2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56026b54abb0 con 0x56026b3bb840 2024-09-21T06:39:29.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.305+0000 7ff1a01c2740 10 client.0 _ll_register_callbacks cb 0x56026b69e120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:39:29.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.305+0000 7ff193fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56026b3dc520 0x56026b78e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:29.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.305+0000 7ff193fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56026b3dc520 0x56026b78e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34068/0 (socket says 172.21.15.67:34068) 2024-09-21T06:39:29.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.305+0000 7ff193fff640 1 -- 172.21.15.67:0/1600823101 learned_addr learned my addr 172.21.15.67:0/1600823101 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:39:29.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.305+0000 7ff192ffd640 1 --2- 172.21.15.67:0/1600823101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56026b3bb840 0x56026b78e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:29.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.305+0000 7ff1937fe640 1 --2- 172.21.15.67:0/1600823101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b3b3660 0x56026b78dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:29.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.305+0000 7ff193fff640 1 -- 172.21.15.67:0/1600823101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56026b3bb840 msgr2=0x56026b78e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:39:29.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.305+0000 7ff193fff640 1 --2- 172.21.15.67:0/1600823101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56026b3bb840 0x56026b78e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.305+0000 7ff193fff640 1 -- 172.21.15.67:0/1600823101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b3b3660 msgr2=0x56026b78dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:39:29.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.305+0000 7ff193fff640 1 --2- 172.21.15.67:0/1600823101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b3b3660 0x56026b78dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.305+0000 7ff193fff640 1 -- 172.21.15.67:0/1600823101 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56026b673780 con 0x56026b3dc520 2024-09-21T06:39:29.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.305+0000 7ff1937fe640 1 --2- 172.21.15.67:0/1600823101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b3b3660 0x56026b78dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:39:29.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.306+0000 7ff193fff640 1 --2- 172.21.15.67:0/1600823101 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56026b3dc520 0x56026b78e550 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff18c012900 tx=0x7ff18c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e86a462d90be73c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:39:29.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.306+0000 7ff172ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:39:29.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.306+0000 7ff172ffd640 1 -- 172.21.15.67:0/1600823101 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c00ece0 con 0x56026b3dc520 2024-09-21T06:39:29.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.306+0000 7ff172ffd640 1 -- 172.21.15.67:0/1600823101 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7ff18c01f420 con 0x56026b3dc520 2024-09-21T06:39:29.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.306+0000 7ff1a01c2740 1 -- 172.21.15.67:0/1600823101 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56026b6fc140 con 0x56026b3dc520 2024-09-21T06:39:29.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.306+0000 7ff172ffd640 1 -- 172.21.15.67:0/1600823101 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff18c009670 con 0x56026b3dc520 2024-09-21T06:39:29.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.306+0000 7ff1a01c2740 10 client.15567 fetch_fsmap learned FSMap version 21 2024-09-21T06:39:29.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.306+0000 7ff1a01c2740 1 -- 172.21.15.67:0/1600823101 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56026b447b00 con 0x56026b3dc520 2024-09-21T06:39:29.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.306+0000 7ff172ffd640 1 -- 172.21.15.67:0/1600823101 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x56026b6fc140 con 0x56026b3dc520 2024-09-21T06:39:29.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.306+0000 7ff172ffd640 1 -- 172.21.15.67:0/1600823101 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ff18c00f040 con 0x56026b3dc520 2024-09-21T06:39:29.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.307+0000 7ff1a01c2740 10 client.15567 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:39:29.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.307+0000 7ff1a01c2740 10 client.15567 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:39:29.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.307+0000 7ff1a01c2740 20 client.15567 populate_metadata read hostname 'smithi067' 2024-09-21T06:39:29.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.307+0000 7ff1a01c2740 1 --2- 172.21.15.67:0/1600823101 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x56026b39aea0 0x56026b79fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:29.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.307+0000 7ff1a01c2740 4 client.15567 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:39:29.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.307+0000 7ff1a01c2740 1 -- 172.21.15.67:0/1600823101 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56026b6f7bf0 con 0x56026b39aea0 2024-09-21T06:39:29.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.307+0000 7ff1937fe640 1 --2- 172.21.15.67:0/1600823101 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x56026b39aea0 0x56026b79fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:29.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.308+0000 7ff1937fe640 1 --2- 172.21.15.67:0/1600823101 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x56026b39aea0 0x56026b79fff0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e94a5f03fe224497 server_cookie=aee711f54cfe9210 in_seq=0 out_seq=0 2024-09-21T06:39:29.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.308+0000 7ff172ffd640 10 client.15567 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:39:29.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.308+0000 7ff172ffd640 1 -- 172.21.15.67:0/1600823101 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56026b6f7bf0 con 0x56026b39aea0 2024-09-21T06:39:29.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.308+0000 7ff172ffd640 10 client.15567 handle_command_reply: tid=0 2024-09-21T06:39:29.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.308+0000 7ff1a01c2740 1 client.15567 shutdown 2024-09-21T06:39:29.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.309+0000 7ff1a01c2740 1 -- 172.21.15.67:0/1600823101 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56026b3dc520 msgr2=0x56026b78e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:29.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.309+0000 7ff1a01c2740 1 --2- 172.21.15.67:0/1600823101 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56026b3dc520 0x56026b78e550 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff18c012900 tx=0x7ff18c004a40 comp rx=0 tx=0).stop 2024-09-21T06:39:29.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.309+0000 7ff1a01c2740 1 -- 172.21.15.67:0/1600823101 shutdown_connections 2024-09-21T06:39:29.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.309+0000 7ff1a01c2740 1 --2- 172.21.15.67:0/1600823101 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x56026b39aea0 0x56026b79fff0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.309+0000 7ff1a01c2740 1 --2- 172.21.15.67:0/1600823101 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56026b3dc520 0x56026b78e550 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.309+0000 7ff1a01c2740 1 --2- 172.21.15.67:0/1600823101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56026b3bb840 0x56026b78e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.309+0000 7ff1a01c2740 1 --2- 172.21.15.67:0/1600823101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b3b3660 0x56026b78dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.309+0000 7ff1a01c2740 1 -- 172.21.15.67:0/1600823101 >> 172.21.15.67:0/1600823101 conn(0x56026b3d0910 msgr2=0x56026b7856a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:39:29.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.309+0000 7ff172ffd640 0 client.15567 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:39:29.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.309+0000 7ff1a01c2740 1 -- 172.21.15.67:0/1600823101 shutdown_connections 2024-09-21T06:39:29.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.309+0000 7ff1a01c2740 1 -- 172.21.15.67:0/1600823101 wait complete. 2024-09-21T06:39:29.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.309+0000 7ff1a01c2740 20 client.15567 trim_cache size 0 max 16384 2024-09-21T06:39:29.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.326+0000 7ff1a01c2740 1 Processor -- start 2024-09-21T06:39:29.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.326+0000 7ff1a01c2740 1 -- start start 2024-09-21T06:39:29.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.326+0000 7ff1a01c2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56026b3b3660 0x56026b78e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:29.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.326+0000 7ff1a01c2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b3bb840 0x56026b78f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:29.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.326+0000 7ff1a01c2740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56026b3dc520 0x56026b78acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:29.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.326+0000 7ff1a01c2740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff1941b0b60 con 0x56026b3dc520 2024-09-21T06:39:29.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.326+0000 7ff1a01c2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff1941b33e0 con 0x56026b3bb840 2024-09-21T06:39:29.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.326+0000 7ff1a01c2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff1941b11d0 con 0x56026b3b3660 2024-09-21T06:39:29.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.327+0000 7ff193fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56026b3dc520 0x56026b78acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:29.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.327+0000 7ff193fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56026b3dc520 0x56026b78acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34070/0 (socket says 172.21.15.67:34070) 2024-09-21T06:39:29.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.327+0000 7ff193fff640 1 -- 172.21.15.67:0/2391960466 learned_addr learned my addr 172.21.15.67:0/2391960466 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:39:29.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.327+0000 7ff192ffd640 1 --2- 172.21.15.67:0/2391960466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b3bb840 0x56026b78f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:29.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.327+0000 7ff1937fe640 1 --2- 172.21.15.67:0/2391960466 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56026b3b3660 0x56026b78e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:29.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.327+0000 7ff193fff640 1 -- 172.21.15.67:0/2391960466 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56026b3b3660 msgr2=0x56026b78e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:39:29.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.327+0000 7ff193fff640 1 --2- 172.21.15.67:0/2391960466 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56026b3b3660 0x56026b78e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.327+0000 7ff193fff640 1 -- 172.21.15.67:0/2391960466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b3bb840 msgr2=0x56026b78f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:39:29.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.327+0000 7ff193fff640 1 --2- 172.21.15.67:0/2391960466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b3bb840 0x56026b78f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.327+0000 7ff193fff640 1 -- 172.21.15.67:0/2391960466 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56026b4be960 con 0x56026b3dc520 2024-09-21T06:39:29.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.327+0000 7ff1937fe640 1 --2- 172.21.15.67:0/2391960466 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56026b3b3660 0x56026b78e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:39:29.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.327+0000 7ff192ffd640 1 --2- 172.21.15.67:0/2391960466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b3bb840 0x56026b78f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:39:29.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.327+0000 7ff193fff640 1 --2- 172.21.15.67:0/2391960466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56026b3dc520 0x56026b78acc0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00a010 tx=0x7ff18c012560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc33d7b62e647f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:39:29.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.327+0000 7ff1797fa640 1 -- 172.21.15.67:0/2391960466 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c027920 con 0x56026b3dc520 2024-09-21T06:39:29.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.327+0000 7ff1797fa640 1 -- 172.21.15.67:0/2391960466 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff18c027ac0 con 0x56026b3dc520 2024-09-21T06:39:29.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.327+0000 7ff1797fa640 1 -- 172.21.15.67:0/2391960466 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c01f8b0 con 0x56026b3dc520 2024-09-21T06:39:29.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.328+0000 7ff1a01c2740 1 -- 172.21.15.67:0/2391960466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56026b3dc520 msgr2=0x56026b78acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:29.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.328+0000 7ff1a01c2740 1 --2- 172.21.15.67:0/2391960466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56026b3dc520 0x56026b78acc0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00a010 tx=0x7ff18c012560 comp rx=0 tx=0).stop 2024-09-21T06:39:29.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.328+0000 7ff1a01c2740 1 -- 172.21.15.67:0/2391960466 shutdown_connections 2024-09-21T06:39:29.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.328+0000 7ff1a01c2740 1 --2- 172.21.15.67:0/2391960466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b3bb840 0x56026b78f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.328+0000 7ff1a01c2740 1 --2- 172.21.15.67:0/2391960466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56026b3dc520 0x56026b78acc0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.328+0000 7ff1a01c2740 1 --2- 172.21.15.67:0/2391960466 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56026b3b3660 0x56026b78e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.328+0000 7ff1a01c2740 1 -- 172.21.15.67:0/2391960466 >> 172.21.15.67:0/2391960466 conn(0x56026b3d0910 msgr2=0x56026b78eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:39:29.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.328+0000 7ff1a01c2740 1 -- 172.21.15.67:0/2391960466 shutdown_connections 2024-09-21T06:39:29.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.328+0000 7ff1a01c2740 1 -- 172.21.15.67:0/2391960466 wait complete. 2024-09-21T06:39:29.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.328+0000 7ff1a01c2740 1 -- start start 2024-09-21T06:39:29.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.329+0000 7ff1a01c2740 1 Processor -- start 2024-09-21T06:39:29.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.329+0000 7ff1a01c2740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56026b3b3660 0x56026b7963d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:29.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.329+0000 7ff1a01c2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b3bb840 0x56026b7938a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:29.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.329+0000 7ff1a01c2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56026b3dc520 0x56026b793de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:29.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.329+0000 7ff1a01c2740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff1941b18a0 con 0x56026b3b3660 2024-09-21T06:39:29.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.329+0000 7ff1a01c2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff1941b2d10 con 0x56026b3bb840 2024-09-21T06:39:29.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.329+0000 7ff1a01c2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff1941b1f70 con 0x56026b3dc520 2024-09-21T06:39:29.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.329+0000 7ff1937fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56026b3b3660 0x56026b7963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:29.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.329+0000 7ff1937fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56026b3b3660 0x56026b7963d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34076/0 (socket says 172.21.15.67:34076) 2024-09-21T06:39:29.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.329+0000 7ff1937fe640 1 -- 172.21.15.67:0/2219764940 learned_addr learned my addr 172.21.15.67:0/2219764940 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:39:29.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.329+0000 7ff1a01c2740 10 client.0 _ll_register_callbacks cb 0x7ff18c009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:39:29.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.329+0000 7ff193fff640 1 --2- 172.21.15.67:0/2219764940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56026b3dc520 0x56026b793de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:29.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.329+0000 7ff192ffd640 1 --2- 172.21.15.67:0/2219764940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b3bb840 0x56026b7938a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:29.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.330+0000 7ff1937fe640 1 -- 172.21.15.67:0/2219764940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56026b3dc520 msgr2=0x56026b793de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:39:29.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.330+0000 7ff1937fe640 1 --2- 172.21.15.67:0/2219764940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56026b3dc520 0x56026b793de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.330+0000 7ff1937fe640 1 -- 172.21.15.67:0/2219764940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b3bb840 msgr2=0x56026b7938a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:39:29.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.330+0000 7ff1937fe640 1 --2- 172.21.15.67:0/2219764940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b3bb840 0x56026b7938a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.330+0000 7ff1937fe640 1 -- 172.21.15.67:0/2219764940 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56026b7943a0 con 0x56026b3b3660 2024-09-21T06:39:29.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.330+0000 7ff1937fe640 1 --2- 172.21.15.67:0/2219764940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56026b3b3660 0x56026b7963d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff1840ad6c0 tx=0x7ff18400fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e56196fb3e8bfae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:39:29.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.330+0000 7ff172ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:39:29.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.330+0000 7ff172ffd640 1 -- 172.21.15.67:0/2219764940 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1840b7150 con 0x56026b3b3660 2024-09-21T06:39:29.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.330+0000 7ff172ffd640 1 -- 172.21.15.67:0/2219764940 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7ff1840a99a0 con 0x56026b3b3660 2024-09-21T06:39:29.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.330+0000 7ff1a01c2740 1 -- 172.21.15.67:0/2219764940 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff18c027ac0 con 0x56026b3b3660 2024-09-21T06:39:29.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.330+0000 7ff172ffd640 1 -- 172.21.15.67:0/2219764940 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff1840b6cc0 con 0x56026b3b3660 2024-09-21T06:39:29.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.330+0000 7ff1a01c2740 10 client.15573 fetch_fsmap learned FSMap version 21 2024-09-21T06:39:29.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.330+0000 7ff1a01c2740 1 -- 172.21.15.67:0/2219764940 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56026b80a650 con 0x56026b3b3660 2024-09-21T06:39:29.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.330+0000 7ff172ffd640 1 -- 172.21.15.67:0/2219764940 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff18c027ac0 con 0x56026b3b3660 2024-09-21T06:39:29.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.331+0000 7ff172ffd640 1 -- 172.21.15.67:0/2219764940 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ff1840a9260 con 0x56026b3b3660 2024-09-21T06:39:29.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.331+0000 7ff1a01c2740 10 client.15573 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:39:29.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.331+0000 7ff1a01c2740 10 client.15573 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:39:29.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.331+0000 7ff1a01c2740 20 client.15573 populate_metadata read hostname 'smithi067' 2024-09-21T06:39:29.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.331+0000 7ff1a01c2740 1 --2- 172.21.15.67:0/2219764940 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x56026b370250 0x56026b80cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:29.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.331+0000 7ff1a01c2740 4 client.15573 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "aa36f322", "release": true, "await": true}] 2024-09-21T06:39:29.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.331+0000 7ff1a01c2740 1 -- 172.21.15.67:0/2219764940 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "quiesce db", "set_id": "aa36f322", "release": true, "await": true}) -- 0x56026b6f5020 con 0x56026b370250 2024-09-21T06:39:29.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.331+0000 7ff192ffd640 1 --2- 172.21.15.67:0/2219764940 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x56026b370250 0x56026b80cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:29.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.331+0000 7ff172ffd640 10 client.15573 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:39:29.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.331+0000 7ff192ffd640 1 --2- 172.21.15.67:0/2219764940 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x56026b370250 0x56026b80cdc0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ff935fae4a9c010 server_cookie=5320abf23f702777 in_seq=0 out_seq=0 2024-09-21T06:39:29.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.335+0000 7ff172ffd640 1 -- 172.21.15.67:0/2219764940 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56026b6f5020 con 0x56026b370250 2024-09-21T06:39:29.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.335+0000 7ff172ffd640 10 client.15573 handle_command_reply: tid=0 2024-09-21T06:39:29.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.335+0000 7ff1a01c2740 1 client.15573 shutdown 2024-09-21T06:39:29.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.335+0000 7ff1a01c2740 1 -- 172.21.15.67:0/2219764940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56026b3b3660 msgr2=0x56026b7963d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:29.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.335+0000 7ff1a01c2740 1 --2- 172.21.15.67:0/2219764940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56026b3b3660 0x56026b7963d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff1840ad6c0 tx=0x7ff18400fd40 comp rx=0 tx=0).stop 2024-09-21T06:39:29.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.335+0000 7ff1a01c2740 1 -- 172.21.15.67:0/2219764940 shutdown_connections 2024-09-21T06:39:29.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.335+0000 7ff1a01c2740 1 --2- 172.21.15.67:0/2219764940 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x56026b370250 0x56026b80cdc0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.335+0000 7ff1a01c2740 1 --2- 172.21.15.67:0/2219764940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b3bb840 0x56026b7938a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.335+0000 7ff1a01c2740 1 --2- 172.21.15.67:0/2219764940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56026b3dc520 0x56026b793de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.335+0000 7ff1a01c2740 1 --2- 172.21.15.67:0/2219764940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56026b3b3660 0x56026b7963d0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.335+0000 7ff1a01c2740 1 -- 172.21.15.67:0/2219764940 >> 172.21.15.67:0/2219764940 conn(0x56026b3d0910 msgr2=0x56026b78e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:39:29.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.335+0000 7ff172ffd640 0 client.15573 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:39:29.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.335+0000 7ff1a01c2740 1 -- 172.21.15.67:0/2219764940 shutdown_connections 2024-09-21T06:39:29.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.335+0000 7ff1a01c2740 1 -- 172.21.15.67:0/2219764940 wait complete. 2024-09-21T06:39:29.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.336+0000 7ff1a01c2740 20 client.15573 trim_cache size 0 max 16384 2024-09-21T06:39:29.339 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-21T06:39:29.339 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 21, 2024-09-21T06:39:29.340 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-09-21T06:39:29.340 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 12, 2024-09-21T06:39:29.340 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-21T06:39:29.340 INFO:teuthology.orchestra.run.smithi067.stdout: "aa36f322": { 2024-09-21T06:39:29.340 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 12, 2024-09-21T06:39:29.340 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-21T06:39:29.340 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:39:29.340 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-21T06:39:29.340 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:39:29.340 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:39:29.340 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-21T06:39:29.340 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 147.8, 2024-09-21T06:39:29.340 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-21T06:39:29.340 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-21T06:39:29.341 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-21T06:39:29.341 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:39:29.341 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-21T06:39:29.341 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:39:29.341 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:39:29.341 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:39:29.341 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:39:29.341 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:39:29.341 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:39:29.341 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-21T06:39:29.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.336+0000 7ff199c4e640 1 -- 172.21.15.67:0/1860909310 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7ff17c0b6510 msgr2=0x7ff17c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:29.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.336+0000 7ff199c4e640 1 --2- 172.21.15.67:0/1860909310 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7ff17c0b6510 0x7ff17c0b8a20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff18c005fd0 tx=0x7ff18c009580 comp rx=0 tx=0).stop 2024-09-21T06:39:29.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.336+0000 7ff199c4e640 1 -- 172.21.15.67:0/1860909310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff194102ae0 msgr2=0x7ff19410ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:29.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.336+0000 7ff199c4e640 1 --2- 172.21.15.67:0/1860909310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff194102ae0 0x7ff19410ff00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff18400c920 tx=0x7ff18400cde0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.336+0000 7ff199c4e640 1 -- 172.21.15.67:0/1860909310 shutdown_connections 2024-09-21T06:39:29.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.336+0000 7ff199c4e640 1 --2- 172.21.15.67:0/1860909310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff194108d70 0x7ff194110980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.336+0000 7ff199c4e640 1 --2- 172.21.15.67:0/1860909310 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7ff17c0b6510 0x7ff17c0b8a20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.336+0000 7ff199c4e640 1 --2- 172.21.15.67:0/1860909310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff194108420 0x7ff194110440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.336+0000 7ff199c4e640 1 --2- 172.21.15.67:0/1860909310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff194102ae0 0x7ff19410ff00 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:29.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.336+0000 7ff199c4e640 1 -- 172.21.15.67:0/1860909310 >> 172.21.15.67:0/1860909310 conn(0x7ff194079770 msgr2=0x7ff194106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:39:29.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.337+0000 7ff199c4e640 1 -- 172.21.15.67:0/1860909310 shutdown_connections 2024-09-21T06:39:29.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:29.337+0000 7ff199c4e640 1 -- 172.21.15.67:0/1860909310 wait complete. 2024-09-21T06:39:29.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:29 smithi203 ceph-mon[33735]: pgmap v447: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 341 B/s wr, 1 op/s 2024-09-21T06:39:29.786 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:29 smithi067 ceph-mon[31232]: pgmap v447: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 341 B/s wr, 1 op/s 2024-09-21T06:39:29.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:29 smithi100 ceph-mon[33401]: pgmap v447: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 341 B/s wr, 1 op/s 2024-09-21T06:39:30.374 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'aa36f322', total seconds elapsed: 31.9 2024-09-21T06:39:30.375 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.8 2024-09-21T06:39:30.375 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:39:30.786 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:39:31.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.450+0000 7f5a40c20640 1 -- 172.21.15.67:0/2728691753 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a3c1094d0 msgr2=0x7f5a3c1098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:31.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.450+0000 7f5a40c20640 1 --2- 172.21.15.67:0/2728691753 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a3c1094d0 0x7f5a3c1098e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5a30009900 tx=0x7f5a3002a410 comp rx=0 tx=0).stop 2024-09-21T06:39:31.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.450+0000 7f5a40c20640 1 -- 172.21.15.67:0/2728691753 shutdown_connections 2024-09-21T06:39:31.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.450+0000 7f5a40c20640 1 --2- 172.21.15.67:0/2728691753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a3c109e20 0x7f5a3c110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:31.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.450+0000 7f5a40c20640 1 --2- 172.21.15.67:0/2728691753 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a3c1094d0 0x7f5a3c1098e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:31.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.450+0000 7f5a40c20640 1 --2- 172.21.15.67:0/2728691753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a3c103b90 0x7f5a3c103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:31.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.450+0000 7f5a40c20640 1 -- 172.21.15.67:0/2728691753 >> 172.21.15.67:0/2728691753 conn(0x7f5a3c0ff5e0 msgr2=0x7f5a3c101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:39:31.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.450+0000 7f5a40c20640 1 -- 172.21.15.67:0/2728691753 shutdown_connections 2024-09-21T06:39:31.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.450+0000 7f5a40c20640 1 -- 172.21.15.67:0/2728691753 wait complete. 2024-09-21T06:39:31.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.451+0000 7f5a40c20640 1 Processor -- start 2024-09-21T06:39:31.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.451+0000 7f5a40c20640 1 -- start start 2024-09-21T06:39:31.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.452+0000 7f5a40c20640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a3c103b90 0x7f5a3c077120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:31.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.452+0000 7f5a40c20640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a3c1094d0 0x7f5a3c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:31.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.452+0000 7f5a40c20640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a3c109e20 0x7f5a3c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:31.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.452+0000 7f5a40c20640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c1139e0 con 0x7f5a3c109e20 2024-09-21T06:39:31.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.452+0000 7f5a40c20640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c113860 con 0x7f5a3c1094d0 2024-09-21T06:39:31.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.452+0000 7f5a40c20640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c113b60 con 0x7f5a3c103b90 2024-09-21T06:39:31.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.452+0000 7f5a3ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a3c109e20 0x7f5a3c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:31.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.452+0000 7f5a3ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a3c109e20 0x7f5a3c075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34084/0 (socket says 172.21.15.67:34084) 2024-09-21T06:39:31.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.452+0000 7f5a39d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a3c1094d0 0x7f5a3c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:31.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.452+0000 7f5a3ad9d640 1 -- 172.21.15.67:0/1996396881 learned_addr learned my addr 172.21.15.67:0/1996396881 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:39:31.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.452+0000 7f5a3a59c640 1 --2- 172.21.15.67:0/1996396881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a3c103b90 0x7f5a3c077120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:31.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.453+0000 7f5a3ad9d640 1 -- 172.21.15.67:0/1996396881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a3c103b90 msgr2=0x7f5a3c077120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:31.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.453+0000 7f5a3ad9d640 1 --2- 172.21.15.67:0/1996396881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a3c103b90 0x7f5a3c077120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:31.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.453+0000 7f5a3ad9d640 1 -- 172.21.15.67:0/1996396881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a3c1094d0 msgr2=0x7f5a3c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:31.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.453+0000 7f5a3ad9d640 1 --2- 172.21.15.67:0/1996396881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a3c1094d0 0x7f5a3c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:31.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.453+0000 7f5a3ad9d640 1 -- 172.21.15.67:0/1996396881 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a3c076470 con 0x7f5a3c109e20 2024-09-21T06:39:31.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.453+0000 7f5a3a59c640 1 --2- 172.21.15.67:0/1996396881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a3c103b90 0x7f5a3c077120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:39:31.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.453+0000 7f5a3ad9d640 1 --2- 172.21.15.67:0/1996396881 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a3c109e20 0x7f5a3c075c50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5a3400eaa0 tx=0x7f5a3400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:39:31.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.453+0000 7f5a2b7fe640 1 -- 172.21.15.67:0/1996396881 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3400cdb0 con 0x7f5a3c109e20 2024-09-21T06:39:31.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.453+0000 7f5a2b7fe640 1 -- 172.21.15.67:0/1996396881 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a34005020 con 0x7f5a3c109e20 2024-09-21T06:39:31.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.453+0000 7f5a2b7fe640 1 -- 172.21.15.67:0/1996396881 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a340116d0 con 0x7f5a3c109e20 2024-09-21T06:39:31.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.453+0000 7f5a40c20640 1 -- 172.21.15.67:0/1996396881 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a3c1acd90 con 0x7f5a3c109e20 2024-09-21T06:39:31.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.454+0000 7f5a40c20640 1 -- 172.21.15.67:0/1996396881 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a3c1ad1f0 con 0x7f5a3c109e20 2024-09-21T06:39:31.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.455+0000 7f5a40c20640 1 -- 172.21.15.67:0/1996396881 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a3c103fa0 con 0x7f5a3c109e20 2024-09-21T06:39:31.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.457+0000 7f5a2b7fe640 1 -- 172.21.15.67:0/1996396881 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f5a340051d0 con 0x7f5a3c109e20 2024-09-21T06:39:31.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.458+0000 7f5a2b7fe640 1 --2- 172.21.15.67:0/1996396881 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5a240b6590 0x7f5a240b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:31.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.458+0000 7f5a3a59c640 1 --2- 172.21.15.67:0/1996396881 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5a240b6590 0x7f5a240b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:31.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.458+0000 7f5a2b7fe640 1 -- 172.21.15.67:0/1996396881 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f5a340e7c20 con 0x7f5a3c109e20 2024-09-21T06:39:31.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.459+0000 7f5a3a59c640 1 --2- 172.21.15.67:0/1996396881 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5a240b6590 0x7f5a240b8aa0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c005fd0 tx=0x7f5a2c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:39:31.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.464+0000 7f5a2b7fe640 1 -- 172.21.15.67:0/1996396881 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5a34016030 con 0x7f5a3c109e20 2024-09-21T06:39:31.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:31 smithi203 ceph-mon[33735]: pgmap v448: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 341 B/s wr, 1 op/s 2024-09-21T06:39:31.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:31 smithi067 ceph-mon[31232]: pgmap v448: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 341 B/s wr, 1 op/s 2024-09-21T06:39:31.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:31 smithi100 ceph-mon[33401]: pgmap v448: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 341 B/s wr, 1 op/s 2024-09-21T06:39:31.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.862+0000 7f5a40c20640 1 -- 172.21.15.67:0/1996396881 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a3c076a10 con 0x7f5a3c109e20 2024-09-21T06:39:31.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.863+0000 7f5a2b7fe640 1 -- 172.21.15.67:0/1996396881 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7f5a3c076a10 con 0x7f5a3c109e20 2024-09-21T06:39:31.865 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:39:31.865 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:39:31.865 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:39:31.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.866+0000 7f5a40c20640 1 -- 172.21.15.67:0/1996396881 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5a240b6590 msgr2=0x7f5a240b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:31.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.866+0000 7f5a40c20640 1 --2- 172.21.15.67:0/1996396881 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5a240b6590 0x7f5a240b8aa0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c005fd0 tx=0x7f5a2c005850 comp rx=0 tx=0).stop 2024-09-21T06:39:31.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.866+0000 7f5a40c20640 1 -- 172.21.15.67:0/1996396881 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a3c109e20 msgr2=0x7f5a3c075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:31.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.866+0000 7f5a40c20640 1 --2- 172.21.15.67:0/1996396881 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a3c109e20 0x7f5a3c075c50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5a3400eaa0 tx=0x7f5a3400ef60 comp rx=0 tx=0).stop 2024-09-21T06:39:31.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.866+0000 7f5a40c20640 1 -- 172.21.15.67:0/1996396881 shutdown_connections 2024-09-21T06:39:31.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.866+0000 7f5a40c20640 1 --2- 172.21.15.67:0/1996396881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a3c1094d0 0x7f5a3c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:31.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.866+0000 7f5a40c20640 1 --2- 172.21.15.67:0/1996396881 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5a240b6590 0x7f5a240b8aa0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:31.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.866+0000 7f5a40c20640 1 --2- 172.21.15.67:0/1996396881 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a3c109e20 0x7f5a3c075c50 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:31.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.866+0000 7f5a40c20640 1 --2- 172.21.15.67:0/1996396881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a3c103b90 0x7f5a3c077120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:31.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.866+0000 7f5a40c20640 1 -- 172.21.15.67:0/1996396881 >> 172.21.15.67:0/1996396881 conn(0x7f5a3c0ff5e0 msgr2=0x7f5a3c101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:39:31.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.866+0000 7f5a40c20640 1 -- 172.21.15.67:0/1996396881 shutdown_connections 2024-09-21T06:39:31.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:31.866+0000 7f5a40c20640 1 -- 172.21.15.67:0/1996396881 wait complete. 2024-09-21T06:39:32.541 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:32 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1996396881' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:39:32.714 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0' 2024-09-21T06:39:32.714 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24671 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0 2024-09-21T06:39:32.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:32 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1996396881' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:39:32.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:32 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1996396881' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:39:33.094 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:39:33.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.688+0000 7f21e3c65640 1 -- 172.21.15.67:0/813176045 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21dc075750 msgr2=0x7f21dc075b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:33.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.688+0000 7f21e3c65640 1 --2- 172.21.15.67:0/813176045 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21dc075750 0x7f21dc075b60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f21d8009900 tx=0x7f21d802a410 comp rx=0 tx=0).stop 2024-09-21T06:39:33.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.688+0000 7f21e3c65640 1 -- 172.21.15.67:0/813176045 shutdown_connections 2024-09-21T06:39:33.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.688+0000 7f21e3c65640 1 --2- 172.21.15.67:0/813176045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21dc0760a0 0x7f21dc10fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.688+0000 7f21e3c65640 1 --2- 172.21.15.67:0/813176045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21dc110150 0x7f21dc112570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.688+0000 7f21e3c65640 1 --2- 172.21.15.67:0/813176045 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21dc075750 0x7f21dc075b60 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.688+0000 7f21e3c65640 1 -- 172.21.15.67:0/813176045 >> 172.21.15.67:0/813176045 conn(0x7f21dc0ff5e0 msgr2=0x7f21dc101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:39:33.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.688+0000 7f21e3c65640 1 -- 172.21.15.67:0/813176045 shutdown_connections 2024-09-21T06:39:33.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.688+0000 7f21e3c65640 1 -- 172.21.15.67:0/813176045 wait complete. 2024-09-21T06:39:33.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.689+0000 7f21e3c65640 1 Processor -- start 2024-09-21T06:39:33.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.689+0000 7f21e3c65640 1 -- start start 2024-09-21T06:39:33.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.689+0000 7f21e3c65640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21dc075750 0x7f21dc1075e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:33.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.689+0000 7f21e3c65640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21dc0760a0 0x7f21dc1038c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:33.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.689+0000 7f21e3c65640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21dc110150 0x7f21dc103fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:33.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.689+0000 7f21e3c65640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc115b70 con 0x7f21dc110150 2024-09-21T06:39:33.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.689+0000 7f21e3c65640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc1159f0 con 0x7f21dc075750 2024-09-21T06:39:33.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.689+0000 7f21e3c65640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc115cf0 con 0x7f21dc0760a0 2024-09-21T06:39:33.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.690+0000 7f21e2202640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21dc110150 0x7f21dc103fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:33.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.690+0000 7f21e1a01640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21dc075750 0x7f21dc1075e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:33.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.690+0000 7f21e1a01640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21dc075750 0x7f21dc1075e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.67:52414/0 (socket says 172.21.15.67:52414) 2024-09-21T06:39:33.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.690+0000 7f21e1a01640 1 -- 172.21.15.67:0/4187326627 learned_addr learned my addr 172.21.15.67:0/4187326627 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:39:33.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.690+0000 7f21e2202640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21dc110150 0x7f21dc103fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34094/0 (socket says 172.21.15.67:34094) 2024-09-21T06:39:33.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.690+0000 7f21e1200640 1 --2- 172.21.15.67:0/4187326627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21dc0760a0 0x7f21dc1038c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:33.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.690+0000 7f21e1a01640 1 -- 172.21.15.67:0/4187326627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21dc0760a0 msgr2=0x7f21dc1038c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:33.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.690+0000 7f21e1a01640 1 --2- 172.21.15.67:0/4187326627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21dc0760a0 0x7f21dc1038c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.690+0000 7f21e1a01640 1 -- 172.21.15.67:0/4187326627 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21dc110150 msgr2=0x7f21dc103fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:33.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.690+0000 7f21e1a01640 1 --2- 172.21.15.67:0/4187326627 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21dc110150 0x7f21dc103fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.690+0000 7f21e1a01640 1 -- 172.21.15.67:0/4187326627 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21dc1acb30 con 0x7f21dc075750 2024-09-21T06:39:33.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.690+0000 7f21e2202640 1 --2- 172.21.15.67:0/4187326627 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21dc110150 0x7f21dc103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:39:33.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.690+0000 7f21e1a01640 1 --2- 172.21.15.67:0/4187326627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21dc075750 0x7f21dc1075e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f21d8009230 tx=0x7f21d80319a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:39:33.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.691+0000 7f21d2ffd640 1 -- 172.21.15.67:0/4187326627 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d803f030 con 0x7f21dc075750 2024-09-21T06:39:33.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.691+0000 7f21d2ffd640 1 -- 172.21.15.67:0/4187326627 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21d80043d0 con 0x7f21dc075750 2024-09-21T06:39:33.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.691+0000 7f21e3c65640 1 -- 172.21.15.67:0/4187326627 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21dc1acd00 con 0x7f21dc075750 2024-09-21T06:39:33.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.691+0000 7f21d2ffd640 1 -- 172.21.15.67:0/4187326627 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d8035070 con 0x7f21dc075750 2024-09-21T06:39:33.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.691+0000 7f21e3c65640 1 -- 172.21.15.67:0/4187326627 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21dc1ad190 con 0x7f21dc075750 2024-09-21T06:39:33.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.694+0000 7f21d2ffd640 1 -- 172.21.15.67:0/4187326627 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f21d8042030 con 0x7f21dc075750 2024-09-21T06:39:33.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.695+0000 7f21ea1d9740 1 Processor -- start 2024-09-21T06:39:33.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.695+0000 7f21ea1d9740 1 -- start start 2024-09-21T06:39:33.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.695+0000 7f21d2ffd640 1 --2- 172.21.15.67:0/4187326627 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f21c80b6510 0x7f21c80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:33.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.696+0000 7f21e2202640 1 --2- 172.21.15.67:0/4187326627 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f21c80b6510 0x7f21c80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:33.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.696+0000 7f21ea1d9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c381c21840 0x55c381fc8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:33.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.696+0000 7f21d2ffd640 1 -- 172.21.15.67:0/4187326627 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f21d8107ac0 con 0x7f21dc075750 2024-09-21T06:39:33.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.696+0000 7f21ea1d9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c381bf8980 0x55c381fc8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:33.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.696+0000 7f21ea1d9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c381c00b60 0x55c381fc9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:33.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.696+0000 7f21ea1d9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c381f3eb90 con 0x55c381c00b60 2024-09-21T06:39:33.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.696+0000 7f21ea1d9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c381f2dfc0 con 0x55c381bf8980 2024-09-21T06:39:33.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.696+0000 7f21ea1d9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c381f31dd0 con 0x55c381c21840 2024-09-21T06:39:33.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.696+0000 7f21e2202640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c381c00b60 0x55c381fc9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:33.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.696+0000 7f21e2202640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c381c00b60 0x55c381fc9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34104/0 (socket says 172.21.15.67:34104) 2024-09-21T06:39:33.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.696+0000 7f21e1200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c381bf8980 0x55c381fc8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:33.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.696+0000 7f21e2202640 1 -- 172.21.15.67:0/1458470954 learned_addr learned my addr 172.21.15.67:0/1458470954 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:39:33.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.697+0000 7f21e1200640 1 -- 172.21.15.67:0/1458470954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c381c21840 msgr2=0x55c381fc8600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:39:33.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.697+0000 7f21e1200640 1 --2- 172.21.15.67:0/1458470954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c381c21840 0x55c381fc8600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.697+0000 7f21e1200640 1 -- 172.21.15.67:0/1458470954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c381c00b60 msgr2=0x55c381fc9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:39:33.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.697+0000 7f21e1200640 1 --2- 172.21.15.67:0/1458470954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c381c00b60 0x55c381fc9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.697+0000 7f21e1200640 1 -- 172.21.15.67:0/1458470954 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c381eb5060 con 0x55c381bf8980 2024-09-21T06:39:33.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.697+0000 7f21e1200640 1 --2- 172.21.15.67:0/1458470954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c381bf8980 0x55c381fc8b40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f21dc06bad0 tx=0x7f21cc0113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15d35cf677ab5acb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:39:33.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.697+0000 7f21c7fff640 1 -- 172.21.15.67:0/1458470954 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc011de0 con 0x55c381bf8980 2024-09-21T06:39:33.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.697+0000 7f21c7fff640 1 -- 172.21.15.67:0/1458470954 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21cc0128e0 con 0x55c381bf8980 2024-09-21T06:39:33.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.698+0000 7f21c7fff640 1 -- 172.21.15.67:0/1458470954 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc00f8c0 con 0x55c381bf8980 2024-09-21T06:39:33.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.699+0000 7f21ea1d9740 1 -- 172.21.15.67:0/1458470954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c381bf8980 msgr2=0x55c381fc8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:33.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.699+0000 7f21e2202640 1 --2- 172.21.15.67:0/4187326627 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f21c80b6510 0x7f21c80b8a20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f21d400f5b0 tx=0x7f21d4011c30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:39:33.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.699+0000 7f21ea1d9740 1 --2- 172.21.15.67:0/1458470954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c381bf8980 0x55c381fc8b40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f21dc06bad0 tx=0x7f21cc0113e0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.699+0000 7f21ea1d9740 1 -- 172.21.15.67:0/1458470954 shutdown_connections 2024-09-21T06:39:33.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.699+0000 7f21ea1d9740 1 --2- 172.21.15.67:0/1458470954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c381bf8980 0x55c381fc8b40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.699+0000 7f21ea1d9740 1 --2- 172.21.15.67:0/1458470954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c381c00b60 0x55c381fc9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.699+0000 7f21ea1d9740 1 --2- 172.21.15.67:0/1458470954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c381c21840 0x55c381fc8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.699+0000 7f21ea1d9740 1 -- 172.21.15.67:0/1458470954 >> 172.21.15.67:0/1458470954 conn(0x55c381c15990 msgr2=0x55c381fc64f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:39:33.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.699+0000 7f21ea1d9740 1 -- 172.21.15.67:0/1458470954 shutdown_connections 2024-09-21T06:39:33.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.699+0000 7f21ea1d9740 1 -- 172.21.15.67:0/1458470954 wait complete. 2024-09-21T06:39:33.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.700+0000 7f21ea1d9740 1 -- start start 2024-09-21T06:39:33.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.700+0000 7f21ea1d9740 1 Processor -- start 2024-09-21T06:39:33.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.701+0000 7f21ea1d9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c381bf8980 0x55c381fd4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:33.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.701+0000 7f21ea1d9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c381c00b60 0x55c381fd4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:33.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.701+0000 7f21ea1d9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c381c21840 0x55c381fe0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:33.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.701+0000 7f21e1200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c381c00b60 0x55c381fd4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:33.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.701+0000 7f21ea1d9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c381d8f7a0 con 0x55c381c21840 2024-09-21T06:39:33.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.701+0000 7f21ea1d9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c381ed30a0 con 0x55c381bf8980 2024-09-21T06:39:33.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.701+0000 7f21ea1d9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c381fd6de0 con 0x55c381c00b60 2024-09-21T06:39:33.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.701+0000 7f21e2202640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c381c21840 0x55c381fe0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:33.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.701+0000 7f21e1200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c381c00b60 0x55c381fd4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.67:36758/0 (socket says 172.21.15.67:36758) 2024-09-21T06:39:33.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.701+0000 7f21e1200640 1 -- 172.21.15.67:0/1139869242 learned_addr learned my addr 172.21.15.67:0/1139869242 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:39:33.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.701+0000 7f21e1a01640 1 --2- 172.21.15.67:0/1139869242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c381bf8980 0x55c381fd4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:33.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.701+0000 7f21ea1d9740 10 client.0 _ll_register_callbacks cb 0x55c381eb03c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:39:33.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.702+0000 7f21e1200640 1 -- 172.21.15.67:0/1139869242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c381bf8980 msgr2=0x55c381fd4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:39:33.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.702+0000 7f21e1200640 1 --2- 172.21.15.67:0/1139869242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c381bf8980 0x55c381fd4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.702+0000 7f21e1200640 1 -- 172.21.15.67:0/1139869242 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c381c21840 msgr2=0x55c381fe0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:39:33.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.702+0000 7f21e1200640 1 --2- 172.21.15.67:0/1139869242 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c381c21840 0x55c381fe0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.702+0000 7f21e1200640 1 -- 172.21.15.67:0/1139869242 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c381ec1880 con 0x55c381c00b60 2024-09-21T06:39:33.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.702+0000 7f21e2202640 1 --2- 172.21.15.67:0/1139869242 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c381c21840 0x55c381fe0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:39:33.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.702+0000 7f21e1200640 1 --2- 172.21.15.67:0/1139869242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c381c00b60 0x55c381fd4800 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00f480 tx=0x7f21cc012680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80dc5e80041120b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:39:33.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.702+0000 7f21c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T06:39:33.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.702+0000 7f21c57fa640 1 -- 172.21.15.67:0/1139869242 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc005030 con 0x55c381c00b60 2024-09-21T06:39:33.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.703+0000 7f21c57fa640 1 -- 172.21.15.67:0/1139869242 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f21cc0051d0 con 0x55c381c00b60 2024-09-21T06:39:33.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.703+0000 7f21ea1d9740 1 -- 172.21.15.67:0/1139869242 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21cc0128e0 con 0x55c381c00b60 2024-09-21T06:39:33.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.703+0000 7f21c57fa640 1 -- 172.21.15.67:0/1139869242 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f21cc032050 con 0x55c381c00b60 2024-09-21T06:39:33.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.703+0000 7f21ea1d9740 10 client.25288 fetch_fsmap learned FSMap version 21 2024-09-21T06:39:33.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.703+0000 7f21ea1d9740 1 -- 172.21.15.67:0/1139869242 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c381c9bb90 con 0x55c381c00b60 2024-09-21T06:39:33.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.703+0000 7f21c57fa640 1 -- 172.21.15.67:0/1139869242 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21cc0128e0 con 0x55c381c00b60 2024-09-21T06:39:33.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.703+0000 7f21c57fa640 1 -- 172.21.15.67:0/1139869242 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f21cc00f5c0 con 0x55c381c00b60 2024-09-21T06:39:33.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.703+0000 7f21ea1d9740 10 client.25288 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:39:33.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.703+0000 7f21ea1d9740 10 client.25288 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:39:33.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.703+0000 7f21ea1d9740 20 client.25288 populate_metadata read hostname 'smithi067' 2024-09-21T06:39:33.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.703+0000 7f21ea1d9740 1 --2- 172.21.15.67:0/1139869242 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55c381be01c0 0x55c381fe60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:33.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.704+0000 7f21ea1d9740 4 client.25288 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:39:33.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.704+0000 7f21ea1d9740 1 -- 172.21.15.67:0/1139869242 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c381f40ae0 con 0x55c381be01c0 2024-09-21T06:39:33.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.704+0000 7f21e1a01640 1 --2- 172.21.15.67:0/1139869242 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55c381be01c0 0x55c381fe60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:33.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.705+0000 7f21c57fa640 10 client.25288 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:39:33.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.705+0000 7f21e1a01640 1 --2- 172.21.15.67:0/1139869242 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55c381be01c0 0x55c381fe60b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e21f83f6fda7447b server_cookie=19993eb151614fb8 in_seq=0 out_seq=0 2024-09-21T06:39:33.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.705+0000 7f21c57fa640 1 -- 172.21.15.67:0/1139869242 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c381f40ae0 con 0x55c381be01c0 2024-09-21T06:39:33.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.705+0000 7f21c57fa640 10 client.25288 handle_command_reply: tid=0 2024-09-21T06:39:33.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.705+0000 7f21ea1d9740 1 client.25288 shutdown 2024-09-21T06:39:33.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.706+0000 7f21ea1d9740 1 -- 172.21.15.67:0/1139869242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c381c00b60 msgr2=0x55c381fd4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:33.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.706+0000 7f21ea1d9740 1 --2- 172.21.15.67:0/1139869242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c381c00b60 0x55c381fd4800 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00f480 tx=0x7f21cc012680 comp rx=0 tx=0).stop 2024-09-21T06:39:33.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.706+0000 7f21ea1d9740 1 -- 172.21.15.67:0/1139869242 shutdown_connections 2024-09-21T06:39:33.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.706+0000 7f21ea1d9740 1 --2- 172.21.15.67:0/1139869242 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55c381be01c0 0x55c381fe60b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.706+0000 7f21ea1d9740 1 --2- 172.21.15.67:0/1139869242 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c381c21840 0x55c381fe0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.706+0000 7f21c57fa640 0 client.25288 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:39:33.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.706+0000 7f21ea1d9740 1 --2- 172.21.15.67:0/1139869242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c381c00b60 0x55c381fd4800 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.706+0000 7f21ea1d9740 1 --2- 172.21.15.67:0/1139869242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c381bf8980 0x55c381fd4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.706+0000 7f21ea1d9740 1 -- 172.21.15.67:0/1139869242 >> 172.21.15.67:0/1139869242 conn(0x55c381c15990 msgr2=0x55c381fcb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:39:33.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.706+0000 7f21ea1d9740 1 -- 172.21.15.67:0/1139869242 shutdown_connections 2024-09-21T06:39:33.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.706+0000 7f21ea1d9740 1 -- 172.21.15.67:0/1139869242 wait complete. 2024-09-21T06:39:33.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.706+0000 7f21ea1d9740 20 client.25288 trim_cache size 0 max 16384 2024-09-21T06:39:33.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:33 smithi203 ceph-mon[33735]: pgmap v449: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 15 MiB/s wr, 10 op/s 2024-09-21T06:39:33.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.737+0000 7f21ea1d9740 1 Processor -- start 2024-09-21T06:39:33.739 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:33 smithi067 ceph-mon[31232]: pgmap v449: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 15 MiB/s wr, 10 op/s 2024-09-21T06:39:33.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.738+0000 7f21ea1d9740 1 -- start start 2024-09-21T06:39:33.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.738+0000 7f21ea1d9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c381be01c0 0x55c381fe0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:33.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.738+0000 7f21ea1d9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c381bf8980 0x55c381fe05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:33.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.738+0000 7f21ea1d9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c381c00b60 0x55c381fd1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:33.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.738+0000 7f21ea1d9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc1b50f0 con 0x55c381be01c0 2024-09-21T06:39:33.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.738+0000 7f21ea1d9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc1b7970 con 0x55c381c00b60 2024-09-21T06:39:33.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.738+0000 7f21ea1d9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc1b5760 con 0x55c381bf8980 2024-09-21T06:39:33.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.738+0000 7f21e1200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c381bf8980 0x55c381fe05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:33.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.738+0000 7f21e1200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c381bf8980 0x55c381fe05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.67:36772/0 (socket says 172.21.15.67:36772) 2024-09-21T06:39:33.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.738+0000 7f21e1200640 1 -- 172.21.15.67:0/2803679660 learned_addr learned my addr 172.21.15.67:0/2803679660 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:39:33.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.738+0000 7f21e1a01640 1 --2- 172.21.15.67:0/2803679660 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c381be01c0 0x55c381fe0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:33.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.738+0000 7f21e2202640 1 --2- 172.21.15.67:0/2803679660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c381c00b60 0x55c381fd1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:33.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.739+0000 7f21e1200640 1 -- 172.21.15.67:0/2803679660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c381c00b60 msgr2=0x55c381fd1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:39:33.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.739+0000 7f21e1200640 1 --2- 172.21.15.67:0/2803679660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c381c00b60 0x55c381fd1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.739+0000 7f21e1200640 1 -- 172.21.15.67:0/2803679660 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c381be01c0 msgr2=0x55c381fe0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:39:33.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.739+0000 7f21e1200640 1 --2- 172.21.15.67:0/2803679660 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c381be01c0 0x55c381fe0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.739+0000 7f21e1200640 1 -- 172.21.15.67:0/2803679660 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c381fcabe0 con 0x55c381bf8980 2024-09-21T06:39:33.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.739+0000 7f21e1a01640 1 --2- 172.21.15.67:0/2803679660 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c381be01c0 0x55c381fe0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:39:33.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.739+0000 7f21e1200640 1 --2- 172.21.15.67:0/2803679660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c381bf8980 0x55c381fe05b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00fb50 tx=0x7f21cc0092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f1ecfdac3485a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:39:33.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.739+0000 7f21c77fe640 1 -- 172.21.15.67:0/2803679660 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc012860 con 0x55c381bf8980 2024-09-21T06:39:33.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.739+0000 7f21c77fe640 1 -- 172.21.15.67:0/2803679660 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21cc029070 con 0x55c381bf8980 2024-09-21T06:39:33.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.739+0000 7f21c77fe640 1 -- 172.21.15.67:0/2803679660 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc00f710 con 0x55c381bf8980 2024-09-21T06:39:33.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.741+0000 7f21ea1d9740 1 -- 172.21.15.67:0/2803679660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c381bf8980 msgr2=0x55c381fe05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:33.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.741+0000 7f21ea1d9740 1 --2- 172.21.15.67:0/2803679660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c381bf8980 0x55c381fe05b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00fb50 tx=0x7f21cc0092e0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.741+0000 7f21ea1d9740 1 -- 172.21.15.67:0/2803679660 shutdown_connections 2024-09-21T06:39:33.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.741+0000 7f21ea1d9740 1 --2- 172.21.15.67:0/2803679660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c381c00b60 0x55c381fd1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.741+0000 7f21ea1d9740 1 --2- 172.21.15.67:0/2803679660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c381bf8980 0x55c381fe05b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.741+0000 7f21ea1d9740 1 --2- 172.21.15.67:0/2803679660 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c381be01c0 0x55c381fe0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.741+0000 7f21ea1d9740 1 -- 172.21.15.67:0/2803679660 >> 172.21.15.67:0/2803679660 conn(0x55c381c15990 msgr2=0x55c381fd4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:39:33.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.741+0000 7f21ea1d9740 1 -- 172.21.15.67:0/2803679660 shutdown_connections 2024-09-21T06:39:33.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.741+0000 7f21ea1d9740 1 -- 172.21.15.67:0/2803679660 wait complete. 2024-09-21T06:39:33.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.741+0000 7f21ea1d9740 1 -- start start 2024-09-21T06:39:33.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.742+0000 7f21ea1d9740 1 Processor -- start 2024-09-21T06:39:33.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.742+0000 7f21ea1d9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c381be01c0 0x55c381fe9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:33.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.742+0000 7f21ea1d9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c381bf8980 0x55c381fe7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:33.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.742+0000 7f21ea1d9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c381c00b60 0x55c381fe8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:33.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.742+0000 7f21e1a01640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c381be01c0 0x55c381fe9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:33.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.743+0000 7f21e1a01640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c381be01c0 0x55c381fe9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34144/0 (socket says 172.21.15.67:34144) 2024-09-21T06:39:33.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.743+0000 7f21e1a01640 1 -- 172.21.15.67:0/1200755277 learned_addr learned my addr 172.21.15.67:0/1200755277 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:39:33.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.743+0000 7f21ea1d9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc1b5e30 con 0x55c381be01c0 2024-09-21T06:39:33.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.743+0000 7f21ea1d9740 1 -- 172.21.15.67:0/1200755277 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc1b72a0 con 0x55c381bf8980 2024-09-21T06:39:33.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.743+0000 7f21ea1d9740 1 -- 172.21.15.67:0/1200755277 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc1b6500 con 0x55c381c00b60 2024-09-21T06:39:33.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.743+0000 7f21e1200640 1 --2- 172.21.15.67:0/1200755277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c381bf8980 0x55c381fe7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:33.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.743+0000 7f21e2202640 1 --2- 172.21.15.67:0/1200755277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c381c00b60 0x55c381fe8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:33.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.743+0000 7f21ea1d9740 10 client.0 _ll_register_callbacks cb 0x7f21d4011b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:39:33.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.743+0000 7f21e1200640 1 -- 172.21.15.67:0/1200755277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c381c00b60 msgr2=0x55c381fe8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:39:33.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.743+0000 7f21e1200640 1 --2- 172.21.15.67:0/1200755277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c381c00b60 0x55c381fe8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.743+0000 7f21e1200640 1 -- 172.21.15.67:0/1200755277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c381be01c0 msgr2=0x55c381fe9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:39:33.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.743+0000 7f21e1200640 1 --2- 172.21.15.67:0/1200755277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c381be01c0 0x55c381fe9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.743+0000 7f21e1200640 1 -- 172.21.15.67:0/1200755277 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c382050840 con 0x55c381bf8980 2024-09-21T06:39:33.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.743+0000 7f21e1a01640 1 --2- 172.21.15.67:0/1200755277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c381be01c0 0x55c381fe9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:39:33.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.744+0000 7f21e1200640 1 --2- 172.21.15.67:0/1200755277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c381bf8980 0x55c381fe7f30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f21cc012770 tx=0x7f21cc0119e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60f2e69b5239a57e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:39:33.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.744+0000 7f21c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-21T06:39:33.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.744+0000 7f21c57fa640 1 -- 172.21.15.67:0/1200755277 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc042020 con 0x55c381bf8980 2024-09-21T06:39:33.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.744+0000 7f21c57fa640 1 -- 172.21.15.67:0/1200755277 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f21cc00f640 con 0x55c381bf8980 2024-09-21T06:39:33.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.744+0000 7f21ea1d9740 1 -- 172.21.15.67:0/1200755277 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21cc029070 con 0x55c381bf8980 2024-09-21T06:39:33.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.744+0000 7f21c57fa640 1 -- 172.21.15.67:0/1200755277 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f21cc0050e0 con 0x55c381bf8980 2024-09-21T06:39:33.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.744+0000 7f21ea1d9740 10 client.25322 fetch_fsmap learned FSMap version 21 2024-09-21T06:39:33.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.744+0000 7f21ea1d9740 1 -- 172.21.15.67:0/1200755277 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c3820518f0 con 0x55c381bf8980 2024-09-21T06:39:33.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.745+0000 7f21c57fa640 1 -- 172.21.15.67:0/1200755277 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21cc029070 con 0x55c381bf8980 2024-09-21T06:39:33.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.745+0000 7f21c57fa640 1 -- 172.21.15.67:0/1200755277 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f21cc003a10 con 0x55c381bf8980 2024-09-21T06:39:33.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.745+0000 7f21ea1d9740 10 client.25322 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:39:33.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.745+0000 7f21ea1d9740 10 client.25322 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:39:33.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.745+0000 7f21ea1d9740 20 client.25322 populate_metadata read hostname 'smithi067' 2024-09-21T06:39:33.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.745+0000 7f21ea1d9740 1 --2- 172.21.15.67:0/1200755277 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55c381c21840 0x55c382054110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:33.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.745+0000 7f21ea1d9740 4 client.25322 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}] 2024-09-21T06:39:33.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.745+0000 7f21ea1d9740 1 -- 172.21.15.67:0/1200755277 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}) -- 0x55c381f3c590 con 0x55c381c21840 2024-09-21T06:39:33.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.745+0000 7f21e1a01640 1 --2- 172.21.15.67:0/1200755277 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55c381c21840 0x55c382054110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:33.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.746+0000 7f21c57fa640 10 client.25322 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:39:33.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.746+0000 7f21e1a01640 1 --2- 172.21.15.67:0/1200755277 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55c381c21840 0x55c382054110 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bff456e78525e48 server_cookie=8d9b678bf4eaef44 in_seq=0 out_seq=0 2024-09-21T06:39:33.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.781+0000 7f21c57fa640 1 -- 172.21.15.67:0/1200755277 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c381f3c590 con 0x55c381c21840 2024-09-21T06:39:33.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.781+0000 7f21c57fa640 10 client.25322 handle_command_reply: tid=0 2024-09-21T06:39:33.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.781+0000 7f21ea1d9740 1 client.25322 shutdown 2024-09-21T06:39:33.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.781+0000 7f21ea1d9740 1 -- 172.21.15.67:0/1200755277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c381bf8980 msgr2=0x55c381fe7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:33.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.781+0000 7f21ea1d9740 1 --2- 172.21.15.67:0/1200755277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c381bf8980 0x55c381fe7f30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f21cc012770 tx=0x7f21cc0119e0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.781+0000 7f21ea1d9740 1 -- 172.21.15.67:0/1200755277 shutdown_connections 2024-09-21T06:39:33.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.781+0000 7f21ea1d9740 1 --2- 172.21.15.67:0/1200755277 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55c381c21840 0x55c382054110 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.781+0000 7f21ea1d9740 1 --2- 172.21.15.67:0/1200755277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c381bf8980 0x55c381fe7f30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.781+0000 7f21c57fa640 0 client.25322 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:39:33.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.781+0000 7f21ea1d9740 1 --2- 172.21.15.67:0/1200755277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c381c00b60 0x55c381fe8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.781+0000 7f21ea1d9740 1 --2- 172.21.15.67:0/1200755277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c381be01c0 0x55c381fe9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.781+0000 7f21ea1d9740 1 -- 172.21.15.67:0/1200755277 >> 172.21.15.67:0/1200755277 conn(0x55c381c15990 msgr2=0x55c381fd4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:39:33.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.781+0000 7f21ea1d9740 1 -- 172.21.15.67:0/1200755277 shutdown_connections 2024-09-21T06:39:33.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.781+0000 7f21ea1d9740 1 -- 172.21.15.67:0/1200755277 wait complete. 2024-09-21T06:39:33.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.781+0000 7f21ea1d9740 20 client.25322 trim_cache size 0 max 16384 2024-09-21T06:39:33.784 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-21T06:39:33.784 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 21, 2024-09-21T06:39:33.784 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-09-21T06:39:33.784 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 14, 2024-09-21T06:39:33.784 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-21T06:39:33.784 INFO:teuthology.orchestra.run.smithi067.stdout: "f4328142": { 2024-09-21T06:39:33.784 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 14, 2024-09-21T06:39:33.785 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-21T06:39:33.785 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:39:33.785 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-21T06:39:33.785 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:39:33.785 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:39:33.785 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-21T06:39:33.785 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 147.8, 2024-09-21T06:39:33.785 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-21T06:39:33.785 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-21T06:39:33.785 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-21T06:39:33.785 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:39:33.785 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-21T06:39:33.785 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:39:33.785 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:39:33.786 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:39:33.786 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:39:33.786 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:39:33.786 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:39:33.786 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-21T06:39:33.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.782+0000 7f21e3c65640 1 -- 172.21.15.67:0/4187326627 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f21c80b6510 msgr2=0x7f21c80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:33.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.782+0000 7f21e3c65640 1 --2- 172.21.15.67:0/4187326627 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f21c80b6510 0x7f21c80b8a20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f21d400f5b0 tx=0x7f21d4011c30 comp rx=0 tx=0).stop 2024-09-21T06:39:33.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.782+0000 7f21e3c65640 1 -- 172.21.15.67:0/4187326627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21dc075750 msgr2=0x7f21dc1075e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:33.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.782+0000 7f21e3c65640 1 --2- 172.21.15.67:0/4187326627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21dc075750 0x7f21dc1075e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f21d8009230 tx=0x7f21d80319a0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.782+0000 7f21e3c65640 1 -- 172.21.15.67:0/4187326627 shutdown_connections 2024-09-21T06:39:33.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.782+0000 7f21e3c65640 1 --2- 172.21.15.67:0/4187326627 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f21c80b6510 0x7f21c80b8a20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.782+0000 7f21e3c65640 1 --2- 172.21.15.67:0/4187326627 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21dc110150 0x7f21dc103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.782+0000 7f21e3c65640 1 --2- 172.21.15.67:0/4187326627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21dc0760a0 0x7f21dc1038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.782+0000 7f21e3c65640 1 --2- 172.21.15.67:0/4187326627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21dc075750 0x7f21dc1075e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:33.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.782+0000 7f21e3c65640 1 -- 172.21.15.67:0/4187326627 >> 172.21.15.67:0/4187326627 conn(0x7f21dc0ff5e0 msgr2=0x7f21dc109cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:39:33.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.782+0000 7f21e3c65640 1 -- 172.21.15.67:0/4187326627 shutdown_connections 2024-09-21T06:39:33.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:33.782+0000 7f21e3c65640 1 -- 172.21.15.67:0/4187326627 wait complete. 2024-09-21T06:39:33.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:33 smithi100 ceph-mon[33401]: pgmap v449: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 15 MiB/s wr, 10 op/s 2024-09-21T06:39:34.625 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f4328142', quiesce took 4.3 seconds. Will release after: 23.5 2024-09-21T06:39:35.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:35 smithi203 ceph-mon[33735]: pgmap v450: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 22 MiB/s wr, 13 op/s 2024-09-21T06:39:35.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:35 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:39:35.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:35 smithi067 ceph-mon[31232]: pgmap v450: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 22 MiB/s wr, 13 op/s 2024-09-21T06:39:35.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:35 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:39:35.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:35 smithi100 ceph-mon[33401]: pgmap v450: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 22 MiB/s wr, 13 op/s 2024-09-21T06:39:35.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:35 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:39:36.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:36 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:39:36.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:36 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:39:36.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:36 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:39:36.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:36 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:39:36.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:36 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:39:36.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:36 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:39:36.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:36 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:39:36.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:36 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:39:36.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:36 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:39:37.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:37 smithi203 ceph-mon[33735]: pgmap v451: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 36 MiB/s wr, 21 op/s 2024-09-21T06:39:37.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:37 smithi067 ceph-mon[31232]: pgmap v451: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 36 MiB/s wr, 21 op/s 2024-09-21T06:39:37.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:37 smithi100 ceph-mon[33401]: pgmap v451: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 36 MiB/s wr, 21 op/s 2024-09-21T06:39:39.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:39 smithi067 ceph-mon[31232]: pgmap v452: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 51 MiB/s wr, 28 op/s 2024-09-21T06:39:39.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:39 smithi100 ceph-mon[33401]: pgmap v452: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 51 MiB/s wr, 28 op/s 2024-09-21T06:39:39.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:39 smithi203 ceph-mon[33735]: pgmap v452: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 51 MiB/s wr, 28 op/s 2024-09-21T06:39:41.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:41 smithi067 ceph-mon[31232]: pgmap v453: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 52 MiB/s wr, 28 op/s 2024-09-21T06:39:41.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:41 smithi100 ceph-mon[33401]: pgmap v453: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 52 MiB/s wr, 28 op/s 2024-09-21T06:39:41.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:41 smithi203 ceph-mon[33735]: pgmap v453: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 52 MiB/s wr, 28 op/s 2024-09-21T06:39:43.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:43 smithi067 ceph-mon[31232]: pgmap v454: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 52 MiB/s wr, 30 op/s 2024-09-21T06:39:43.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:43 smithi100 ceph-mon[33401]: pgmap v454: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 52 MiB/s wr, 30 op/s 2024-09-21T06:39:43.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:43 smithi203 ceph-mon[33735]: pgmap v454: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 52 MiB/s wr, 30 op/s 2024-09-21T06:39:45.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:45 smithi067 ceph-mon[31232]: pgmap v455: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 37 MiB/s wr, 21 op/s 2024-09-21T06:39:45.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:45 smithi100 ceph-mon[33401]: pgmap v455: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 37 MiB/s wr, 21 op/s 2024-09-21T06:39:45.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:45 smithi203 ceph-mon[33735]: pgmap v455: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 37 MiB/s wr, 21 op/s 2024-09-21T06:39:47.780 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:47 smithi067 ceph-mon[31232]: pgmap v456: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 30 MiB/s wr, 18 op/s 2024-09-21T06:39:47.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:47 smithi203 ceph-mon[33735]: pgmap v456: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 30 MiB/s wr, 18 op/s 2024-09-21T06:39:47.805 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:47 smithi100 ceph-mon[33401]: pgmap v456: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 30 MiB/s wr, 18 op/s 2024-09-21T06:39:49.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:49 smithi067 ceph-mon[31232]: pgmap v457: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 17 MiB/s wr, 10 op/s 2024-09-21T06:39:49.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:49 smithi100 ceph-mon[33401]: pgmap v457: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 17 MiB/s wr, 10 op/s 2024-09-21T06:39:49.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:49 smithi203 ceph-mon[33735]: pgmap v457: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 17 MiB/s wr, 10 op/s 2024-09-21T06:39:51.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:51 smithi067 ceph-mon[31232]: pgmap v458: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 1.3 MiB/s wr, 2 op/s 2024-09-21T06:39:51.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:51 smithi100 ceph-mon[33401]: pgmap v458: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 1.3 MiB/s wr, 2 op/s 2024-09-21T06:39:51.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:51 smithi203 ceph-mon[33735]: pgmap v458: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 1.3 MiB/s wr, 2 op/s 2024-09-21T06:39:53.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:53 smithi067 ceph-mon[31232]: pgmap v459: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 486 KiB/s wr, 3 op/s 2024-09-21T06:39:53.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:53 smithi100 ceph-mon[33401]: pgmap v459: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 486 KiB/s wr, 3 op/s 2024-09-21T06:39:53.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:53 smithi203 ceph-mon[33735]: pgmap v459: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 486 KiB/s wr, 3 op/s 2024-09-21T06:39:55.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:55 smithi067 ceph-mon[31232]: pgmap v460: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 107 KiB/s wr, 1 op/s 2024-09-21T06:39:55.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:55 smithi100 ceph-mon[33401]: pgmap v460: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 107 KiB/s wr, 1 op/s 2024-09-21T06:39:55.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:55 smithi203 ceph-mon[33735]: pgmap v460: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 107 KiB/s wr, 1 op/s 2024-09-21T06:39:57.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:57 smithi067 ceph-mon[31232]: pgmap v461: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:39:57.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:57 smithi100 ceph-mon[33401]: pgmap v461: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:39:57.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:57 smithi203 ceph-mon[33735]: pgmap v461: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:39:58.126 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:39:58.399 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:39:59.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.398+0000 7fe429f15640 1 -- 172.21.15.67:0/3052386687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe424108cf0 msgr2=0x7fe42410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:59.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.398+0000 7fe429f15640 1 --2- 172.21.15.67:0/3052386687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe424108cf0 0x7fe42410f520 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe41c009ca0 tx=0x7fe41c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:39:59.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.398+0000 7fe429f15640 1 -- 172.21.15.67:0/3052386687 shutdown_connections 2024-09-21T06:39:59.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.398+0000 7fe429f15640 1 --2- 172.21.15.67:0/3052386687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe424108cf0 0x7fe42410f520 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:59.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.398+0000 7fe429f15640 1 --2- 172.21.15.67:0/3052386687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4241083a0 0x7fe4241087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:59.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.398+0000 7fe429f15640 1 --2- 172.21.15.67:0/3052386687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe424102a60 0x7fe424102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:59.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.398+0000 7fe429f15640 1 -- 172.21.15.67:0/3052386687 >> 172.21.15.67:0/3052386687 conn(0x7fe424079730 msgr2=0x7fe424079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:39:59.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.398+0000 7fe429f15640 1 -- 172.21.15.67:0/3052386687 shutdown_connections 2024-09-21T06:39:59.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.398+0000 7fe429f15640 1 -- 172.21.15.67:0/3052386687 wait complete. 2024-09-21T06:39:59.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.399+0000 7fe429f15640 1 Processor -- start 2024-09-21T06:39:59.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.399+0000 7fe429f15640 1 -- start start 2024-09-21T06:39:59.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.399+0000 7fe429f15640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe424102a60 0x7fe4240ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:59.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.399+0000 7fe429f15640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4241083a0 0x7fe4240ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:59.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.399+0000 7fe429f15640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe424108cf0 0x7fe4241000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:59.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.399+0000 7fe429f15640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe424112860 con 0x7fe424108cf0 2024-09-21T06:39:59.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.399+0000 7fe429f15640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe4241126e0 con 0x7fe424102a60 2024-09-21T06:39:59.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.399+0000 7fe429f15640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe4241129e0 con 0x7fe4241083a0 2024-09-21T06:39:59.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.399+0000 7fe423fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe424108cf0 0x7fe4241000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:59.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.399+0000 7fe423fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe424108cf0 0x7fe4241000f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49520/0 (socket says 172.21.15.67:49520) 2024-09-21T06:39:59.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.399+0000 7fe423fff640 1 -- 172.21.15.67:0/2765813640 learned_addr learned my addr 172.21.15.67:0/2765813640 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:39:59.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.400+0000 7fe4237fe640 1 --2- 172.21.15.67:0/2765813640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe424102a60 0x7fe4240ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:59.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.400+0000 7fe422ffd640 1 --2- 172.21.15.67:0/2765813640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4241083a0 0x7fe4240ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:59.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.400+0000 7fe423fff640 1 -- 172.21.15.67:0/2765813640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4241083a0 msgr2=0x7fe4240ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:59.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.400+0000 7fe423fff640 1 --2- 172.21.15.67:0/2765813640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4241083a0 0x7fe4240ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:59.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.400+0000 7fe423fff640 1 -- 172.21.15.67:0/2765813640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe424102a60 msgr2=0x7fe4240ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:59.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.400+0000 7fe423fff640 1 --2- 172.21.15.67:0/2765813640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe424102a60 0x7fe4240ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:59.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.400+0000 7fe423fff640 1 -- 172.21.15.67:0/2765813640 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4241a8600 con 0x7fe424108cf0 2024-09-21T06:39:59.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.400+0000 7fe423fff640 1 --2- 172.21.15.67:0/2765813640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe424108cf0 0x7fe4241000f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe41c005e30 tx=0x7fe41c0093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:39:59.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.400+0000 7fe420ff9640 1 -- 172.21.15.67:0/2765813640 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c032d10 con 0x7fe424108cf0 2024-09-21T06:39:59.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.400+0000 7fe429f15640 1 -- 172.21.15.67:0/2765813640 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4241a8890 con 0x7fe424108cf0 2024-09-21T06:39:59.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.400+0000 7fe429f15640 1 -- 172.21.15.67:0/2765813640 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4241a8e20 con 0x7fe424108cf0 2024-09-21T06:39:59.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.400+0000 7fe420ff9640 1 -- 172.21.15.67:0/2765813640 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe41c032eb0 con 0x7fe424108cf0 2024-09-21T06:39:59.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.400+0000 7fe420ff9640 1 -- 172.21.15.67:0/2765813640 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c030b20 con 0x7fe424108cf0 2024-09-21T06:39:59.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.402+0000 7fe429f15640 1 -- 172.21.15.67:0/2765813640 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe424102e70 con 0x7fe424108cf0 2024-09-21T06:39:59.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.404+0000 7fe420ff9640 1 -- 172.21.15.67:0/2765813640 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fe41c030cc0 con 0x7fe424108cf0 2024-09-21T06:39:59.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.405+0000 7fe420ff9640 1 --2- 172.21.15.67:0/2765813640 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe40c0b6510 0x7fe40c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:39:59.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.405+0000 7fe4237fe640 1 --2- 172.21.15.67:0/2765813640 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe40c0b6510 0x7fe40c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:39:59.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.405+0000 7fe420ff9640 1 -- 172.21.15.67:0/2765813640 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fe41c108970 con 0x7fe424108cf0 2024-09-21T06:39:59.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.405+0000 7fe4237fe640 1 --2- 172.21.15.67:0/2765813640 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe40c0b6510 0x7fe40c0b8a20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe4241010c0 tx=0x7fe41400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:39:59.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.408+0000 7fe420ff9640 1 -- 172.21.15.67:0/2765813640 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe41c0cee30 con 0x7fe424108cf0 2024-09-21T06:39:59.573 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:39:59 smithi067 ceph-mon[31232]: pgmap v462: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:39:59.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.636+0000 7fe429f15640 1 -- 172.21.15.67:0/2765813640 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe424100970 con 0x7fe424108cf0 2024-09-21T06:39:59.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.637+0000 7fe420ff9640 1 -- 172.21.15.67:0/2765813640 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7fe41c0d3210 con 0x7fe424108cf0 2024-09-21T06:39:59.640 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:39:59.640 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:39:59.640 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:39:59.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.640+0000 7fe429f15640 1 -- 172.21.15.67:0/2765813640 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe40c0b6510 msgr2=0x7fe40c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:59.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.640+0000 7fe429f15640 1 --2- 172.21.15.67:0/2765813640 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe40c0b6510 0x7fe40c0b8a20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe4241010c0 tx=0x7fe41400b410 comp rx=0 tx=0).stop 2024-09-21T06:39:59.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.640+0000 7fe429f15640 1 -- 172.21.15.67:0/2765813640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe424108cf0 msgr2=0x7fe4241000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:39:59.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.640+0000 7fe429f15640 1 --2- 172.21.15.67:0/2765813640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe424108cf0 0x7fe4241000f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe41c005e30 tx=0x7fe41c0093d0 comp rx=0 tx=0).stop 2024-09-21T06:39:59.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.640+0000 7fe429f15640 1 -- 172.21.15.67:0/2765813640 shutdown_connections 2024-09-21T06:39:59.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.640+0000 7fe429f15640 1 --2- 172.21.15.67:0/2765813640 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fe40c0b6510 0x7fe40c0b8a20 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:59.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.640+0000 7fe429f15640 1 --2- 172.21.15.67:0/2765813640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe424108cf0 0x7fe4241000f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:59.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.640+0000 7fe429f15640 1 --2- 172.21.15.67:0/2765813640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4241083a0 0x7fe4240ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:59.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.640+0000 7fe429f15640 1 --2- 172.21.15.67:0/2765813640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe424102a60 0x7fe4240ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:39:59.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.640+0000 7fe429f15640 1 -- 172.21.15.67:0/2765813640 >> 172.21.15.67:0/2765813640 conn(0x7fe424079730 msgr2=0x7fe42406b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:39:59.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.641+0000 7fe429f15640 1 -- 172.21.15.67:0/2765813640 shutdown_connections 2024-09-21T06:39:59.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:39:59.641+0000 7fe429f15640 1 -- 172.21.15.67:0/2765813640 wait complete. 2024-09-21T06:39:59.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:39:59 smithi100 ceph-mon[33401]: pgmap v462: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:39:59.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:39:59 smithi203 ceph-mon[33735]: pgmap v462: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:40:00.394 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db --set-id f4328142 --release --await' 2024-09-21T06:40:00.394 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24671 quiesce db --set-id f4328142 --release --await 2024-09-21T06:40:00.660 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:40:00.780 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:00 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2765813640' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:40:00.781 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:00 smithi067 ceph-mon[31232]: overall HEALTH_OK 2024-09-21T06:40:00.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:00 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2765813640' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:40:00.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:00 smithi100 ceph-mon[33401]: overall HEALTH_OK 2024-09-21T06:40:00.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:00 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2765813640' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:40:00.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:00 smithi203 ceph-mon[33735]: overall HEALTH_OK 2024-09-21T06:40:01.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.206+0000 7f2c56c8c640 1 -- 172.21.15.67:0/1000877338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c50102a60 msgr2=0x7f2c50102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:40:01.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.206+0000 7f2c56c8c640 1 --2- 172.21.15.67:0/1000877338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c50102a60 0x7f2c50102e70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009ca0 tx=0x7f2c4802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.206+0000 7f2c56c8c640 1 -- 172.21.15.67:0/1000877338 shutdown_connections 2024-09-21T06:40:01.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.206+0000 7f2c56c8c640 1 --2- 172.21.15.67:0/1000877338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c501083a0 0x7f2c501087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.206+0000 7f2c56c8c640 1 --2- 172.21.15.67:0/1000877338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c50108cf0 0x7f2c5010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.206+0000 7f2c56c8c640 1 --2- 172.21.15.67:0/1000877338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c50102a60 0x7f2c50102e70 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.206+0000 7f2c56c8c640 1 -- 172.21.15.67:0/1000877338 >> 172.21.15.67:0/1000877338 conn(0x7f2c50079730 msgr2=0x7f2c50079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:40:01.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.206+0000 7f2c56c8c640 1 -- 172.21.15.67:0/1000877338 shutdown_connections 2024-09-21T06:40:01.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.206+0000 7f2c56c8c640 1 -- 172.21.15.67:0/1000877338 wait complete. 2024-09-21T06:40:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.207+0000 7f2c56c8c640 1 Processor -- start 2024-09-21T06:40:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.207+0000 7f2c56c8c640 1 -- start start 2024-09-21T06:40:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.207+0000 7f2c56c8c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c50102a60 0x7f2c5019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:40:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.207+0000 7f2c56c8c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c501083a0 0x7f2c5019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:40:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.207+0000 7f2c56c8c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c50108cf0 0x7f2c5019fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:40:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.207+0000 7f2c56c8c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2c50113960 con 0x7f2c501083a0 2024-09-21T06:40:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.207+0000 7f2c56c8c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2c501137e0 con 0x7f2c50108cf0 2024-09-21T06:40:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.207+0000 7f2c56c8c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2c50113ae0 con 0x7f2c50102a60 2024-09-21T06:40:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.207+0000 7f2c4ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c501083a0 0x7f2c5019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:40:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.207+0000 7f2c4ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c501083a0 0x7f2c5019a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49552/0 (socket says 172.21.15.67:49552) 2024-09-21T06:40:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.207+0000 7f2c4ffff640 1 -- 172.21.15.67:0/4130824901 learned_addr learned my addr 172.21.15.67:0/4130824901 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:40:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.207+0000 7f2c54a28640 1 --2- 172.21.15.67:0/4130824901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c50102a60 0x7f2c5019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:40:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.208+0000 7f2c55229640 1 --2- 172.21.15.67:0/4130824901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c50108cf0 0x7f2c5019fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:40:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.208+0000 7f2c4ffff640 1 -- 172.21.15.67:0/4130824901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c50102a60 msgr2=0x7f2c5019a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:40:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.208+0000 7f2c4ffff640 1 --2- 172.21.15.67:0/4130824901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c50102a60 0x7f2c5019a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.208+0000 7f2c4ffff640 1 -- 172.21.15.67:0/4130824901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c50108cf0 msgr2=0x7f2c5019fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:40:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.208+0000 7f2c4ffff640 1 --2- 172.21.15.67:0/4130824901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c50108cf0 0x7f2c5019fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.208+0000 7f2c4ffff640 1 -- 172.21.15.67:0/4130824901 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c501a02c0 con 0x7f2c501083a0 2024-09-21T06:40:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.208+0000 7f2c4ffff640 1 --2- 172.21.15.67:0/4130824901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c501083a0 0x7f2c5019a8f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2c44005fd0 tx=0x7f2c4400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:40:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.208+0000 7f2c4dffb640 1 -- 172.21.15.67:0/4130824901 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c44017070 con 0x7f2c501083a0 2024-09-21T06:40:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.208+0000 7f2c4dffb640 1 -- 172.21.15.67:0/4130824901 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c44009c30 con 0x7f2c501083a0 2024-09-21T06:40:01.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.208+0000 7f2c4dffb640 1 -- 172.21.15.67:0/4130824901 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c44003c70 con 0x7f2c501083a0 2024-09-21T06:40:01.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.208+0000 7f2c56c8c640 1 -- 172.21.15.67:0/4130824901 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c501a05b0 con 0x7f2c501083a0 2024-09-21T06:40:01.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.208+0000 7f2c56c8c640 1 -- 172.21.15.67:0/4130824901 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c501a6d10 con 0x7f2c501083a0 2024-09-21T06:40:01.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.211+0000 7f2c5d200740 1 Processor -- start 2024-09-21T06:40:01.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.211+0000 7f2c4dffb640 1 -- 172.21.15.67:0/4130824901 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f2c44003e10 con 0x7f2c501083a0 2024-09-21T06:40:01.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.211+0000 7f2c5d200740 1 -- start start 2024-09-21T06:40:01.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.211+0000 7f2c5d200740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560f81f0520 0x5560f8596510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:40:01.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.211+0000 7f2c5d200740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560f81c7660 0x5560f8596a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:40:01.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.211+0000 7f2c5d200740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5560f81cf840 0x5560f8596f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:40:01.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.211+0000 7f2c5d200740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5560f8481dc0 con 0x5560f81cf840 2024-09-21T06:40:01.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.211+0000 7f2c5d200740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5560f850e1f0 con 0x5560f81f0520 2024-09-21T06:40:01.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.211+0000 7f2c5d200740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5560f84fd620 con 0x5560f81c7660 2024-09-21T06:40:01.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.211+0000 7f2c4ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5560f81cf840 0x5560f8596f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:40:01.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.211+0000 7f2c4ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5560f81cf840 0x5560f8596f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49562/0 (socket says 172.21.15.67:49562) 2024-09-21T06:40:01.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.211+0000 7f2c4ffff640 1 -- 172.21.15.67:0/1222366590 learned_addr learned my addr 172.21.15.67:0/1222366590 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:40:01.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.212+0000 7f2c54a28640 1 --2- 172.21.15.67:0/1222366590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560f81c7660 0x5560f8596a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:40:01.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.212+0000 7f2c55229640 1 --2- 172.21.15.67:0/1222366590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560f81f0520 0x5560f8596510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:40:01.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.212+0000 7f2c4dffb640 1 --2- 172.21.15.67:0/4130824901 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f2c380b6590 0x7f2c380b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:40:01.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.212+0000 7f2c4ffff640 1 -- 172.21.15.67:0/1222366590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560f81c7660 msgr2=0x5560f8596a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:40:01.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.212+0000 7f2c4ffff640 1 --2- 172.21.15.67:0/1222366590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560f81c7660 0x5560f8596a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.212+0000 7f2c4ffff640 1 -- 172.21.15.67:0/1222366590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560f81f0520 msgr2=0x5560f8596510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:40:01.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.212+0000 7f2c4ffff640 1 --2- 172.21.15.67:0/1222366590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560f81f0520 0x5560f8596510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.212+0000 7f2c4ffff640 1 -- 172.21.15.67:0/1222366590 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560f84c5fd0 con 0x5560f81cf840 2024-09-21T06:40:01.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.212+0000 7f2c54a28640 1 --2- 172.21.15.67:0/1222366590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560f81c7660 0x5560f8596a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:40:01.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.212+0000 7f2c55229640 1 --2- 172.21.15.67:0/4130824901 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f2c380b6590 0x7f2c380b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:40:01.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.212+0000 7f2c4ffff640 1 --2- 172.21.15.67:0/1222366590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5560f81cf840 0x5560f8596f90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2c4407e8a0 tx=0x7f2c44071190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6c733a2052bbd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:40:01.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.212+0000 7f2c4dffb640 1 -- 172.21.15.67:0/4130824901 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f2c440e7340 con 0x7f2c501083a0 2024-09-21T06:40:01.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.212+0000 7f2c36ffd640 1 -- 172.21.15.67:0/1222366590 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c440aced0 con 0x5560f81cf840 2024-09-21T06:40:01.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.213+0000 7f2c36ffd640 1 -- 172.21.15.67:0/1222366590 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c4407e8d0 con 0x5560f81cf840 2024-09-21T06:40:01.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.213+0000 7f2c36ffd640 1 -- 172.21.15.67:0/1222366590 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4407ecb0 con 0x5560f81cf840 2024-09-21T06:40:01.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.213+0000 7f2c5d200740 1 -- 172.21.15.67:0/1222366590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5560f81cf840 msgr2=0x5560f8596f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:40:01.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.213+0000 7f2c5d200740 1 --2- 172.21.15.67:0/1222366590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5560f81cf840 0x5560f8596f90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2c4407e8a0 tx=0x7f2c44071190 comp rx=0 tx=0).stop 2024-09-21T06:40:01.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.213+0000 7f2c55229640 1 --2- 172.21.15.67:0/4130824901 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f2c380b6590 0x7f2c380b8aa0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2c40009c90 tx=0x7f2c400046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:40:01.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.213+0000 7f2c5d200740 1 -- 172.21.15.67:0/1222366590 shutdown_connections 2024-09-21T06:40:01.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.213+0000 7f2c5d200740 1 --2- 172.21.15.67:0/1222366590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5560f81cf840 0x5560f8596f90 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.213+0000 7f2c5d200740 1 --2- 172.21.15.67:0/1222366590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560f81c7660 0x5560f8596a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.213+0000 7f2c5d200740 1 --2- 172.21.15.67:0/1222366590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560f81f0520 0x5560f8596510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.213+0000 7f2c5d200740 1 -- 172.21.15.67:0/1222366590 >> 172.21.15.67:0/1222366590 conn(0x5560f81e4910 msgr2=0x5560f8594400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:40:01.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.214+0000 7f2c5d200740 1 -- 172.21.15.67:0/1222366590 shutdown_connections 2024-09-21T06:40:01.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.214+0000 7f2c5d200740 1 -- 172.21.15.67:0/1222366590 wait complete. 2024-09-21T06:40:01.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.214+0000 7f2c5d200740 1 -- start start 2024-09-21T06:40:01.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.214+0000 7f2c5d200740 1 Processor -- start 2024-09-21T06:40:01.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.214+0000 7f2c5d200740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5560f81c7660 0x5560f85a5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:40:01.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.214+0000 7f2c5d200740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560f81cf840 0x5560f85a64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:40:01.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.214+0000 7f2c5d200740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560f81f0520 0x5560f85a6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:40:01.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.215+0000 7f2c5d200740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5560f8498300 con 0x5560f81c7660 2024-09-21T06:40:01.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.215+0000 7f2c5d200740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5560f8501430 con 0x5560f81f0520 2024-09-21T06:40:01.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.215+0000 7f2c5d200740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5560f835ebb0 con 0x5560f81cf840 2024-09-21T06:40:01.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.215+0000 7f2c5d200740 10 client.0 _ll_register_callbacks cb 0x5560f84b2120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:40:01.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.215+0000 7f2c54a28640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5560f81c7660 0x5560f85a5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:40:01.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.215+0000 7f2c54a28640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5560f81c7660 0x5560f85a5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49574/0 (socket says 172.21.15.67:49574) 2024-09-21T06:40:01.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.215+0000 7f2c54a28640 1 -- 172.21.15.67:0/1071421631 learned_addr learned my addr 172.21.15.67:0/1071421631 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:40:01.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.215+0000 7f2c55229640 1 --2- 172.21.15.67:0/1071421631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560f81f0520 0x5560f85a6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:40:01.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.215+0000 7f2c4ffff640 1 --2- 172.21.15.67:0/1071421631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560f81cf840 0x5560f85a64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:40:01.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.215+0000 7f2c54a28640 1 -- 172.21.15.67:0/1071421631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560f81cf840 msgr2=0x5560f85a64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:40:01.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.215+0000 7f2c54a28640 1 --2- 172.21.15.67:0/1071421631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560f81cf840 0x5560f85a64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.215+0000 7f2c54a28640 1 -- 172.21.15.67:0/1071421631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560f81f0520 msgr2=0x5560f85a6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:40:01.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.215+0000 7f2c54a28640 1 --2- 172.21.15.67:0/1071421631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560f81f0520 0x5560f85a6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.215+0000 7f2c54a28640 1 -- 172.21.15.67:0/1071421631 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560f8487780 con 0x5560f81c7660 2024-09-21T06:40:01.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.215+0000 7f2c4ffff640 1 --2- 172.21.15.67:0/1071421631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560f81cf840 0x5560f85a64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:40:01.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.215+0000 7f2c54a28640 1 --2- 172.21.15.67:0/1071421631 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5560f81c7660 0x5560f85a5f80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009ca0 tx=0x7f2c48031980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b705c87add45ad7b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:40:01.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.215+0000 7f2c2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:40:01.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.215+0000 7f2c2ffff640 1 -- 172.21.15.67:0/1071421631 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c48003b70 con 0x5560f81c7660 2024-09-21T06:40:01.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.215+0000 7f2c2ffff640 1 -- 172.21.15.67:0/1071421631 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f2c48003d10 con 0x5560f81c7660 2024-09-21T06:40:01.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.215+0000 7f2c5d200740 1 -- 172.21.15.67:0/1071421631 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5560f8510140 con 0x5560f81c7660 2024-09-21T06:40:01.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.216+0000 7f2c2ffff640 1 -- 172.21.15.67:0/1071421631 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2c4804f050 con 0x5560f81c7660 2024-09-21T06:40:01.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.216+0000 7f2c5d200740 10 client.15615 fetch_fsmap learned FSMap version 21 2024-09-21T06:40:01.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.216+0000 7f2c5d200740 1 -- 172.21.15.67:0/1071421631 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5560f825bb00 con 0x5560f81c7660 2024-09-21T06:40:01.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.216+0000 7f2c2ffff640 1 -- 172.21.15.67:0/1071421631 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5560f8510140 con 0x5560f81c7660 2024-09-21T06:40:01.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.216+0000 7f2c2ffff640 1 -- 172.21.15.67:0/1071421631 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f2c480307f0 con 0x5560f81c7660 2024-09-21T06:40:01.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.216+0000 7f2c5d200740 10 client.15615 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:40:01.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.216+0000 7f2c5d200740 10 client.15615 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:40:01.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.216+0000 7f2c5d200740 20 client.15615 populate_metadata read hostname 'smithi067' 2024-09-21T06:40:01.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.216+0000 7f2c5d200740 1 --2- 172.21.15.67:0/1071421631 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x5560f81aeea0 0x5560f85b80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:40:01.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.216+0000 7f2c5d200740 4 client.15615 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:40:01.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.216+0000 7f2c5d200740 1 -- 172.21.15.67:0/1071421631 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560f850bbf0 con 0x5560f81aeea0 2024-09-21T06:40:01.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.217+0000 7f2c4ffff640 1 --2- 172.21.15.67:0/1071421631 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x5560f81aeea0 0x5560f85b80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:40:01.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.217+0000 7f2c2ffff640 10 client.15615 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:40:01.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.217+0000 7f2c4ffff640 1 --2- 172.21.15.67:0/1071421631 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x5560f81aeea0 0x5560f85b80d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82b78935b4465b97 server_cookie=b4e373b12df60c8c in_seq=0 out_seq=0 2024-09-21T06:40:01.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.218+0000 7f2c2ffff640 1 -- 172.21.15.67:0/1071421631 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5560f850bbf0 con 0x5560f81aeea0 2024-09-21T06:40:01.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.218+0000 7f2c2ffff640 10 client.15615 handle_command_reply: tid=0 2024-09-21T06:40:01.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.218+0000 7f2c5d200740 1 client.15615 shutdown 2024-09-21T06:40:01.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.218+0000 7f2c5d200740 1 -- 172.21.15.67:0/1071421631 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5560f81c7660 msgr2=0x5560f85a5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:40:01.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.218+0000 7f2c5d200740 1 --2- 172.21.15.67:0/1071421631 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5560f81c7660 0x5560f85a5f80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009ca0 tx=0x7f2c48031980 comp rx=0 tx=0).stop 2024-09-21T06:40:01.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.218+0000 7f2c5d200740 1 -- 172.21.15.67:0/1071421631 shutdown_connections 2024-09-21T06:40:01.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.218+0000 7f2c5d200740 1 --2- 172.21.15.67:0/1071421631 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x5560f81aeea0 0x5560f85b80d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.218+0000 7f2c5d200740 1 --2- 172.21.15.67:0/1071421631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560f81f0520 0x5560f85a6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.218+0000 7f2c5d200740 1 --2- 172.21.15.67:0/1071421631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560f81cf840 0x5560f85a64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.218+0000 7f2c5d200740 1 --2- 172.21.15.67:0/1071421631 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5560f81c7660 0x5560f85a5f80 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.218+0000 7f2c5d200740 1 -- 172.21.15.67:0/1071421631 >> 172.21.15.67:0/1071421631 conn(0x5560f81e4910 msgr2=0x5560f8599660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:40:01.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.218+0000 7f2c2ffff640 0 client.15615 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:40:01.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.218+0000 7f2c5d200740 1 -- 172.21.15.67:0/1071421631 shutdown_connections 2024-09-21T06:40:01.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.218+0000 7f2c5d200740 1 -- 172.21.15.67:0/1071421631 wait complete. 2024-09-21T06:40:01.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.218+0000 7f2c5d200740 20 client.15615 trim_cache size 0 max 16384 2024-09-21T06:40:01.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.235+0000 7f2c5d200740 1 Processor -- start 2024-09-21T06:40:01.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.236+0000 7f2c5d200740 1 -- start start 2024-09-21T06:40:01.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.236+0000 7f2c5d200740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560f81c7660 0x5560f85a1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:40:01.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.236+0000 7f2c5d200740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5560f81cf840 0x5560f8599610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:40:01.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.236+0000 7f2c5d200740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560f81f0520 0x5560f8597c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:40:01.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.236+0000 7f2c5d200740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2c501ac910 con 0x5560f81cf840 2024-09-21T06:40:01.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.236+0000 7f2c5d200740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2c501af190 con 0x5560f81c7660 2024-09-21T06:40:01.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.236+0000 7f2c5d200740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2c501acf80 con 0x5560f81f0520 2024-09-21T06:40:01.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.236+0000 7f2c4ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5560f81cf840 0x5560f8599610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:40:01.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.236+0000 7f2c4ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5560f81cf840 0x5560f8599610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49588/0 (socket says 172.21.15.67:49588) 2024-09-21T06:40:01.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.236+0000 7f2c4ffff640 1 -- 172.21.15.67:0/1880366348 learned_addr learned my addr 172.21.15.67:0/1880366348 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:40:01.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.236+0000 7f2c55229640 1 --2- 172.21.15.67:0/1880366348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560f81f0520 0x5560f8597c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:40:01.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.236+0000 7f2c54a28640 1 --2- 172.21.15.67:0/1880366348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560f81c7660 0x5560f85a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:40:01.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.236+0000 7f2c4ffff640 1 -- 172.21.15.67:0/1880366348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560f81f0520 msgr2=0x5560f8597c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:40:01.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.236+0000 7f2c4ffff640 1 --2- 172.21.15.67:0/1880366348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560f81f0520 0x5560f8597c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.236+0000 7f2c4ffff640 1 -- 172.21.15.67:0/1880366348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560f81c7660 msgr2=0x5560f85a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:40:01.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.236+0000 7f2c4ffff640 1 --2- 172.21.15.67:0/1880366348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560f81c7660 0x5560f85a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.236+0000 7f2c4ffff640 1 -- 172.21.15.67:0/1880366348 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560f8599cf0 con 0x5560f81cf840 2024-09-21T06:40:01.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.236+0000 7f2c54a28640 1 --2- 172.21.15.67:0/1880366348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560f81c7660 0x5560f85a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:40:01.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.236+0000 7f2c4ffff640 1 --2- 172.21.15.67:0/1880366348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5560f81cf840 0x5560f8599610 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2c440e4de0 tx=0x7f2c440eae20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34fbcf032bb91983 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:40:01.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.237+0000 7f2c367fc640 1 -- 172.21.15.67:0/1880366348 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4407eeb0 con 0x5560f81cf840 2024-09-21T06:40:01.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.237+0000 7f2c367fc640 1 -- 172.21.15.67:0/1880366348 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c440ad800 con 0x5560f81cf840 2024-09-21T06:40:01.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.237+0000 7f2c367fc640 1 -- 172.21.15.67:0/1880366348 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4407f0b0 con 0x5560f81cf840 2024-09-21T06:40:01.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.237+0000 7f2c5d200740 1 -- 172.21.15.67:0/1880366348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5560f81cf840 msgr2=0x5560f8599610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:40:01.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.237+0000 7f2c5d200740 1 --2- 172.21.15.67:0/1880366348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5560f81cf840 0x5560f8599610 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2c440e4de0 tx=0x7f2c440eae20 comp rx=0 tx=0).stop 2024-09-21T06:40:01.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.237+0000 7f2c5d200740 1 -- 172.21.15.67:0/1880366348 shutdown_connections 2024-09-21T06:40:01.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.237+0000 7f2c5d200740 1 --2- 172.21.15.67:0/1880366348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560f81f0520 0x5560f8597c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.237+0000 7f2c5d200740 1 --2- 172.21.15.67:0/1880366348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5560f81cf840 0x5560f8599610 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.237+0000 7f2c5d200740 1 --2- 172.21.15.67:0/1880366348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560f81c7660 0x5560f85a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.237+0000 7f2c5d200740 1 -- 172.21.15.67:0/1880366348 >> 172.21.15.67:0/1880366348 conn(0x5560f81e4910 msgr2=0x5560f8597540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:40:01.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.237+0000 7f2c5d200740 1 -- 172.21.15.67:0/1880366348 shutdown_connections 2024-09-21T06:40:01.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.237+0000 7f2c5d200740 1 -- 172.21.15.67:0/1880366348 wait complete. 2024-09-21T06:40:01.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.238+0000 7f2c5d200740 1 -- start start 2024-09-21T06:40:01.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.238+0000 7f2c5d200740 1 Processor -- start 2024-09-21T06:40:01.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.238+0000 7f2c5d200740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560f81c7660 0x5560f85ae670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:40:01.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.238+0000 7f2c5d200740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560f81cf840 0x5560f85aab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:40:01.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.238+0000 7f2c5d200740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5560f81f0520 0x5560f85ab070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:40:01.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.239+0000 7f2c5d200740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2c501ad650 con 0x5560f81f0520 2024-09-21T06:40:01.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.239+0000 7f2c5d200740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2c501aeac0 con 0x5560f81c7660 2024-09-21T06:40:01.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.239+0000 7f2c5d200740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2c501add20 con 0x5560f81cf840 2024-09-21T06:40:01.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.239+0000 7f2c55229640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5560f81f0520 0x5560f85ab070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:40:01.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.239+0000 7f2c5d200740 10 client.0 _ll_register_callbacks cb 0x7f2c48009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:40:01.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.239+0000 7f2c55229640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5560f81f0520 0x5560f85ab070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49596/0 (socket says 172.21.15.67:49596) 2024-09-21T06:40:01.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.239+0000 7f2c55229640 1 -- 172.21.15.67:0/2899459811 learned_addr learned my addr 172.21.15.67:0/2899459811 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:40:01.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.239+0000 7f2c54a28640 1 --2- 172.21.15.67:0/2899459811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560f81c7660 0x5560f85ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:40:01.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.239+0000 7f2c4ffff640 1 --2- 172.21.15.67:0/2899459811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560f81cf840 0x5560f85aab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:40:01.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.239+0000 7f2c55229640 1 -- 172.21.15.67:0/2899459811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560f81cf840 msgr2=0x5560f85aab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:40:01.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.239+0000 7f2c55229640 1 --2- 172.21.15.67:0/2899459811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560f81cf840 0x5560f85aab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.239+0000 7f2c55229640 1 -- 172.21.15.67:0/2899459811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560f81c7660 msgr2=0x5560f85ae670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:40:01.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.239+0000 7f2c55229640 1 --2- 172.21.15.67:0/2899459811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560f81c7660 0x5560f85ae670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.239+0000 7f2c55229640 1 -- 172.21.15.67:0/2899459811 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560f82d2960 con 0x5560f81f0520 2024-09-21T06:40:01.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.239+0000 7f2c55229640 1 --2- 172.21.15.67:0/2899459811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5560f81f0520 0x5560f85ab070 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2c40005d80 tx=0x7f2c40004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e71e175134cba89 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:40:01.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.239+0000 7f2c2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:40:01.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.239+0000 7f2c2ffff640 1 -- 172.21.15.67:0/2899459811 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c40012650 con 0x5560f81f0520 2024-09-21T06:40:01.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.239+0000 7f2c5d200740 1 -- 172.21.15.67:0/2899459811 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c440ad800 con 0x5560f81f0520 2024-09-21T06:40:01.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.239+0000 7f2c2ffff640 1 -- 172.21.15.67:0/2899459811 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f2c400127f0 con 0x5560f81f0520 2024-09-21T06:40:01.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.240+0000 7f2c2ffff640 1 -- 172.21.15.67:0/2899459811 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2c400129f0 con 0x5560f81f0520 2024-09-21T06:40:01.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.240+0000 7f2c5d200740 10 client.15621 fetch_fsmap learned FSMap version 21 2024-09-21T06:40:01.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.240+0000 7f2c5d200740 1 -- 172.21.15.67:0/2899459811 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5560f86235f0 con 0x5560f81f0520 2024-09-21T06:40:01.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.240+0000 7f2c2ffff640 1 -- 172.21.15.67:0/2899459811 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c440ad800 con 0x5560f81f0520 2024-09-21T06:40:01.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.240+0000 7f2c2ffff640 1 -- 172.21.15.67:0/2899459811 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f2c40011b60 con 0x5560f81f0520 2024-09-21T06:40:01.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.240+0000 7f2c5d200740 10 client.15621 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:40:01.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.240+0000 7f2c5d200740 10 client.15621 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:40:01.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.240+0000 7f2c5d200740 20 client.15621 populate_metadata read hostname 'smithi067' 2024-09-21T06:40:01.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.240+0000 7f2c5d200740 1 --2- 172.21.15.67:0/2899459811 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x5560f8184250 0x5560f8625e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:40:01.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.240+0000 7f2c5d200740 4 client.15621 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f4328142", "release": true, "await": true}] 2024-09-21T06:40:01.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.240+0000 7f2c5d200740 1 -- 172.21.15.67:0/2899459811 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f4328142", "release": true, "await": true}) -- 0x5560f8509020 con 0x5560f8184250 2024-09-21T06:40:01.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.240+0000 7f2c54a28640 1 --2- 172.21.15.67:0/2899459811 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x5560f8184250 0x5560f8625e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:40:01.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.241+0000 7f2c2ffff640 10 client.15621 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:40:01.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.241+0000 7f2c54a28640 1 --2- 172.21.15.67:0/2899459811 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x5560f8184250 0x5560f8625e10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ba3c6582bd7b19f server_cookie=6f54188836de69c0 in_seq=0 out_seq=0 2024-09-21T06:40:01.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.244+0000 7f2c2ffff640 1 -- 172.21.15.67:0/2899459811 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5560f8509020 con 0x5560f8184250 2024-09-21T06:40:01.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.244+0000 7f2c2ffff640 10 client.15621 handle_command_reply: tid=0 2024-09-21T06:40:01.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.244+0000 7f2c5d200740 1 client.15621 shutdown 2024-09-21T06:40:01.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.244+0000 7f2c5d200740 1 -- 172.21.15.67:0/2899459811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5560f81f0520 msgr2=0x5560f85ab070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:40:01.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.244+0000 7f2c5d200740 1 --2- 172.21.15.67:0/2899459811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5560f81f0520 0x5560f85ab070 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2c40005d80 tx=0x7f2c40004730 comp rx=0 tx=0).stop 2024-09-21T06:40:01.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.244+0000 7f2c5d200740 1 -- 172.21.15.67:0/2899459811 shutdown_connections 2024-09-21T06:40:01.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.244+0000 7f2c5d200740 1 --2- 172.21.15.67:0/2899459811 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x5560f8184250 0x5560f8625e10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.244+0000 7f2c5d200740 1 --2- 172.21.15.67:0/2899459811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5560f81f0520 0x5560f85ab070 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.244+0000 7f2c5d200740 1 --2- 172.21.15.67:0/2899459811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560f81cf840 0x5560f85aab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.244+0000 7f2c2ffff640 0 client.15621 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:40:01.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.244+0000 7f2c5d200740 1 --2- 172.21.15.67:0/2899459811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560f81c7660 0x5560f85ae670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.244+0000 7f2c5d200740 1 -- 172.21.15.67:0/2899459811 >> 172.21.15.67:0/2899459811 conn(0x5560f81e4910 msgr2=0x5560f8597f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:40:01.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.244+0000 7f2c5d200740 1 -- 172.21.15.67:0/2899459811 shutdown_connections 2024-09-21T06:40:01.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.244+0000 7f2c5d200740 1 -- 172.21.15.67:0/2899459811 wait complete. 2024-09-21T06:40:01.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.245+0000 7f2c5d200740 20 client.15621 trim_cache size 0 max 16384 2024-09-21T06:40:01.249 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-21T06:40:01.249 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 21, 2024-09-21T06:40:01.249 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-09-21T06:40:01.249 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 16, 2024-09-21T06:40:01.249 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-21T06:40:01.249 INFO:teuthology.orchestra.run.smithi067.stdout: "f4328142": { 2024-09-21T06:40:01.249 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 16, 2024-09-21T06:40:01.249 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-21T06:40:01.249 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:40:01.250 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-21T06:40:01.250 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:40:01.250 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:40:01.250 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-21T06:40:01.250 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 147.8, 2024-09-21T06:40:01.250 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-21T06:40:01.250 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-21T06:40:01.250 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-21T06:40:01.250 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:40:01.250 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-21T06:40:01.250 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:40:01.251 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:40:01.251 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:40:01.251 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:40:01.251 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:40:01.251 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:40:01.251 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-21T06:40:01.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.245+0000 7f2c56c8c640 1 -- 172.21.15.67:0/4130824901 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f2c380b6590 msgr2=0x7f2c380b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:40:01.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.245+0000 7f2c56c8c640 1 --2- 172.21.15.67:0/4130824901 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f2c380b6590 0x7f2c380b8aa0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2c40009c90 tx=0x7f2c400046c0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.245+0000 7f2c56c8c640 1 -- 172.21.15.67:0/4130824901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c501083a0 msgr2=0x7f2c5019a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:40:01.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.245+0000 7f2c56c8c640 1 --2- 172.21.15.67:0/4130824901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c501083a0 0x7f2c5019a8f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2c44005fd0 tx=0x7f2c4400dde0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.246+0000 7f2c56c8c640 1 -- 172.21.15.67:0/4130824901 shutdown_connections 2024-09-21T06:40:01.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.246+0000 7f2c56c8c640 1 --2- 172.21.15.67:0/4130824901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c50108cf0 0x7f2c5019fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.246+0000 7f2c56c8c640 1 --2- 172.21.15.67:0/4130824901 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f2c380b6590 0x7f2c380b8aa0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.246+0000 7f2c56c8c640 1 --2- 172.21.15.67:0/4130824901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c501083a0 0x7f2c5019a8f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.246+0000 7f2c56c8c640 1 --2- 172.21.15.67:0/4130824901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c50102a60 0x7f2c5019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:40:01.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.246+0000 7f2c56c8c640 1 -- 172.21.15.67:0/4130824901 >> 172.21.15.67:0/4130824901 conn(0x7f2c50079730 msgr2=0x7f2c50101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:40:01.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.246+0000 7f2c56c8c640 1 -- 172.21.15.67:0/4130824901 shutdown_connections 2024-09-21T06:40:01.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:40:01.246+0000 7f2c56c8c640 1 -- 172.21.15.67:0/4130824901 wait complete. 2024-09-21T06:40:01.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:01 smithi067 ceph-mon[31232]: pgmap v463: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:40:01.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:01 smithi100 ceph-mon[33401]: pgmap v463: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:40:01.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:01 smithi203 ceph-mon[33735]: pgmap v463: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:40:02.084 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f4328142', total seconds elapsed: 31.7 2024-09-21T06:40:03.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:03 smithi067 ceph-mon[31232]: pgmap v464: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1.7 MiB/s wr, 3 op/s 2024-09-21T06:40:03.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:03 smithi100 ceph-mon[33401]: pgmap v464: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1.7 MiB/s wr, 3 op/s 2024-09-21T06:40:03.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:03 smithi203 ceph-mon[33735]: pgmap v464: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1.7 MiB/s wr, 3 op/s 2024-09-21T06:40:05.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:05 smithi067 ceph-mon[31232]: pgmap v465: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 7.0 MiB/s wr, 5 op/s 2024-09-21T06:40:05.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:05 smithi100 ceph-mon[33401]: pgmap v465: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 7.0 MiB/s wr, 5 op/s 2024-09-21T06:40:05.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:05 smithi203 ceph-mon[33735]: pgmap v465: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 7.0 MiB/s wr, 5 op/s 2024-09-21T06:40:07.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:07 smithi067 ceph-mon[31232]: pgmap v466: 129 pgs: 129 active+clean; 6.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 23 MiB/s wr, 14 op/s 2024-09-21T06:40:07.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:07 smithi100 ceph-mon[33401]: pgmap v466: 129 pgs: 129 active+clean; 6.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 23 MiB/s wr, 14 op/s 2024-09-21T06:40:07.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:07 smithi203 ceph-mon[33735]: pgmap v466: 129 pgs: 129 active+clean; 6.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 23 MiB/s wr, 14 op/s 2024-09-21T06:40:09.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:09 smithi067 ceph-mon[31232]: pgmap v467: 129 pgs: 129 active+clean; 6.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 63 MiB/s wr, 33 op/s 2024-09-21T06:40:09.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:09 smithi100 ceph-mon[33401]: pgmap v467: 129 pgs: 129 active+clean; 6.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 63 MiB/s wr, 33 op/s 2024-09-21T06:40:09.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:09 smithi203 ceph-mon[33735]: pgmap v467: 129 pgs: 129 active+clean; 6.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 63 MiB/s wr, 33 op/s 2024-09-21T06:40:11.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:11 smithi067 ceph-mon[31232]: pgmap v468: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 68 MiB/s wr, 35 op/s 2024-09-21T06:40:11.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:11 smithi100 ceph-mon[33401]: pgmap v468: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 68 MiB/s wr, 35 op/s 2024-09-21T06:40:11.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:11 smithi203 ceph-mon[33735]: pgmap v468: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 68 MiB/s wr, 35 op/s 2024-09-21T06:40:13.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:13 smithi067 ceph-mon[31232]: pgmap v469: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 115 MiB/s wr, 59 op/s 2024-09-21T06:40:13.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:13 smithi100 ceph-mon[33401]: pgmap v469: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 115 MiB/s wr, 59 op/s 2024-09-21T06:40:13.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:13 smithi203 ceph-mon[33735]: pgmap v469: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 115 MiB/s wr, 59 op/s 2024-09-21T06:40:15.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:15 smithi100 ceph-mon[33401]: pgmap v470: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 131 MiB/s wr, 67 op/s 2024-09-21T06:40:15.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:15 smithi203 ceph-mon[33735]: pgmap v470: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 131 MiB/s wr, 67 op/s 2024-09-21T06:40:16.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:15 smithi067 ceph-mon[31232]: pgmap v470: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 131 MiB/s wr, 67 op/s 2024-09-21T06:40:17.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:17 smithi100 ceph-mon[33401]: pgmap v471: 129 pgs: 129 active+clean; 8.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 150 MiB/s wr, 77 op/s 2024-09-21T06:40:17.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:17 smithi203 ceph-mon[33735]: pgmap v471: 129 pgs: 129 active+clean; 8.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 150 MiB/s wr, 77 op/s 2024-09-21T06:40:18.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:17 smithi067 ceph-mon[31232]: pgmap v471: 129 pgs: 129 active+clean; 8.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 150 MiB/s wr, 77 op/s 2024-09-21T06:40:19.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:19 smithi100 ceph-mon[33401]: pgmap v472: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 168 MiB/s wr, 85 op/s 2024-09-21T06:40:19.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:19 smithi203 ceph-mon[33735]: pgmap v472: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 168 MiB/s wr, 85 op/s 2024-09-21T06:40:20.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:19 smithi067 ceph-mon[31232]: pgmap v472: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 168 MiB/s wr, 85 op/s 2024-09-21T06:40:21.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:21 smithi100 ceph-mon[33401]: pgmap v473: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 135 MiB/s wr, 69 op/s 2024-09-21T06:40:21.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:21 smithi203 ceph-mon[33735]: pgmap v473: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 135 MiB/s wr, 69 op/s 2024-09-21T06:40:22.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:21 smithi067 ceph-mon[31232]: pgmap v473: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 135 MiB/s wr, 69 op/s 2024-09-21T06:40:23.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:23 smithi203 ceph-mon[33735]: pgmap v474: 129 pgs: 129 active+clean; 9.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 180 MiB/s wr, 92 op/s 2024-09-21T06:40:24.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:23 smithi067 ceph-mon[31232]: pgmap v474: 129 pgs: 129 active+clean; 9.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 180 MiB/s wr, 92 op/s 2024-09-21T06:40:24.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:23 smithi100 ceph-mon[33401]: pgmap v474: 129 pgs: 129 active+clean; 9.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 180 MiB/s wr, 92 op/s 2024-09-21T06:40:25.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:25 smithi203 ceph-mon[33735]: pgmap v475: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 148 MiB/s wr, 75 op/s 2024-09-21T06:40:26.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:25 smithi067 ceph-mon[31232]: pgmap v475: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 148 MiB/s wr, 75 op/s 2024-09-21T06:40:26.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:25 smithi100 ceph-mon[33401]: pgmap v475: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 148 MiB/s wr, 75 op/s 2024-09-21T06:40:27.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:27 smithi203 ceph-mon[33735]: pgmap v476: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 154 MiB/s wr, 80 op/s 2024-09-21T06:40:28.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:27 smithi067 ceph-mon[31232]: pgmap v476: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 154 MiB/s wr, 80 op/s 2024-09-21T06:40:28.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:27 smithi100 ceph-mon[33401]: pgmap v476: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 154 MiB/s wr, 80 op/s 2024-09-21T06:40:29.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:29 smithi203 ceph-mon[33735]: pgmap v477: 129 pgs: 129 active+clean; 9.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 160 MiB/s wr, 94 op/s 2024-09-21T06:40:30.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:29 smithi067 ceph-mon[31232]: pgmap v477: 129 pgs: 129 active+clean; 9.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 160 MiB/s wr, 94 op/s 2024-09-21T06:40:30.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:29 smithi100 ceph-mon[33401]: pgmap v477: 129 pgs: 129 active+clean; 9.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 160 MiB/s wr, 94 op/s 2024-09-21T06:40:31.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:31 smithi203 ceph-mon[33735]: pgmap v478: 129 pgs: 129 active+clean; 9.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 132 MiB/s wr, 90 op/s 2024-09-21T06:40:32.041 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:31 smithi067 ceph-mon[31232]: pgmap v478: 129 pgs: 129 active+clean; 9.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 132 MiB/s wr, 90 op/s 2024-09-21T06:40:32.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:31 smithi100 ceph-mon[33401]: pgmap v478: 129 pgs: 129 active+clean; 9.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 132 MiB/s wr, 90 op/s 2024-09-21T06:40:33.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:33 smithi203 ceph-mon[33735]: pgmap v479: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 146 MiB/s wr, 348 op/s 2024-09-21T06:40:34.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:33 smithi067 ceph-mon[31232]: pgmap v479: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 146 MiB/s wr, 348 op/s 2024-09-21T06:40:34.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:33 smithi100 ceph-mon[33401]: pgmap v479: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 146 MiB/s wr, 348 op/s 2024-09-21T06:40:35.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:35 smithi203 ceph-mon[33735]: pgmap v480: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 103 MiB/s wr, 420 op/s 2024-09-21T06:40:36.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:35 smithi067 ceph-mon[31232]: pgmap v480: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 103 MiB/s wr, 420 op/s 2024-09-21T06:40:36.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:35 smithi100 ceph-mon[33401]: pgmap v480: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 103 MiB/s wr, 420 op/s 2024-09-21T06:40:36.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:36 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:40:37.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:36 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:40:37.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:36 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:40:37.942 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:37 smithi067 ceph-mon[31232]: pgmap v481: 129 pgs: 129 active+clean; 10 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 96 MiB/s wr, 543 op/s 2024-09-21T06:40:37.942 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:37 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:40:37.942 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:37 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:40:37.943 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:37 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:40:37.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:37 smithi203 ceph-mon[33735]: pgmap v481: 129 pgs: 129 active+clean; 10 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 96 MiB/s wr, 543 op/s 2024-09-21T06:40:37.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:37 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:40:37.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:37 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:40:37.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:37 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:40:38.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:37 smithi100 ceph-mon[33401]: pgmap v481: 129 pgs: 129 active+clean; 10 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 96 MiB/s wr, 543 op/s 2024-09-21T06:40:38.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:37 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:40:38.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:37 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:40:38.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:37 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:40:39.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:39 smithi203 ceph-mon[33735]: pgmap v482: 129 pgs: 129 active+clean; 10 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 85 MiB/s wr, 740 op/s 2024-09-21T06:40:40.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:39 smithi067 ceph-mon[31232]: pgmap v482: 129 pgs: 129 active+clean; 10 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 85 MiB/s wr, 740 op/s 2024-09-21T06:40:40.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:39 smithi100 ceph-mon[33401]: pgmap v482: 129 pgs: 129 active+clean; 10 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 85 MiB/s wr, 740 op/s 2024-09-21T06:40:41.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:41 smithi203 ceph-mon[33735]: pgmap v483: 129 pgs: 129 active+clean; 10 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 58 MiB/s wr, 750 op/s 2024-09-21T06:40:42.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:41 smithi067 ceph-mon[31232]: pgmap v483: 129 pgs: 129 active+clean; 10 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 58 MiB/s wr, 750 op/s 2024-09-21T06:40:42.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:41 smithi100 ceph-mon[33401]: pgmap v483: 129 pgs: 129 active+clean; 10 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 58 MiB/s wr, 750 op/s 2024-09-21T06:40:43.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:43 smithi203 ceph-mon[33735]: pgmap v484: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 69 MiB/s wr, 1.01k op/s 2024-09-21T06:40:44.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:43 smithi067 ceph-mon[31232]: pgmap v484: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 69 MiB/s wr, 1.01k op/s 2024-09-21T06:40:44.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:43 smithi100 ceph-mon[33401]: pgmap v484: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 69 MiB/s wr, 1.01k op/s 2024-09-21T06:40:45.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:45 smithi203 ceph-mon[33735]: pgmap v485: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 53 MiB/s wr, 852 op/s 2024-09-21T06:40:46.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:45 smithi067 ceph-mon[31232]: pgmap v485: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 53 MiB/s wr, 852 op/s 2024-09-21T06:40:46.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:45 smithi100 ceph-mon[33401]: pgmap v485: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 53 MiB/s wr, 852 op/s 2024-09-21T06:40:47.855 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:47 smithi100 ceph-mon[33401]: pgmap v486: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 55 MiB/s wr, 875 op/s 2024-09-21T06:40:47.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:47 smithi203 ceph-mon[33735]: pgmap v486: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 55 MiB/s wr, 875 op/s 2024-09-21T06:40:48.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:47 smithi067 ceph-mon[31232]: pgmap v486: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 55 MiB/s wr, 875 op/s 2024-09-21T06:40:49.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:49 smithi203 ceph-mon[33735]: pgmap v487: 129 pgs: 129 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 60 MiB/s wr, 955 op/s 2024-09-21T06:40:49.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:49 smithi100 ceph-mon[33401]: pgmap v487: 129 pgs: 129 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 60 MiB/s wr, 955 op/s 2024-09-21T06:40:50.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:49 smithi067 ceph-mon[31232]: pgmap v487: 129 pgs: 129 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 60 MiB/s wr, 955 op/s 2024-09-21T06:40:51.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:51 smithi203 ceph-mon[33735]: pgmap v488: 129 pgs: 129 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 49 MiB/s wr, 781 op/s 2024-09-21T06:40:52.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:51 smithi067 ceph-mon[31232]: pgmap v488: 129 pgs: 129 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 49 MiB/s wr, 781 op/s 2024-09-21T06:40:52.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:51 smithi100 ceph-mon[33401]: pgmap v488: 129 pgs: 129 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 49 MiB/s wr, 781 op/s 2024-09-21T06:40:53.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:53 smithi203 ceph-mon[33735]: pgmap v489: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-21T06:40:54.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:53 smithi067 ceph-mon[31232]: pgmap v489: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-21T06:40:54.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:53 smithi100 ceph-mon[33401]: pgmap v489: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-21T06:40:55.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:55 smithi203 ceph-mon[33735]: pgmap v490: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 52 MiB/s wr, 832 op/s 2024-09-21T06:40:56.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:55 smithi067 ceph-mon[31232]: pgmap v490: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 52 MiB/s wr, 832 op/s 2024-09-21T06:40:56.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:55 smithi100 ceph-mon[33401]: pgmap v490: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 52 MiB/s wr, 832 op/s 2024-09-21T06:40:57.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:57 smithi203 ceph-mon[33735]: pgmap v491: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 53 MiB/s wr, 854 op/s 2024-09-21T06:40:58.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:57 smithi067 ceph-mon[31232]: pgmap v491: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 53 MiB/s wr, 854 op/s 2024-09-21T06:40:58.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:57 smithi100 ceph-mon[33401]: pgmap v491: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 53 MiB/s wr, 854 op/s 2024-09-21T06:40:59.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:40:59 smithi203 ceph-mon[33735]: pgmap v492: 129 pgs: 129 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 59 MiB/s wr, 942 op/s 2024-09-21T06:41:00.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:40:59 smithi067 ceph-mon[31232]: pgmap v492: 129 pgs: 129 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 59 MiB/s wr, 942 op/s 2024-09-21T06:41:00.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:40:59 smithi100 ceph-mon[33401]: pgmap v492: 129 pgs: 129 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 59 MiB/s wr, 942 op/s 2024-09-21T06:41:01.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:01 smithi203 ceph-mon[33735]: pgmap v493: 129 pgs: 129 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 48 MiB/s wr, 769 op/s 2024-09-21T06:41:02.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:01 smithi067 ceph-mon[31232]: pgmap v493: 129 pgs: 129 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 48 MiB/s wr, 769 op/s 2024-09-21T06:41:02.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:01 smithi100 ceph-mon[33401]: pgmap v493: 129 pgs: 129 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 48 MiB/s wr, 769 op/s 2024-09-21T06:41:03.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:03 smithi203 ceph-mon[33735]: pgmap v494: 129 pgs: 129 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 63 MiB/s wr, 1.00k op/s 2024-09-21T06:41:04.041 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:03 smithi067 ceph-mon[31232]: pgmap v494: 129 pgs: 129 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 63 MiB/s wr, 1.00k op/s 2024-09-21T06:41:04.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:03 smithi100 ceph-mon[33401]: pgmap v494: 129 pgs: 129 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 63 MiB/s wr, 1.00k op/s 2024-09-21T06:41:05.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:05 smithi203 ceph-mon[33735]: pgmap v495: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 51 MiB/s wr, 821 op/s 2024-09-21T06:41:06.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:05 smithi067 ceph-mon[31232]: pgmap v495: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 51 MiB/s wr, 821 op/s 2024-09-21T06:41:06.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:05 smithi100 ceph-mon[33401]: pgmap v495: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 51 MiB/s wr, 821 op/s 2024-09-21T06:41:07.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:07 smithi203 ceph-mon[33735]: pgmap v496: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 53 MiB/s wr, 847 op/s 2024-09-21T06:41:08.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:07 smithi067 ceph-mon[31232]: pgmap v496: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 53 MiB/s wr, 847 op/s 2024-09-21T06:41:08.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:07 smithi100 ceph-mon[33401]: pgmap v496: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 53 MiB/s wr, 847 op/s 2024-09-21T06:41:09.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:09 smithi203 ceph-mon[33735]: pgmap v497: 129 pgs: 129 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 58 MiB/s wr, 930 op/s 2024-09-21T06:41:10.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:09 smithi067 ceph-mon[31232]: pgmap v497: 129 pgs: 129 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 58 MiB/s wr, 930 op/s 2024-09-21T06:41:10.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:09 smithi100 ceph-mon[33401]: pgmap v497: 129 pgs: 129 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 58 MiB/s wr, 930 op/s 2024-09-21T06:41:11.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:11 smithi203 ceph-mon[33735]: pgmap v498: 129 pgs: 129 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 47 MiB/s wr, 757 op/s 2024-09-21T06:41:12.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:11 smithi067 ceph-mon[31232]: pgmap v498: 129 pgs: 129 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 47 MiB/s wr, 757 op/s 2024-09-21T06:41:12.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:11 smithi100 ceph-mon[33401]: pgmap v498: 129 pgs: 129 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 47 MiB/s wr, 757 op/s 2024-09-21T06:41:13.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:13 smithi203 ceph-mon[33735]: pgmap v499: 129 pgs: 129 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 61 MiB/s wr, 981 op/s 2024-09-21T06:41:14.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:13 smithi067 ceph-mon[31232]: pgmap v499: 129 pgs: 129 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 61 MiB/s wr, 981 op/s 2024-09-21T06:41:14.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:13 smithi100 ceph-mon[33401]: pgmap v499: 129 pgs: 129 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 61 MiB/s wr, 981 op/s 2024-09-21T06:41:15.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:15 smithi203 ceph-mon[33735]: pgmap v500: 129 pgs: 129 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 50 MiB/s wr, 799 op/s 2024-09-21T06:41:16.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:15 smithi067 ceph-mon[31232]: pgmap v500: 129 pgs: 129 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 50 MiB/s wr, 799 op/s 2024-09-21T06:41:16.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:15 smithi100 ceph-mon[33401]: pgmap v500: 129 pgs: 129 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 50 MiB/s wr, 799 op/s 2024-09-21T06:41:17.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:17 smithi203 ceph-mon[33735]: pgmap v501: 129 pgs: 129 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 51 MiB/s wr, 825 op/s 2024-09-21T06:41:18.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:17 smithi067 ceph-mon[31232]: pgmap v501: 129 pgs: 129 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 51 MiB/s wr, 825 op/s 2024-09-21T06:41:18.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:17 smithi100 ceph-mon[33401]: pgmap v501: 129 pgs: 129 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 51 MiB/s wr, 825 op/s 2024-09-21T06:41:19.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:19 smithi203 ceph-mon[33735]: pgmap v502: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 56 MiB/s wr, 899 op/s 2024-09-21T06:41:20.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:19 smithi067 ceph-mon[31232]: pgmap v502: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 56 MiB/s wr, 899 op/s 2024-09-21T06:41:20.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:19 smithi100 ceph-mon[33401]: pgmap v502: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 56 MiB/s wr, 899 op/s 2024-09-21T06:41:21.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:21 smithi203 ceph-mon[33735]: pgmap v503: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 46 MiB/s wr, 732 op/s 2024-09-21T06:41:22.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:21 smithi067 ceph-mon[31232]: pgmap v503: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 46 MiB/s wr, 732 op/s 2024-09-21T06:41:22.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:21 smithi100 ceph-mon[33401]: pgmap v503: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 46 MiB/s wr, 732 op/s 2024-09-21T06:41:23.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:23 smithi203 ceph-mon[33735]: pgmap v504: 129 pgs: 129 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 59 MiB/s wr, 951 op/s 2024-09-21T06:41:24.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:23 smithi067 ceph-mon[31232]: pgmap v504: 129 pgs: 129 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 59 MiB/s wr, 951 op/s 2024-09-21T06:41:24.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:23 smithi100 ceph-mon[33401]: pgmap v504: 129 pgs: 129 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 59 MiB/s wr, 951 op/s 2024-09-21T06:41:25.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:25 smithi203 ceph-mon[33735]: pgmap v505: 129 pgs: 129 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 48 MiB/s wr, 775 op/s 2024-09-21T06:41:26.041 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:25 smithi067 ceph-mon[31232]: pgmap v505: 129 pgs: 129 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 48 MiB/s wr, 775 op/s 2024-09-21T06:41:26.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:25 smithi100 ceph-mon[33401]: pgmap v505: 129 pgs: 129 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 48 MiB/s wr, 775 op/s 2024-09-21T06:41:27.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:27 smithi203 ceph-mon[33735]: pgmap v506: 129 pgs: 129 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 50 MiB/s wr, 804 op/s 2024-09-21T06:41:28.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:27 smithi067 ceph-mon[31232]: pgmap v506: 129 pgs: 129 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 50 MiB/s wr, 804 op/s 2024-09-21T06:41:28.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:27 smithi100 ceph-mon[33401]: pgmap v506: 129 pgs: 129 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 50 MiB/s wr, 804 op/s 2024-09-21T06:41:29.046 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:28 smithi067 ceph-mon[31232]: pgmap v507: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 54 MiB/s wr, 871 op/s 2024-09-21T06:41:29.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:28 smithi100 ceph-mon[33401]: pgmap v507: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 54 MiB/s wr, 871 op/s 2024-09-21T06:41:29.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:28 smithi203 ceph-mon[33735]: pgmap v507: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 54 MiB/s wr, 871 op/s 2024-09-21T06:41:31.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:31 smithi100 ceph-mon[33401]: pgmap v508: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 44 MiB/s wr, 712 op/s 2024-09-21T06:41:31.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:31 smithi203 ceph-mon[33735]: pgmap v508: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 44 MiB/s wr, 712 op/s 2024-09-21T06:41:31.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:31 smithi067 ceph-mon[31232]: pgmap v508: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 44 MiB/s wr, 712 op/s 2024-09-21T06:41:33.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:33 smithi100 ceph-mon[33401]: pgmap v509: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 58 MiB/s wr, 923 op/s 2024-09-21T06:41:33.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:33 smithi203 ceph-mon[33735]: pgmap v509: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 58 MiB/s wr, 923 op/s 2024-09-21T06:41:33.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:33 smithi067 ceph-mon[31232]: pgmap v509: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 58 MiB/s wr, 923 op/s 2024-09-21T06:41:35.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:35 smithi100 ceph-mon[33401]: pgmap v510: 129 pgs: 129 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 47 MiB/s wr, 754 op/s 2024-09-21T06:41:35.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:35 smithi203 ceph-mon[33735]: pgmap v510: 129 pgs: 129 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 47 MiB/s wr, 754 op/s 2024-09-21T06:41:35.541 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:35 smithi067 ceph-mon[31232]: pgmap v510: 129 pgs: 129 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 47 MiB/s wr, 754 op/s 2024-09-21T06:41:37.343 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:37 smithi067 ceph-mon[31232]: pgmap v511: 129 pgs: 129 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 49 MiB/s wr, 783 op/s 2024-09-21T06:41:37.344 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:37 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:41:37.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:37 smithi203 ceph-mon[33735]: pgmap v511: 129 pgs: 129 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 49 MiB/s wr, 783 op/s 2024-09-21T06:41:37.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:37 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:41:37.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:37 smithi100 ceph-mon[33401]: pgmap v511: 129 pgs: 129 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 49 MiB/s wr, 783 op/s 2024-09-21T06:41:37.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:37 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:41:38.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:38 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:41:38.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:38 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:41:38.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:38 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:41:38.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:38 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:41:38.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:38 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:41:38.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:38 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:41:38.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:38 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:41:38.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:38 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:41:38.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:38 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:41:39.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:39 smithi203 ceph-mon[33735]: pgmap v512: 129 pgs: 129 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 53 MiB/s wr, 848 op/s 2024-09-21T06:41:39.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:39 smithi067 ceph-mon[31232]: pgmap v512: 129 pgs: 129 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 53 MiB/s wr, 848 op/s 2024-09-21T06:41:39.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:39 smithi100 ceph-mon[33401]: pgmap v512: 129 pgs: 129 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 53 MiB/s wr, 848 op/s 2024-09-21T06:41:41.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:41 smithi203 ceph-mon[33735]: pgmap v513: 129 pgs: 129 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 44 MiB/s wr, 698 op/s 2024-09-21T06:41:41.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:41 smithi067 ceph-mon[31232]: pgmap v513: 129 pgs: 129 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 44 MiB/s wr, 698 op/s 2024-09-21T06:41:41.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:41 smithi100 ceph-mon[33401]: pgmap v513: 129 pgs: 129 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 44 MiB/s wr, 698 op/s 2024-09-21T06:41:43.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:43 smithi203 ceph-mon[33735]: pgmap v514: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 57 MiB/s wr, 906 op/s 2024-09-21T06:41:43.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:43 smithi067 ceph-mon[31232]: pgmap v514: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 57 MiB/s wr, 906 op/s 2024-09-21T06:41:43.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:43 smithi100 ceph-mon[33401]: pgmap v514: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 57 MiB/s wr, 906 op/s 2024-09-21T06:41:45.342 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-09-21T06:41:45.342 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:41:45.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:45 smithi203 ceph-mon[33735]: pgmap v515: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 46 MiB/s wr, 742 op/s 2024-09-21T06:41:45.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:45 smithi067 ceph-mon[31232]: pgmap v515: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 46 MiB/s wr, 742 op/s 2024-09-21T06:41:45.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:45 smithi100 ceph-mon[33401]: pgmap v515: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 46 MiB/s wr, 742 op/s 2024-09-21T06:41:45.849 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:41:47.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:47 smithi203 ceph-mon[33735]: pgmap v516: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 48 MiB/s wr, 764 op/s 2024-09-21T06:41:47.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:47 smithi067 ceph-mon[31232]: pgmap v516: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 48 MiB/s wr, 764 op/s 2024-09-21T06:41:47.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:47 smithi100 ceph-mon[33401]: pgmap v516: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 48 MiB/s wr, 764 op/s 2024-09-21T06:41:47.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.806+0000 7f50b0935640 1 -- 172.21.15.67:0/1845133888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50ac10a650 msgr2=0x7f50ac10aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:41:47.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.806+0000 7f50b0935640 1 --2- 172.21.15.67:0/1845133888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50ac10a650 0x7f50ac10aa60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f509c009900 tx=0x7f509c02a400 comp rx=0 tx=0).stop 2024-09-21T06:41:47.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.806+0000 7f50b0935640 1 -- 172.21.15.67:0/1845133888 shutdown_connections 2024-09-21T06:41:47.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.806+0000 7f50b0935640 1 --2- 172.21.15.67:0/1845133888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50ac110640 0x7f50ac116e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:47.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.806+0000 7f50b0935640 1 --2- 172.21.15.67:0/1845133888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50ac10fcf0 0x7f50ac110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:47.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.806+0000 7f50b0935640 1 --2- 172.21.15.67:0/1845133888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50ac10a650 0x7f50ac10aa60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:47.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.806+0000 7f50b0935640 1 -- 172.21.15.67:0/1845133888 >> 172.21.15.67:0/1845133888 conn(0x7f50ac06ee50 msgr2=0x7f50ac06f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:41:47.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.806+0000 7f50b0935640 1 -- 172.21.15.67:0/1845133888 shutdown_connections 2024-09-21T06:41:47.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.806+0000 7f50b0935640 1 -- 172.21.15.67:0/1845133888 wait complete. 2024-09-21T06:41:47.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.807+0000 7f50b0935640 1 Processor -- start 2024-09-21T06:41:47.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.807+0000 7f50b0935640 1 -- start start 2024-09-21T06:41:47.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.808+0000 7f50b0935640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50ac10a650 0x7f50ac120370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:47.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.808+0000 7f50b0935640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50ac10fcf0 0x7f50ac11a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:47.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.808+0000 7f50ab7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50ac10a650 0x7f50ac120370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:47.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.808+0000 7f50ab7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50ac10a650 0x7f50ac120370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.67:40414/0 (socket says 172.21.15.67:40414) 2024-09-21T06:41:47.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.808+0000 7f50b0935640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50ac110640 0x7f50ac11ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:47.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.808+0000 7f50b0935640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac118a20 con 0x7f50ac10fcf0 2024-09-21T06:41:47.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.808+0000 7f50b0935640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac1188a0 con 0x7f50ac10a650 2024-09-21T06:41:47.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.808+0000 7f50b0935640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac118ba0 con 0x7f50ac110640 2024-09-21T06:41:47.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.808+0000 7f50aaffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50ac10fcf0 0x7f50ac11a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:47.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.808+0000 7f50aaffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50ac10fcf0 0x7f50ac11a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53304/0 (socket says 172.21.15.67:53304) 2024-09-21T06:41:47.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.808+0000 7f50aaffd640 1 -- 172.21.15.67:0/814242338 learned_addr learned my addr 172.21.15.67:0/814242338 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:41:47.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.809+0000 7f50ab7fe640 1 -- 172.21.15.67:0/814242338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50ac110640 msgr2=0x7f50ac11ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:41:47.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.809+0000 7f50ab7fe640 1 --2- 172.21.15.67:0/814242338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50ac110640 0x7f50ac11ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:47.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.809+0000 7f50ab7fe640 1 -- 172.21.15.67:0/814242338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50ac10fcf0 msgr2=0x7f50ac11a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:41:47.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.809+0000 7f50ab7fe640 1 --2- 172.21.15.67:0/814242338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50ac10fcf0 0x7f50ac11a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:47.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.809+0000 7f50ab7fe640 1 -- 172.21.15.67:0/814242338 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50ac11b3b0 con 0x7f50ac10a650 2024-09-21T06:41:47.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.809+0000 7f50ab7fe640 1 --2- 172.21.15.67:0/814242338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50ac10a650 0x7f50ac120370 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f509c02a920 tx=0x7f509c0045f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:41:47.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.809+0000 7f50a8ff9640 1 -- 172.21.15.67:0/814242338 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c040070 con 0x7f50ac10a650 2024-09-21T06:41:47.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.810+0000 7f50a8ff9640 1 -- 172.21.15.67:0/814242338 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f509c031600 con 0x7f50ac10a650 2024-09-21T06:41:47.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.810+0000 7f50b0935640 1 -- 172.21.15.67:0/814242338 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50ac1bdbc0 con 0x7f50ac10a650 2024-09-21T06:41:47.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.810+0000 7f50a8ff9640 1 -- 172.21.15.67:0/814242338 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c035020 con 0x7f50ac10a650 2024-09-21T06:41:47.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.810+0000 7f50b0935640 1 -- 172.21.15.67:0/814242338 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50ac1be0a0 con 0x7f50ac10a650 2024-09-21T06:41:47.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.812+0000 7f50b0935640 1 -- 172.21.15.67:0/814242338 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50ac10b630 con 0x7f50ac10a650 2024-09-21T06:41:47.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.815+0000 7f50a8ff9640 1 -- 172.21.15.67:0/814242338 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f509c043070 con 0x7f50ac10a650 2024-09-21T06:41:47.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.817+0000 7f50a8ff9640 1 --2- 172.21.15.67:0/814242338 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f50980b6640 0x7f50980b8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:47.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.818+0000 7f50a8ff9640 1 -- 172.21.15.67:0/814242338 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f509c107f90 con 0x7f50ac10a650 2024-09-21T06:41:47.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.818+0000 7f50aaffd640 1 --2- 172.21.15.67:0/814242338 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f50980b6640 0x7f50980b8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:47.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.820+0000 7f50aaffd640 1 --2- 172.21.15.67:0/814242338 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f50980b6640 0x7f50980b8b50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f50ac10cbe0 tx=0x7f5094005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:41:47.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:47.827+0000 7f50a8ff9640 1 -- 172.21.15.67:0/814242338 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f509c0ce450 con 0x7f50ac10a650 2024-09-21T06:41:48.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:48.286+0000 7f50b0935640 1 -- 172.21.15.67:0/814242338 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50ac02e640 con 0x7f50ac10a650 2024-09-21T06:41:48.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:48.287+0000 7f50a8ff9640 1 -- 172.21.15.67:0/814242338 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7f509c0d2830 con 0x7f50ac10a650 2024-09-21T06:41:48.289 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:41:48.289 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:41:48.289 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:41:48.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:48.293+0000 7f50b0935640 1 -- 172.21.15.67:0/814242338 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f50980b6640 msgr2=0x7f50980b8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:41:48.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:48.293+0000 7f50b0935640 1 --2- 172.21.15.67:0/814242338 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f50980b6640 0x7f50980b8b50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f50ac10cbe0 tx=0x7f5094005f90 comp rx=0 tx=0).stop 2024-09-21T06:41:48.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:48.293+0000 7f50b0935640 1 -- 172.21.15.67:0/814242338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50ac10a650 msgr2=0x7f50ac120370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:41:48.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:48.293+0000 7f50b0935640 1 --2- 172.21.15.67:0/814242338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50ac10a650 0x7f50ac120370 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f509c02a920 tx=0x7f509c0045f0 comp rx=0 tx=0).stop 2024-09-21T06:41:48.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:48.293+0000 7f50b0935640 1 -- 172.21.15.67:0/814242338 shutdown_connections 2024-09-21T06:41:48.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:48.293+0000 7f50b0935640 1 --2- 172.21.15.67:0/814242338 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f50980b6640 0x7f50980b8b50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:48.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:48.293+0000 7f50b0935640 1 --2- 172.21.15.67:0/814242338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50ac110640 0x7f50ac11ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:48.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:48.293+0000 7f50b0935640 1 --2- 172.21.15.67:0/814242338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50ac10fcf0 0x7f50ac11a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:48.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:48.293+0000 7f50b0935640 1 --2- 172.21.15.67:0/814242338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50ac10a650 0x7f50ac120370 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:48.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:48.293+0000 7f50b0935640 1 -- 172.21.15.67:0/814242338 >> 172.21.15.67:0/814242338 conn(0x7f50ac06ee50 msgr2=0x7f50ac10dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:41:48.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:48.297+0000 7f50b0935640 1 -- 172.21.15.67:0/814242338 shutdown_connections 2024-09-21T06:41:48.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:48.297+0000 7f50b0935640 1 -- 172.21.15.67:0/814242338 wait complete. 2024-09-21T06:41:48.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:48 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/814242338' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:41:48.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:48 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/814242338' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:41:48.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:48 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/814242338' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:41:49.200 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-09-21T06:41:49.200 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24671 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-09-21T06:41:49.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:49 smithi203 ceph-mon[33735]: pgmap v517: 129 pgs: 129 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 52 MiB/s wr, 831 op/s 2024-09-21T06:41:49.758 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:41:49.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:49 smithi067 ceph-mon[31232]: pgmap v517: 129 pgs: 129 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 52 MiB/s wr, 831 op/s 2024-09-21T06:41:49.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:49 smithi100 ceph-mon[33401]: pgmap v517: 129 pgs: 129 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 52 MiB/s wr, 831 op/s 2024-09-21T06:41:50.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.435+0000 7fa3f759e640 1 -- 172.21.15.67:0/1713590458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3f810cb00 msgr2=0x7fa3f810cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:41:50.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.435+0000 7fa3f759e640 1 --2- 172.21.15.67:0/1713590458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3f810cb00 0x7fa3f810cf10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0009ca0 tx=0x7fa3f002a8f0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.436+0000 7fa3f759e640 1 -- 172.21.15.67:0/1713590458 shutdown_connections 2024-09-21T06:41:50.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.436+0000 7fa3f759e640 1 --2- 172.21.15.67:0/1713590458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3f810cb00 0x7fa3f810cf10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.436+0000 7fa3f759e640 1 --2- 172.21.15.67:0/1713590458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3f810d450 0x7fa3f8116ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.436+0000 7fa3f759e640 1 --2- 172.21.15.67:0/1713590458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3f810a650 0x7fa3f810aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.436+0000 7fa3f759e640 1 -- 172.21.15.67:0/1713590458 >> 172.21.15.67:0/1713590458 conn(0x7fa3f806ee90 msgr2=0x7fa3f806f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:41:50.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.436+0000 7fa3f759e640 1 -- 172.21.15.67:0/1713590458 shutdown_connections 2024-09-21T06:41:50.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.436+0000 7fa3f759e640 1 -- 172.21.15.67:0/1713590458 wait complete. 2024-09-21T06:41:50.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.437+0000 7fa3f759e640 1 Processor -- start 2024-09-21T06:41:50.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.437+0000 7fa3f759e640 1 -- start start 2024-09-21T06:41:50.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.437+0000 7fa3f759e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3f810a650 0x7fa3f8116bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:50.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.437+0000 7fa3f759e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3f810cb00 0x7fa3f8113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:50.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.438+0000 7fa3f759e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3f810d450 0x7fa3f81143f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:50.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.438+0000 7fa3f759e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f811a290 con 0x7fa3f810a650 2024-09-21T06:41:50.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.438+0000 7fa3f759e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f811a110 con 0x7fa3f810d450 2024-09-21T06:41:50.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.438+0000 7fa3f759e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f811a410 con 0x7fa3f810cb00 2024-09-21T06:41:50.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.438+0000 7fa3f5d9b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3f810cb00 0x7fa3f8113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:50.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.438+0000 7fa3f5d9b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3f810cb00 0x7fa3f8113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.67:51804/0 (socket says 172.21.15.67:51804) 2024-09-21T06:41:50.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.438+0000 7fa3f5d9b640 1 -- 172.21.15.67:0/3947477176 learned_addr learned my addr 172.21.15.67:0/3947477176 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:41:50.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.438+0000 7fa3f6d9d640 1 --2- 172.21.15.67:0/3947477176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3f810d450 0x7fa3f81143f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:50.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.439+0000 7fa3f5d9b640 1 -- 172.21.15.67:0/3947477176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3f810d450 msgr2=0x7fa3f81143f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:41:50.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.439+0000 7fa3f5d9b640 1 --2- 172.21.15.67:0/3947477176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3f810d450 0x7fa3f81143f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.439+0000 7fa3f5d9b640 1 -- 172.21.15.67:0/3947477176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3f810a650 msgr2=0x7fa3f8116bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:41:50.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.439+0000 7fa3f5d9b640 1 --2- 172.21.15.67:0/3947477176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3f810a650 0x7fa3f8116bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.439+0000 7fa3f5d9b640 1 -- 172.21.15.67:0/3947477176 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f8114bc0 con 0x7fa3f810cb00 2024-09-21T06:41:50.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.439+0000 7fa3f5d9b640 1 --2- 172.21.15.67:0/3947477176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3f810cb00 0x7fa3f8113eb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0005e30 tx=0x7fa3f0030b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:41:50.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.439+0000 7fa3e77fe640 1 -- 172.21.15.67:0/3947477176 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0003b70 con 0x7fa3f810cb00 2024-09-21T06:41:50.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.439+0000 7fa3f759e640 1 -- 172.21.15.67:0/3947477176 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f81b53e0 con 0x7fa3f810cb00 2024-09-21T06:41:50.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.440+0000 7fa3f759e640 1 -- 172.21.15.67:0/3947477176 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f81b58e0 con 0x7fa3f810cb00 2024-09-21T06:41:50.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.440+0000 7fa3e77fe640 1 -- 172.21.15.67:0/3947477176 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa3f002d070 con 0x7fa3f810cb00 2024-09-21T06:41:50.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.440+0000 7fa3e77fe640 1 -- 172.21.15.67:0/3947477176 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0045690 con 0x7fa3f810cb00 2024-09-21T06:41:50.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.445+0000 7fa3e77fe640 1 -- 172.21.15.67:0/3947477176 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa3f003b400 con 0x7fa3f810cb00 2024-09-21T06:41:50.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.446+0000 7fa3e77fe640 1 --2- 172.21.15.67:0/3947477176 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fa3e00b6640 0x7fa3e00b8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:50.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.447+0000 7fa3f659c640 1 --2- 172.21.15.67:0/3947477176 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fa3e00b6640 0x7fa3e00b8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:50.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.447+0000 7fa3e77fe640 1 -- 172.21.15.67:0/3947477176 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fa3f01095e0 con 0x7fa3f810cb00 2024-09-21T06:41:50.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.448+0000 7fa402ee3740 1 Processor -- start 2024-09-21T06:41:50.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.448+0000 7fa402ee3740 1 -- start start 2024-09-21T06:41:50.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.448+0000 7fa402ee3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56271ab92840 0x56271af397e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:50.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.448+0000 7fa402ee3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56271ab69980 0x56271af39d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:50.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.448+0000 7fa402ee3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271ab71b60 0x56271af3a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:50.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.448+0000 7fa402ee3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56271aeafb90 con 0x56271ab92840 2024-09-21T06:41:50.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.448+0000 7fa402ee3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56271ae9efc0 con 0x56271ab71b60 2024-09-21T06:41:50.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.448+0000 7fa402ee3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56271aea2dd0 con 0x56271ab69980 2024-09-21T06:41:50.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.449+0000 7fa3f659c640 1 --2- 172.21.15.67:0/3947477176 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fa3e00b6640 0x7fa3e00b8b50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa3e80058c0 tx=0x7fa3e8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:41:50.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.449+0000 7fa3f659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56271ab92840 0x56271af397e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:50.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.449+0000 7fa3f659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56271ab92840 0x56271af397e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53330/0 (socket says 172.21.15.67:53330) 2024-09-21T06:41:50.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.449+0000 7fa3f659c640 1 -- 172.21.15.67:0/4276052714 learned_addr learned my addr 172.21.15.67:0/4276052714 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:41:50.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.449+0000 7fa3f6d9d640 1 --2- 172.21.15.67:0/4276052714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271ab71b60 0x56271af3a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:50.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.450+0000 7fa3f659c640 1 -- 172.21.15.67:0/4276052714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56271ab69980 msgr2=0x56271af39d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:41:50.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.450+0000 7fa3f659c640 1 --2- 172.21.15.67:0/4276052714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56271ab69980 0x56271af39d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.450+0000 7fa3f659c640 1 -- 172.21.15.67:0/4276052714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271ab71b60 msgr2=0x56271af3a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:41:50.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.450+0000 7fa3f659c640 1 --2- 172.21.15.67:0/4276052714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271ab71b60 0x56271af3a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.450+0000 7fa3f659c640 1 -- 172.21.15.67:0/4276052714 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56271ae26060 con 0x56271ab92840 2024-09-21T06:41:50.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.453+0000 7fa3f659c640 1 --2- 172.21.15.67:0/4276052714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56271ab92840 0x56271af397e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800ed30 tx=0x7fa3e8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22e58bcb8d12198 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:41:50.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.454+0000 7fa3e4ff9640 1 -- 172.21.15.67:0/4276052714 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8020070 con 0x56271ab92840 2024-09-21T06:41:50.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.454+0000 7fa3e4ff9640 1 -- 172.21.15.67:0/4276052714 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa3e80258f0 con 0x56271ab92840 2024-09-21T06:41:50.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.454+0000 7fa3e4ff9640 1 -- 172.21.15.67:0/4276052714 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e801c800 con 0x56271ab92840 2024-09-21T06:41:50.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.456+0000 7fa402ee3740 1 -- 172.21.15.67:0/4276052714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56271ab92840 msgr2=0x56271af397e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:41:50.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.456+0000 7fa402ee3740 1 --2- 172.21.15.67:0/4276052714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56271ab92840 0x56271af397e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800ed30 tx=0x7fa3e8009f90 comp rx=0 tx=0).stop 2024-09-21T06:41:50.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.457+0000 7fa402ee3740 1 -- 172.21.15.67:0/4276052714 shutdown_connections 2024-09-21T06:41:50.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.457+0000 7fa402ee3740 1 --2- 172.21.15.67:0/4276052714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271ab71b60 0x56271af3a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.457+0000 7fa402ee3740 1 --2- 172.21.15.67:0/4276052714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56271ab69980 0x56271af39d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.457+0000 7fa402ee3740 1 --2- 172.21.15.67:0/4276052714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56271ab92840 0x56271af397e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.457+0000 7fa402ee3740 1 -- 172.21.15.67:0/4276052714 >> 172.21.15.67:0/4276052714 conn(0x56271ab86990 msgr2=0x56271af376d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:41:50.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.458+0000 7fa402ee3740 1 -- 172.21.15.67:0/4276052714 shutdown_connections 2024-09-21T06:41:50.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.460+0000 7fa402ee3740 1 -- 172.21.15.67:0/4276052714 wait complete. 2024-09-21T06:41:50.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.460+0000 7fa402ee3740 1 -- start start 2024-09-21T06:41:50.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.460+0000 7fa402ee3740 1 Processor -- start 2024-09-21T06:41:50.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.461+0000 7fa402ee3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271ab69980 0x56271af45390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:50.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.461+0000 7fa402ee3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56271ab71b60 0x56271af458d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:50.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.461+0000 7fa402ee3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56271ab92840 0x56271af51340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:50.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.461+0000 7fa402ee3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56271ad007a0 con 0x56271ab92840 2024-09-21T06:41:50.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.461+0000 7fa402ee3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56271ae440a0 con 0x56271ab69980 2024-09-21T06:41:50.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.461+0000 7fa402ee3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56271af47fc0 con 0x56271ab71b60 2024-09-21T06:41:50.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.461+0000 7fa402ee3740 10 client.0 _ll_register_callbacks cb 0x56271ae213c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:41:50.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.461+0000 7fa3f659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271ab69980 0x56271af45390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:50.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.461+0000 7fa3f659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271ab69980 0x56271af45390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.67:40456/0 (socket says 172.21.15.67:40456) 2024-09-21T06:41:50.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.461+0000 7fa3f659c640 1 -- 172.21.15.67:0/1278249088 learned_addr learned my addr 172.21.15.67:0/1278249088 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:41:50.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.462+0000 7fa3f5d9b640 1 --2- 172.21.15.67:0/1278249088 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56271ab71b60 0x56271af458d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:50.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.462+0000 7fa3f6d9d640 1 --2- 172.21.15.67:0/1278249088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56271ab92840 0x56271af51340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:50.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.462+0000 7fa3f659c640 1 -- 172.21.15.67:0/1278249088 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56271ab71b60 msgr2=0x56271af458d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:41:50.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.462+0000 7fa3f659c640 1 --2- 172.21.15.67:0/1278249088 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56271ab71b60 0x56271af458d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.462+0000 7fa3f659c640 1 -- 172.21.15.67:0/1278249088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56271ab92840 msgr2=0x56271af51340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:41:50.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.462+0000 7fa3f659c640 1 --2- 172.21.15.67:0/1278249088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56271ab92840 0x56271af51340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.462+0000 7fa3f659c640 1 -- 172.21.15.67:0/1278249088 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56271ae32880 con 0x56271ab69980 2024-09-21T06:41:50.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.462+0000 7fa3f659c640 1 --2- 172.21.15.67:0/1278249088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271ab69980 0x56271af45390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8004700 tx=0x7fa3e800ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5eb5d0f16a7b620e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:41:50.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.462+0000 7fa3ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-21T06:41:50.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.463+0000 7fa3ddffb640 1 -- 172.21.15.67:0/1278249088 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8033020 con 0x56271ab69980 2024-09-21T06:41:50.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.463+0000 7fa402ee3740 1 -- 172.21.15.67:0/1278249088 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3e80258f0 con 0x56271ab69980 2024-09-21T06:41:50.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.463+0000 7fa3ddffb640 1 -- 172.21.15.67:0/1278249088 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fa3e8005020 con 0x56271ab69980 2024-09-21T06:41:50.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.463+0000 7fa3ddffb640 1 -- 172.21.15.67:0/1278249088 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa3e803b050 con 0x56271ab69980 2024-09-21T06:41:50.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.463+0000 7fa402ee3740 10 client.25361 fetch_fsmap learned FSMap version 21 2024-09-21T06:41:50.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.463+0000 7fa402ee3740 1 -- 172.21.15.67:0/1278249088 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56271ac0cb90 con 0x56271ab69980 2024-09-21T06:41:50.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.463+0000 7fa3ddffb640 1 -- 172.21.15.67:0/1278249088 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa3e80258f0 con 0x56271ab69980 2024-09-21T06:41:50.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.464+0000 7fa3ddffb640 1 -- 172.21.15.67:0/1278249088 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa3e801c500 con 0x56271ab69980 2024-09-21T06:41:50.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.465+0000 7fa402ee3740 10 client.25361 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:41:50.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.465+0000 7fa402ee3740 10 client.25361 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:41:50.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.465+0000 7fa402ee3740 20 client.25361 populate_metadata read hostname 'smithi067' 2024-09-21T06:41:50.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.465+0000 7fa402ee3740 1 --2- 172.21.15.67:0/1278249088 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x56271ab511c0 0x56271af57270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:50.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.465+0000 7fa402ee3740 4 client.25361 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:41:50.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.465+0000 7fa402ee3740 1 -- 172.21.15.67:0/1278249088 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56271aeb1ae0 con 0x56271ab511c0 2024-09-21T06:41:50.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.465+0000 7fa3f5d9b640 1 --2- 172.21.15.67:0/1278249088 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x56271ab511c0 0x56271af57270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:50.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.466+0000 7fa3f5d9b640 1 --2- 172.21.15.67:0/1278249088 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x56271ab511c0 0x56271af57270 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9f8fb3dea8cc9ff server_cookie=6de62e5c8e447536 in_seq=0 out_seq=0 2024-09-21T06:41:50.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.466+0000 7fa3ddffb640 10 client.25361 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:41:50.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.468+0000 7fa3ddffb640 1 -- 172.21.15.67:0/1278249088 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56271aeb1ae0 con 0x56271ab511c0 2024-09-21T06:41:50.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.468+0000 7fa3ddffb640 10 client.25361 handle_command_reply: tid=0 2024-09-21T06:41:50.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.469+0000 7fa402ee3740 1 client.25361 shutdown 2024-09-21T06:41:50.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.469+0000 7fa402ee3740 1 -- 172.21.15.67:0/1278249088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271ab69980 msgr2=0x56271af45390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:41:50.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.469+0000 7fa402ee3740 1 --2- 172.21.15.67:0/1278249088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271ab69980 0x56271af45390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8004700 tx=0x7fa3e800ef90 comp rx=0 tx=0).stop 2024-09-21T06:41:50.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.469+0000 7fa402ee3740 1 -- 172.21.15.67:0/1278249088 shutdown_connections 2024-09-21T06:41:50.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.469+0000 7fa402ee3740 1 --2- 172.21.15.67:0/1278249088 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x56271ab511c0 0x56271af57270 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.469+0000 7fa402ee3740 1 --2- 172.21.15.67:0/1278249088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56271ab92840 0x56271af51340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.469+0000 7fa402ee3740 1 --2- 172.21.15.67:0/1278249088 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56271ab71b60 0x56271af458d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.469+0000 7fa402ee3740 1 --2- 172.21.15.67:0/1278249088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271ab69980 0x56271af45390 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.469+0000 7fa402ee3740 1 -- 172.21.15.67:0/1278249088 >> 172.21.15.67:0/1278249088 conn(0x56271ab86990 msgr2=0x56271af3c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:41:50.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.469+0000 7fa3ddffb640 0 client.25361 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:41:50.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.469+0000 7fa402ee3740 1 -- 172.21.15.67:0/1278249088 shutdown_connections 2024-09-21T06:41:50.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.470+0000 7fa402ee3740 1 -- 172.21.15.67:0/1278249088 wait complete. 2024-09-21T06:41:50.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.470+0000 7fa402ee3740 20 client.25361 trim_cache size 0 max 16384 2024-09-21T06:41:50.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.506+0000 7fa402ee3740 1 Processor -- start 2024-09-21T06:41:50.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.506+0000 7fa402ee3740 1 -- start start 2024-09-21T06:41:50.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.506+0000 7fa402ee3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271ab69980 0x56271af3cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:50.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.506+0000 7fa402ee3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56271ab71b60 0x56271af3e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:50.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.506+0000 7fa402ee3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56271ab92840 0x56271af3edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:50.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.506+0000 7fa402ee3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d800dd70 con 0x56271ab92840 2024-09-21T06:41:50.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.506+0000 7fa402ee3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d80105f0 con 0x56271ab69980 2024-09-21T06:41:50.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.506+0000 7fa402ee3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d800e3e0 con 0x56271ab71b60 2024-09-21T06:41:50.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.506+0000 7fa3f6d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56271ab92840 0x56271af3edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:50.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.506+0000 7fa3f6d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56271ab92840 0x56271af3edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53340/0 (socket says 172.21.15.67:53340) 2024-09-21T06:41:50.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.506+0000 7fa3f6d9d640 1 -- 172.21.15.67:0/3114099217 learned_addr learned my addr 172.21.15.67:0/3114099217 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:41:50.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.507+0000 7fa3f6d9d640 1 -- 172.21.15.67:0/3114099217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56271ab71b60 msgr2=0x56271af3e870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:41:50.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.507+0000 7fa3f6d9d640 1 --2- 172.21.15.67:0/3114099217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56271ab71b60 0x56271af3e870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.507+0000 7fa3f6d9d640 1 -- 172.21.15.67:0/3114099217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271ab69980 msgr2=0x56271af3cff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:41:50.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.507+0000 7fa3f6d9d640 1 --2- 172.21.15.67:0/3114099217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271ab69980 0x56271af3cff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.507+0000 7fa3f6d9d640 1 -- 172.21.15.67:0/3114099217 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56271af38490 con 0x56271ab92840 2024-09-21T06:41:50.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.508+0000 7fa3f6d9d640 1 --2- 172.21.15.67:0/3114099217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56271ab92840 0x56271af3edb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec0099f0 tx=0x7fa3ec009060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875fb05398e6090a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:41:50.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.509+0000 7fa3dffff640 1 -- 172.21.15.67:0/3114099217 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec01a070 con 0x56271ab92840 2024-09-21T06:41:50.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.509+0000 7fa3dffff640 1 -- 172.21.15.67:0/3114099217 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa3ec012d00 con 0x56271ab92840 2024-09-21T06:41:50.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.510+0000 7fa3dffff640 1 -- 172.21.15.67:0/3114099217 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec014610 con 0x56271ab92840 2024-09-21T06:41:50.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.510+0000 7fa402ee3740 1 -- 172.21.15.67:0/3114099217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56271ab92840 msgr2=0x56271af3edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:41:50.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.510+0000 7fa402ee3740 1 --2- 172.21.15.67:0/3114099217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56271ab92840 0x56271af3edb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec0099f0 tx=0x7fa3ec009060 comp rx=0 tx=0).stop 2024-09-21T06:41:50.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.511+0000 7fa402ee3740 1 -- 172.21.15.67:0/3114099217 shutdown_connections 2024-09-21T06:41:50.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.511+0000 7fa402ee3740 1 --2- 172.21.15.67:0/3114099217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56271ab92840 0x56271af3edb0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.511+0000 7fa402ee3740 1 --2- 172.21.15.67:0/3114099217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56271ab71b60 0x56271af3e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.511+0000 7fa402ee3740 1 --2- 172.21.15.67:0/3114099217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271ab69980 0x56271af3cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.511+0000 7fa402ee3740 1 -- 172.21.15.67:0/3114099217 >> 172.21.15.67:0/3114099217 conn(0x56271ab86990 msgr2=0x56271af3c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:41:50.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.512+0000 7fa402ee3740 1 -- 172.21.15.67:0/3114099217 shutdown_connections 2024-09-21T06:41:50.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.512+0000 7fa402ee3740 1 -- 172.21.15.67:0/3114099217 wait complete. 2024-09-21T06:41:50.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.512+0000 7fa402ee3740 1 -- start start 2024-09-21T06:41:50.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.512+0000 7fa402ee3740 1 Processor -- start 2024-09-21T06:41:50.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.513+0000 7fa402ee3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56271ab69980 0x56271af5a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:50.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.513+0000 7fa402ee3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56271ab71b60 0x56271af56d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:50.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.514+0000 7fa402ee3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271ab92840 0x56271af57280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:50.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.514+0000 7fa402ee3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d800eab0 con 0x56271ab69980 2024-09-21T06:41:50.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.514+0000 7fa402ee3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d800ff20 con 0x56271ab92840 2024-09-21T06:41:50.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.514+0000 7fa402ee3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d800f180 con 0x56271ab71b60 2024-09-21T06:41:50.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.514+0000 7fa402ee3740 10 client.0 _ll_register_callbacks cb 0x7fa3e800ec00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:41:50.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.514+0000 7fa3f5d9b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56271ab71b60 0x56271af56d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:50.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.514+0000 7fa3f5d9b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56271ab71b60 0x56271af56d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.67:51822/0 (socket says 172.21.15.67:51822) 2024-09-21T06:41:50.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.514+0000 7fa3f5d9b640 1 -- 172.21.15.67:0/1974285740 learned_addr learned my addr 172.21.15.67:0/1974285740 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:41:50.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.514+0000 7fa3f6d9d640 1 --2- 172.21.15.67:0/1974285740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271ab92840 0x56271af57280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:50.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.515+0000 7fa3f5d9b640 1 -- 172.21.15.67:0/1974285740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271ab92840 msgr2=0x56271af57280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:41:50.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.515+0000 7fa3f5d9b640 1 --2- 172.21.15.67:0/1974285740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271ab92840 0x56271af57280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.515+0000 7fa3f5d9b640 1 -- 172.21.15.67:0/1974285740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56271ab69980 msgr2=0x56271af5a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:41:50.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.515+0000 7fa3f5d9b640 1 --2- 172.21.15.67:0/1974285740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56271ab69980 0x56271af5a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.515+0000 7fa3f5d9b640 1 -- 172.21.15.67:0/1974285740 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56271af57840 con 0x56271ab71b60 2024-09-21T06:41:50.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.516+0000 7fa3f5d9b640 1 --2- 172.21.15.67:0/1974285740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56271ab71b60 0x56271af56d40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa3f00cb910 tx=0x7fa3f0004940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37c77d89b4e8a468 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:41:50.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.516+0000 7fa3ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T06:41:50.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.516+0000 7fa3ddffb640 1 -- 172.21.15.67:0/1974285740 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0108040 con 0x56271ab71b60 2024-09-21T06:41:50.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.516+0000 7fa402ee3740 1 -- 172.21.15.67:0/1974285740 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3ec012d00 con 0x56271ab71b60 2024-09-21T06:41:50.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.516+0000 7fa3ddffb640 1 -- 172.21.15.67:0/1974285740 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fa3f0106dc0 con 0x56271ab71b60 2024-09-21T06:41:50.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.517+0000 7fa3ddffb640 1 -- 172.21.15.67:0/1974285740 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa3f00a9080 con 0x56271ab71b60 2024-09-21T06:41:50.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.521+0000 7fa402ee3740 10 client.25324 fetch_fsmap learned FSMap version 21 2024-09-21T06:41:50.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.521+0000 7fa402ee3740 1 -- 172.21.15.67:0/1974285740 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56271afc2760 con 0x56271ab71b60 2024-09-21T06:41:50.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.525+0000 7fa3ddffb640 1 -- 172.21.15.67:0/1974285740 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa3ec012d00 con 0x56271ab71b60 2024-09-21T06:41:50.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.525+0000 7fa3ddffb640 1 -- 172.21.15.67:0/1974285740 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa3f0109250 con 0x56271ab71b60 2024-09-21T06:41:50.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.525+0000 7fa402ee3740 10 client.25324 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:41:50.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.525+0000 7fa402ee3740 10 client.25324 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:41:50.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.525+0000 7fa402ee3740 20 client.25324 populate_metadata read hostname 'smithi067' 2024-09-21T06:41:50.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.525+0000 7fa402ee3740 1 --2- 172.21.15.67:0/1974285740 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x56271ab26570 0x56271afc5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:50.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.525+0000 7fa402ee3740 4 client.25324 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-09-21T06:41:50.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.525+0000 7fa402ee3740 1 -- 172.21.15.67:0/1974285740 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x56271aead590 con 0x56271ab26570 2024-09-21T06:41:50.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.525+0000 7fa3f659c640 1 --2- 172.21.15.67:0/1974285740 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x56271ab26570 0x56271afc5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:50.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.526+0000 7fa3ddffb640 10 client.25324 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:41:50.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.526+0000 7fa3f659c640 1 --2- 172.21.15.67:0/1974285740 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x56271ab26570 0x56271afc5010 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26227cf32be921a8 server_cookie=a83878c78623b9e9 in_seq=0 out_seq=0 2024-09-21T06:41:50.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.561+0000 7fa3ddffb640 1 -- 172.21.15.67:0/1974285740 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56271aead590 con 0x56271ab26570 2024-09-21T06:41:50.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.561+0000 7fa3ddffb640 10 client.25324 handle_command_reply: tid=0 2024-09-21T06:41:50.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.561+0000 7fa402ee3740 1 client.25324 shutdown 2024-09-21T06:41:50.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.562+0000 7fa402ee3740 1 -- 172.21.15.67:0/1974285740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56271ab71b60 msgr2=0x56271af56d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:41:50.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.562+0000 7fa402ee3740 1 --2- 172.21.15.67:0/1974285740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56271ab71b60 0x56271af56d40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa3f00cb910 tx=0x7fa3f0004940 comp rx=0 tx=0).stop 2024-09-21T06:41:50.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.562+0000 7fa402ee3740 1 -- 172.21.15.67:0/1974285740 shutdown_connections 2024-09-21T06:41:50.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.562+0000 7fa402ee3740 1 --2- 172.21.15.67:0/1974285740 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x56271ab26570 0x56271afc5010 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.562+0000 7fa402ee3740 1 --2- 172.21.15.67:0/1974285740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271ab92840 0x56271af57280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.562+0000 7fa402ee3740 1 --2- 172.21.15.67:0/1974285740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56271ab71b60 0x56271af56d40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.562+0000 7fa402ee3740 1 --2- 172.21.15.67:0/1974285740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56271ab69980 0x56271af5a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.562+0000 7fa402ee3740 1 -- 172.21.15.67:0/1974285740 >> 172.21.15.67:0/1974285740 conn(0x56271ab86990 msgr2=0x56271af36fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:41:50.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.562+0000 7fa3ddffb640 0 client.25324 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:41:50.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.562+0000 7fa402ee3740 1 -- 172.21.15.67:0/1974285740 shutdown_connections 2024-09-21T06:41:50.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.562+0000 7fa402ee3740 1 -- 172.21.15.67:0/1974285740 wait complete. 2024-09-21T06:41:50.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.562+0000 7fa402ee3740 20 client.25324 trim_cache size 0 max 16384 2024-09-21T06:41:50.565 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-21T06:41:50.565 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 21, 2024-09-21T06:41:50.565 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-09-21T06:41:50.565 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 18, 2024-09-21T06:41:50.565 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-21T06:41:50.565 INFO:teuthology.orchestra.run.smithi067.stdout: "589bb4a6": { 2024-09-21T06:41:50.565 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 18, 2024-09-21T06:41:50.565 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-21T06:41:50.565 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:41:50.565 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-21T06:41:50.565 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:41:50.566 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:41:50.566 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-21T06:41:50.566 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 130.0, 2024-09-21T06:41:50.566 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-21T06:41:50.566 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-21T06:41:50.566 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-21T06:41:50.566 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:41:50.566 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-21T06:41:50.566 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:41:50.566 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:41:50.566 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:41:50.566 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:41:50.567 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:41:50.567 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:41:50.567 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-21T06:41:50.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.563+0000 7fa3f759e640 1 -- 172.21.15.67:0/3947477176 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fa3e00b6640 msgr2=0x7fa3e00b8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:41:50.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.563+0000 7fa3f759e640 1 --2- 172.21.15.67:0/3947477176 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fa3e00b6640 0x7fa3e00b8b50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa3e80058c0 tx=0x7fa3e8005850 comp rx=0 tx=0).stop 2024-09-21T06:41:50.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.563+0000 7fa3f759e640 1 -- 172.21.15.67:0/3947477176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3f810cb00 msgr2=0x7fa3f8113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:41:50.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.563+0000 7fa3f759e640 1 --2- 172.21.15.67:0/3947477176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3f810cb00 0x7fa3f8113eb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0005e30 tx=0x7fa3f0030b90 comp rx=0 tx=0).stop 2024-09-21T06:41:50.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.563+0000 7fa3f759e640 1 -- 172.21.15.67:0/3947477176 shutdown_connections 2024-09-21T06:41:50.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.563+0000 7fa3f759e640 1 --2- 172.21.15.67:0/3947477176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3f810d450 0x7fa3f81143f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.563+0000 7fa3f759e640 1 --2- 172.21.15.67:0/3947477176 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fa3e00b6640 0x7fa3e00b8b50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.563+0000 7fa3f759e640 1 --2- 172.21.15.67:0/3947477176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3f810cb00 0x7fa3f8113eb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.563+0000 7fa3f759e640 1 --2- 172.21.15.67:0/3947477176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3f810a650 0x7fa3f8116bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:50.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.563+0000 7fa3f759e640 1 -- 172.21.15.67:0/3947477176 >> 172.21.15.67:0/3947477176 conn(0x7fa3f806ee90 msgr2=0x7fa3f8073c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:41:50.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.564+0000 7fa3f759e640 1 -- 172.21.15.67:0/3947477176 shutdown_connections 2024-09-21T06:41:50.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:50.564+0000 7fa3f759e640 1 -- 172.21.15.67:0/3947477176 wait complete. 2024-09-21T06:41:51.387 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '589bb4a6', quiesce took 6.0 seconds. Will release after: 4.0 2024-09-21T06:41:51.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:51 smithi203 ceph-mon[33735]: pgmap v518: 129 pgs: 129 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 43 MiB/s wr, 681 op/s 2024-09-21T06:41:51.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:51 smithi067 ceph-mon[31232]: pgmap v518: 129 pgs: 129 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 43 MiB/s wr, 681 op/s 2024-09-21T06:41:51.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:51 smithi100 ceph-mon[33401]: pgmap v518: 129 pgs: 129 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 43 MiB/s wr, 681 op/s 2024-09-21T06:41:53.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:53 smithi203 ceph-mon[33735]: pgmap v519: 129 pgs: 129 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 52 MiB/s wr, 830 op/s 2024-09-21T06:41:53.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:53 smithi067 ceph-mon[31232]: pgmap v519: 129 pgs: 129 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 52 MiB/s wr, 830 op/s 2024-09-21T06:41:53.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:53 smithi100 ceph-mon[33401]: pgmap v519: 129 pgs: 129 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 52 MiB/s wr, 830 op/s 2024-09-21T06:41:55.388 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:41:55.665 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:41:55.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:55 smithi203 ceph-mon[33735]: pgmap v520: 129 pgs: 129 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 39 MiB/s wr, 629 op/s 2024-09-21T06:41:55.786 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:55 smithi067 ceph-mon[31232]: pgmap v520: 129 pgs: 129 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 39 MiB/s wr, 629 op/s 2024-09-21T06:41:55.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:55 smithi100 ceph-mon[33401]: pgmap v520: 129 pgs: 129 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 39 MiB/s wr, 629 op/s 2024-09-21T06:41:56.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.250+0000 7f1fb7bc0640 1 -- 172.21.15.67:0/3393709930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb0109df0 msgr2=0x7f1fb0110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:41:56.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.250+0000 7f1fb7bc0640 1 --2- 172.21.15.67:0/3393709930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb0109df0 0x7f1fb0110620 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8009900 tx=0x7f1fa802a410 comp rx=0 tx=0).stop 2024-09-21T06:41:56.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.250+0000 7f1fb7bc0640 1 -- 172.21.15.67:0/3393709930 shutdown_connections 2024-09-21T06:41:56.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.250+0000 7f1fb7bc0640 1 --2- 172.21.15.67:0/3393709930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fb01094a0 0x7f1fb01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:56.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.250+0000 7f1fb7bc0640 1 --2- 172.21.15.67:0/3393709930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb0109df0 0x7f1fb0110620 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:56.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.250+0000 7f1fb7bc0640 1 --2- 172.21.15.67:0/3393709930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fb0103b60 0x7f1fb0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:56.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.251+0000 7f1fb7bc0640 1 -- 172.21.15.67:0/3393709930 >> 172.21.15.67:0/3393709930 conn(0x7f1fb007b850 msgr2=0x7f1fb007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:41:56.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.251+0000 7f1fb7bc0640 1 -- 172.21.15.67:0/3393709930 shutdown_connections 2024-09-21T06:41:56.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.251+0000 7f1fb7bc0640 1 -- 172.21.15.67:0/3393709930 wait complete. 2024-09-21T06:41:56.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.251+0000 7f1fb7bc0640 1 Processor -- start 2024-09-21T06:41:56.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.252+0000 7f1fb7bc0640 1 -- start start 2024-09-21T06:41:56.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.252+0000 7f1fb7bc0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fb0103b60 0x7f1fb019a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:56.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.252+0000 7f1fb7bc0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb01094a0 0x7f1fb019a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:56.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.252+0000 7f1fb7bc0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fb0109df0 0x7f1fb019fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:56.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.252+0000 7f1fb7bc0640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb0113960 con 0x7f1fb01094a0 2024-09-21T06:41:56.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.252+0000 7f1fb7bc0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb01137e0 con 0x7f1fb0103b60 2024-09-21T06:41:56.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.252+0000 7f1fb7bc0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb0113ae0 con 0x7f1fb0109df0 2024-09-21T06:41:56.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.252+0000 7f1fb515b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb01094a0 0x7f1fb019a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:56.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.252+0000 7f1fb515b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb01094a0 0x7f1fb019a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53356/0 (socket says 172.21.15.67:53356) 2024-09-21T06:41:56.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.252+0000 7f1fb515b640 1 -- 172.21.15.67:0/2301484794 learned_addr learned my addr 172.21.15.67:0/2301484794 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:41:56.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.252+0000 7f1fb595c640 1 --2- 172.21.15.67:0/2301484794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fb0103b60 0x7f1fb019a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:56.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.252+0000 7f1fb615d640 1 --2- 172.21.15.67:0/2301484794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fb0109df0 0x7f1fb019fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:56.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.252+0000 7f1fb515b640 1 -- 172.21.15.67:0/2301484794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fb0109df0 msgr2=0x7f1fb019fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:41:56.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.252+0000 7f1fb515b640 1 --2- 172.21.15.67:0/2301484794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fb0109df0 0x7f1fb019fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:56.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.252+0000 7f1fb515b640 1 -- 172.21.15.67:0/2301484794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fb0103b60 msgr2=0x7f1fb019a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:41:56.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.252+0000 7f1fb515b640 1 --2- 172.21.15.67:0/2301484794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fb0103b60 0x7f1fb019a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:56.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.252+0000 7f1fb515b640 1 -- 172.21.15.67:0/2301484794 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb01a02a0 con 0x7f1fb01094a0 2024-09-21T06:41:56.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.252+0000 7f1fb595c640 1 --2- 172.21.15.67:0/2301484794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fb0103b60 0x7f1fb019a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:41:56.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.253+0000 7f1fb515b640 1 --2- 172.21.15.67:0/2301484794 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb01094a0 0x7f1fb019a8d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1fa400ebc0 tx=0x7f1fa400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:41:56.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.253+0000 7f1fa2ffd640 1 -- 172.21.15.67:0/2301484794 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4010630 con 0x7f1fb01094a0 2024-09-21T06:41:56.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.253+0000 7f1fa2ffd640 1 -- 172.21.15.67:0/2301484794 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1fa4010c60 con 0x7f1fb01094a0 2024-09-21T06:41:56.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.253+0000 7f1fb7bc0640 1 -- 172.21.15.67:0/2301484794 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fb01a0590 con 0x7f1fb01094a0 2024-09-21T06:41:56.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.253+0000 7f1fa2ffd640 1 -- 172.21.15.67:0/2301484794 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4017ad0 con 0x7f1fb01094a0 2024-09-21T06:41:56.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.253+0000 7f1fb7bc0640 1 -- 172.21.15.67:0/2301484794 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fb01a6cf0 con 0x7f1fb01094a0 2024-09-21T06:41:56.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.254+0000 7f1fb7bc0640 1 -- 172.21.15.67:0/2301484794 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fb0103f70 con 0x7f1fb01094a0 2024-09-21T06:41:56.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.256+0000 7f1fa2ffd640 1 -- 172.21.15.67:0/2301484794 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f1fa4017c70 con 0x7f1fb01094a0 2024-09-21T06:41:56.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.256+0000 7f1fa2ffd640 1 --2- 172.21.15.67:0/2301484794 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f1f9c0b6510 0x7f1f9c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:56.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.257+0000 7f1fb595c640 1 --2- 172.21.15.67:0/2301484794 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f1f9c0b6510 0x7f1f9c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:56.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.257+0000 7f1fa2ffd640 1 -- 172.21.15.67:0/2301484794 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f1fa40e8200 con 0x7f1fb01094a0 2024-09-21T06:41:56.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.257+0000 7f1fb595c640 1 --2- 172.21.15.67:0/2301484794 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f1f9c0b6510 0x7f1f9c0b8a20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1fac0058c0 tx=0x7f1fac005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:41:56.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.259+0000 7f1fa2ffd640 1 -- 172.21.15.67:0/2301484794 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1fa4056710 con 0x7f1fb01094a0 2024-09-21T06:41:56.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.486+0000 7f1fb7bc0640 1 -- 172.21.15.67:0/2301484794 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1fb019b460 con 0x7f1fb01094a0 2024-09-21T06:41:56.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.487+0000 7f1fa2ffd640 1 -- 172.21.15.67:0/2301484794 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7f1fb019b460 con 0x7f1fb01094a0 2024-09-21T06:41:56.489 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:41:56.489 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:41:56.489 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:41:56.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.489+0000 7f1fb7bc0640 1 -- 172.21.15.67:0/2301484794 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f1f9c0b6510 msgr2=0x7f1f9c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:41:56.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.489+0000 7f1fb7bc0640 1 --2- 172.21.15.67:0/2301484794 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f1f9c0b6510 0x7f1f9c0b8a20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1fac0058c0 tx=0x7f1fac005850 comp rx=0 tx=0).stop 2024-09-21T06:41:56.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.489+0000 7f1fb7bc0640 1 -- 172.21.15.67:0/2301484794 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb01094a0 msgr2=0x7f1fb019a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:41:56.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.489+0000 7f1fb7bc0640 1 --2- 172.21.15.67:0/2301484794 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb01094a0 0x7f1fb019a8d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1fa400ebc0 tx=0x7f1fa400c6a0 comp rx=0 tx=0).stop 2024-09-21T06:41:56.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.489+0000 7f1fb7bc0640 1 -- 172.21.15.67:0/2301484794 shutdown_connections 2024-09-21T06:41:56.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.489+0000 7f1fb7bc0640 1 --2- 172.21.15.67:0/2301484794 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f1f9c0b6510 0x7f1f9c0b8a20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:56.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.489+0000 7f1fb7bc0640 1 --2- 172.21.15.67:0/2301484794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fb0109df0 0x7f1fb019fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:56.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.489+0000 7f1fb7bc0640 1 --2- 172.21.15.67:0/2301484794 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb01094a0 0x7f1fb019a8d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:56.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.489+0000 7f1fb7bc0640 1 --2- 172.21.15.67:0/2301484794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fb0103b60 0x7f1fb019a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:56.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.489+0000 7f1fb7bc0640 1 -- 172.21.15.67:0/2301484794 >> 172.21.15.67:0/2301484794 conn(0x7f1fb007b850 msgr2=0x7f1fb01074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:41:56.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.490+0000 7f1fb7bc0640 1 -- 172.21.15.67:0/2301484794 shutdown_connections 2024-09-21T06:41:56.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:56.490+0000 7f1fb7bc0640 1 -- 172.21.15.67:0/2301484794 wait complete. 2024-09-21T06:41:57.400 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db --set-id 589bb4a6 --release --await' 2024-09-21T06:41:57.400 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24671 quiesce db --set-id 589bb4a6 --release --await 2024-09-21T06:41:57.663 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:41:57.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:57 smithi203 ceph-mon[33735]: pgmap v521: 129 pgs: 129 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 34 MiB/s wr, 549 op/s 2024-09-21T06:41:57.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:57 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2301484794' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:41:57.784 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:57 smithi067 ceph-mon[31232]: pgmap v521: 129 pgs: 129 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 34 MiB/s wr, 549 op/s 2024-09-21T06:41:57.784 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:57 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2301484794' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:41:57.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:57 smithi100 ceph-mon[33401]: pgmap v521: 129 pgs: 129 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 34 MiB/s wr, 549 op/s 2024-09-21T06:41:57.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:57 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2301484794' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:41:58.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.318+0000 7f21a740b640 1 -- 172.21.15.67:0/4264364106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21a01094d0 msgr2=0x7f21a01098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:41:58.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.318+0000 7f21a740b640 1 --2- 172.21.15.67:0/4264364106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21a01094d0 0x7f21a01098e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2194009900 tx=0x7f219402a410 comp rx=0 tx=0).stop 2024-09-21T06:41:58.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.318+0000 7f21a740b640 1 -- 172.21.15.67:0/4264364106 shutdown_connections 2024-09-21T06:41:58.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.318+0000 7f21a740b640 1 --2- 172.21.15.67:0/4264364106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21a0109e20 0x7f21a0110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.318+0000 7f21a740b640 1 --2- 172.21.15.67:0/4264364106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21a01094d0 0x7f21a01098e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.318+0000 7f21a740b640 1 --2- 172.21.15.67:0/4264364106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21a0103b90 0x7f21a0103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.318+0000 7f21a740b640 1 -- 172.21.15.67:0/4264364106 >> 172.21.15.67:0/4264364106 conn(0x7f21a00ff600 msgr2=0x7f21a0101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:41:58.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.318+0000 7f21a740b640 1 -- 172.21.15.67:0/4264364106 shutdown_connections 2024-09-21T06:41:58.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.318+0000 7f21a740b640 1 -- 172.21.15.67:0/4264364106 wait complete. 2024-09-21T06:41:58.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.319+0000 7f21a740b640 1 Processor -- start 2024-09-21T06:41:58.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.319+0000 7f21a740b640 1 -- start start 2024-09-21T06:41:58.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.319+0000 7f21a740b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21a0103b90 0x7f21a019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:58.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.319+0000 7f21a740b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21a01094d0 0x7f21a019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:58.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.319+0000 7f21a740b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21a0109e20 0x7f21a019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:58.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.319+0000 7f21a740b640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f21a01139b0 con 0x7f21a0109e20 2024-09-21T06:41:58.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.319+0000 7f21a740b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f21a0113830 con 0x7f21a0103b90 2024-09-21T06:41:58.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.319+0000 7f21a740b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f21a0113b30 con 0x7f21a01094d0 2024-09-21T06:41:58.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.319+0000 7f21a59a8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21a0109e20 0x7f21a019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:58.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.319+0000 7f21a59a8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21a0109e20 0x7f21a019fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39382/0 (socket says 172.21.15.67:39382) 2024-09-21T06:41:58.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.319+0000 7f21a59a8640 1 -- 172.21.15.67:0/2585142888 learned_addr learned my addr 172.21.15.67:0/2585142888 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:41:58.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.319+0000 7f21a51a7640 1 --2- 172.21.15.67:0/2585142888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21a0103b90 0x7f21a019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:58.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.320+0000 7f21a49a6640 1 --2- 172.21.15.67:0/2585142888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21a01094d0 0x7f21a019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:58.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.320+0000 7f21a59a8640 1 -- 172.21.15.67:0/2585142888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21a01094d0 msgr2=0x7f21a019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:41:58.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.320+0000 7f21a59a8640 1 --2- 172.21.15.67:0/2585142888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21a01094d0 0x7f21a019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.320+0000 7f21a59a8640 1 -- 172.21.15.67:0/2585142888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21a0103b90 msgr2=0x7f21a019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:41:58.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.320+0000 7f21a59a8640 1 --2- 172.21.15.67:0/2585142888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21a0103b90 0x7f21a019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.320+0000 7f21a59a8640 1 -- 172.21.15.67:0/2585142888 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a01a03f0 con 0x7f21a0109e20 2024-09-21T06:41:58.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.320+0000 7f21a59a8640 1 --2- 172.21.15.67:0/2585142888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21a0109e20 0x7f21a019fcc0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f219000b970 tx=0x7f219000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:41:58.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.320+0000 7f219e7fc640 1 -- 172.21.15.67:0/2585142888 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2190019070 con 0x7f21a0109e20 2024-09-21T06:41:58.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.320+0000 7f21a740b640 1 -- 172.21.15.67:0/2585142888 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21a01a06e0 con 0x7f21a0109e20 2024-09-21T06:41:58.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.320+0000 7f219e7fc640 1 -- 172.21.15.67:0/2585142888 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2190014440 con 0x7f21a0109e20 2024-09-21T06:41:58.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.320+0000 7f219e7fc640 1 -- 172.21.15.67:0/2585142888 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21900075d0 con 0x7f21a0109e20 2024-09-21T06:41:58.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.320+0000 7f21a740b640 1 -- 172.21.15.67:0/2585142888 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21a01a6e40 con 0x7f21a0109e20 2024-09-21T06:41:58.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.324+0000 7f21ad97f740 1 Processor -- start 2024-09-21T06:41:58.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.324+0000 7f219e7fc640 1 -- 172.21.15.67:0/2585142888 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f219001c070 con 0x7f21a0109e20 2024-09-21T06:41:58.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.324+0000 7f21ad97f740 1 -- start start 2024-09-21T06:41:58.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.324+0000 7f21ad97f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ccdf961520 0x55ccdfd07510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:58.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.324+0000 7f21ad97f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccdf938660 0x55ccdfd07a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:58.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.324+0000 7f21ad97f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccdf940840 0x55ccdfd07f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:58.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.325+0000 7f21ad97f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ccdfbf2dc0 con 0x55ccdf961520 2024-09-21T06:41:58.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.325+0000 7f21ad97f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ccdfc7f1f0 con 0x55ccdf938660 2024-09-21T06:41:58.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.325+0000 7f21ad97f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ccdfc6e620 con 0x55ccdf940840 2024-09-21T06:41:58.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.325+0000 7f21a49a6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ccdf961520 0x55ccdfd07510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:58.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.325+0000 7f21a49a6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ccdf961520 0x55ccdfd07510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39386/0 (socket says 172.21.15.67:39386) 2024-09-21T06:41:58.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.325+0000 7f21a49a6640 1 -- 172.21.15.67:0/2487071457 learned_addr learned my addr 172.21.15.67:0/2487071457 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:41:58.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.325+0000 7f21a51a7640 1 --2- 172.21.15.67:0/2487071457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccdf938660 0x55ccdfd07a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:58.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.325+0000 7f21a49a6640 1 --2- 172.21.15.67:0/2487071457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccdf940840 0x55ccdfd07f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:58.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.325+0000 7f21a49a6640 1 -- 172.21.15.67:0/2487071457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccdf940840 msgr2=0x55ccdfd07f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:41:58.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.325+0000 7f21a49a6640 1 --2- 172.21.15.67:0/2487071457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccdf940840 0x55ccdfd07f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.325+0000 7f21a49a6640 1 -- 172.21.15.67:0/2487071457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccdf938660 msgr2=0x55ccdfd07a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:41:58.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.325+0000 7f21a49a6640 1 --2- 172.21.15.67:0/2487071457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccdf938660 0x55ccdfd07a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.325+0000 7f219e7fc640 1 --2- 172.21.15.67:0/2585142888 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f21880b6590 0x7f21880b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:58.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.325+0000 7f21a49a6640 1 -- 172.21.15.67:0/2487071457 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccdfc36fd0 con 0x55ccdf961520 2024-09-21T06:41:58.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.325+0000 7f21a51a7640 1 --2- 172.21.15.67:0/2487071457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccdf938660 0x55ccdfd07a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:41:58.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.325+0000 7f219e7fc640 1 -- 172.21.15.67:0/2585142888 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f21900e6ac0 con 0x7f21a0109e20 2024-09-21T06:41:58.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.325+0000 7f21a49a6640 1 --2- 172.21.15.67:0/2487071457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ccdf961520 0x55ccdfd07510 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2194000c00 tx=0x7f2194009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ece1a361f2ef5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:41:58.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.325+0000 7f21a49a6640 1 --2- 172.21.15.67:0/2585142888 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f21880b6590 0x7f21880b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:58.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.326+0000 7f21877fe640 1 -- 172.21.15.67:0/2487071457 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2194047030 con 0x55ccdf961520 2024-09-21T06:41:58.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.326+0000 7f21877fe640 1 -- 172.21.15.67:0/2487071457 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21940044d0 con 0x55ccdf961520 2024-09-21T06:41:58.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.326+0000 7f21877fe640 1 -- 172.21.15.67:0/2487071457 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2194035070 con 0x55ccdf961520 2024-09-21T06:41:58.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.327+0000 7f21ad97f740 1 -- 172.21.15.67:0/2487071457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ccdf961520 msgr2=0x55ccdfd07510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:41:58.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.327+0000 7f21ad97f740 1 --2- 172.21.15.67:0/2487071457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ccdf961520 0x55ccdfd07510 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2194000c00 tx=0x7f2194009660 comp rx=0 tx=0).stop 2024-09-21T06:41:58.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.327+0000 7f21a49a6640 1 --2- 172.21.15.67:0/2585142888 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f21880b6590 0x7f21880b8aa0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2194031fd0 tx=0x7f219404c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:41:58.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.327+0000 7f21ad97f740 1 -- 172.21.15.67:0/2487071457 shutdown_connections 2024-09-21T06:41:58.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.327+0000 7f21ad97f740 1 --2- 172.21.15.67:0/2487071457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccdf938660 0x55ccdfd07a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.327+0000 7f21ad97f740 1 --2- 172.21.15.67:0/2487071457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccdf940840 0x55ccdfd07f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.327+0000 7f21ad97f740 1 --2- 172.21.15.67:0/2487071457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ccdf961520 0x55ccdfd07510 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.327+0000 7f21ad97f740 1 -- 172.21.15.67:0/2487071457 >> 172.21.15.67:0/2487071457 conn(0x55ccdf955910 msgr2=0x55ccdfd05400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:41:58.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.327+0000 7f21ad97f740 1 -- 172.21.15.67:0/2487071457 shutdown_connections 2024-09-21T06:41:58.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.327+0000 7f21ad97f740 1 -- 172.21.15.67:0/2487071457 wait complete. 2024-09-21T06:41:58.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.327+0000 7f21ad97f740 1 -- start start 2024-09-21T06:41:58.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.327+0000 7f21ad97f740 1 Processor -- start 2024-09-21T06:41:58.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.328+0000 7f21ad97f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccdf938660 0x55ccdfd16f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:58.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.328+0000 7f21ad97f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccdf940840 0x55ccdfd17450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:58.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.328+0000 7f21ad97f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ccdf961520 0x55ccdfd17990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:58.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.328+0000 7f21ad97f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ccdfc09300 con 0x55ccdf961520 2024-09-21T06:41:58.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.328+0000 7f21ad97f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ccdfc72430 con 0x55ccdf940840 2024-09-21T06:41:58.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.328+0000 7f21ad97f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ccdfacfbb0 con 0x55ccdf938660 2024-09-21T06:41:58.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.328+0000 7f21ad97f740 10 client.0 _ll_register_callbacks cb 0x55ccdfc23120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:41:58.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.328+0000 7f21a59a8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ccdf961520 0x55ccdfd17990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:58.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.328+0000 7f21a59a8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ccdf961520 0x55ccdfd17990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39402/0 (socket says 172.21.15.67:39402) 2024-09-21T06:41:58.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.328+0000 7f21a49a6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccdf940840 0x55ccdfd17450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:58.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.328+0000 7f21a59a8640 1 -- 172.21.15.67:0/594632185 learned_addr learned my addr 172.21.15.67:0/594632185 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:41:58.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.328+0000 7f21a51a7640 1 --2- 172.21.15.67:0/594632185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccdf938660 0x55ccdfd16f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:58.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.328+0000 7f21a59a8640 1 -- 172.21.15.67:0/594632185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccdf938660 msgr2=0x55ccdfd16f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:41:58.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.328+0000 7f21a59a8640 1 --2- 172.21.15.67:0/594632185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccdf938660 0x55ccdfd16f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.328+0000 7f21a59a8640 1 -- 172.21.15.67:0/594632185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccdf940840 msgr2=0x55ccdfd17450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:41:58.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.328+0000 7f21a59a8640 1 --2- 172.21.15.67:0/594632185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccdf940840 0x55ccdfd17450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.328+0000 7f21a59a8640 1 -- 172.21.15.67:0/594632185 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccdfbf8780 con 0x55ccdf961520 2024-09-21T06:41:58.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.328+0000 7f21a49a6640 1 --2- 172.21.15.67:0/594632185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccdf940840 0x55ccdfd17450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:41:58.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.329+0000 7f21a59a8640 1 --2- 172.21.15.67:0/594632185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ccdf961520 0x55ccdfd17990 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f21900ac630 tx=0x7f21900ac1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d905aaf6a247265 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:41:58.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.329+0000 7f2184ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:41:58.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.329+0000 7f2184ff9640 1 -- 172.21.15.67:0/594632185 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21900addb0 con 0x55ccdf961520 2024-09-21T06:41:58.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.329+0000 7f2184ff9640 1 -- 172.21.15.67:0/594632185 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f219000cb40 con 0x55ccdf961520 2024-09-21T06:41:58.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.329+0000 7f21ad97f740 1 -- 172.21.15.67:0/594632185 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21940044d0 con 0x55ccdf961520 2024-09-21T06:41:58.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.329+0000 7f2184ff9640 1 -- 172.21.15.67:0/594632185 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2190097070 con 0x55ccdf961520 2024-09-21T06:41:58.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.329+0000 7f21ad97f740 10 client.15654 fetch_fsmap learned FSMap version 21 2024-09-21T06:41:58.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.329+0000 7f21ad97f740 1 -- 172.21.15.67:0/594632185 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ccdf9ccb00 con 0x55ccdf961520 2024-09-21T06:41:58.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.329+0000 7f2184ff9640 1 -- 172.21.15.67:0/594632185 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21940044d0 con 0x55ccdf961520 2024-09-21T06:41:58.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.329+0000 7f2184ff9640 1 -- 172.21.15.67:0/594632185 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f21900a8280 con 0x55ccdf961520 2024-09-21T06:41:58.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.329+0000 7f21ad97f740 10 client.15654 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:41:58.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.329+0000 7f21ad97f740 10 client.15654 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:41:58.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.329+0000 7f21ad97f740 20 client.15654 populate_metadata read hostname 'smithi067' 2024-09-21T06:41:58.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.329+0000 7f21ad97f740 1 --2- 172.21.15.67:0/594632185 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55ccdf91fea0 0x55ccdfd29210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:58.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.330+0000 7f21ad97f740 4 client.15654 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:41:58.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.330+0000 7f21ad97f740 1 -- 172.21.15.67:0/594632185 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccdfc81140 con 0x55ccdf91fea0 2024-09-21T06:41:58.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.330+0000 7f21a51a7640 1 --2- 172.21.15.67:0/594632185 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55ccdf91fea0 0x55ccdfd29210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:58.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.330+0000 7f2184ff9640 10 client.15654 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:41:58.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.330+0000 7f21a51a7640 1 --2- 172.21.15.67:0/594632185 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55ccdf91fea0 0x55ccdfd29210 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff9b8f46fd4c97d9 server_cookie=f19b5cf7577f5002 in_seq=0 out_seq=0 2024-09-21T06:41:58.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.331+0000 7f2184ff9640 1 -- 172.21.15.67:0/594632185 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ccdfc81140 con 0x55ccdf91fea0 2024-09-21T06:41:58.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.331+0000 7f2184ff9640 10 client.15654 handle_command_reply: tid=0 2024-09-21T06:41:58.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.331+0000 7f21ad97f740 1 client.15654 shutdown 2024-09-21T06:41:58.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.331+0000 7f21ad97f740 1 -- 172.21.15.67:0/594632185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ccdf961520 msgr2=0x55ccdfd17990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:41:58.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.331+0000 7f21ad97f740 1 --2- 172.21.15.67:0/594632185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ccdf961520 0x55ccdfd17990 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f21900ac630 tx=0x7f21900ac1d0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.331+0000 7f21ad97f740 1 -- 172.21.15.67:0/594632185 shutdown_connections 2024-09-21T06:41:58.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.331+0000 7f21ad97f740 1 --2- 172.21.15.67:0/594632185 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55ccdf91fea0 0x55ccdfd29210 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.331+0000 7f2184ff9640 0 client.15654 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:41:58.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.331+0000 7f21ad97f740 1 --2- 172.21.15.67:0/594632185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccdf940840 0x55ccdfd17450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.331+0000 7f21ad97f740 1 --2- 172.21.15.67:0/594632185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ccdf961520 0x55ccdfd17990 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.331+0000 7f21ad97f740 1 --2- 172.21.15.67:0/594632185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccdf938660 0x55ccdfd16f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.331+0000 7f21ad97f740 1 -- 172.21.15.67:0/594632185 >> 172.21.15.67:0/594632185 conn(0x55ccdf955910 msgr2=0x55ccdfd03930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:41:58.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.331+0000 7f21ad97f740 1 -- 172.21.15.67:0/594632185 shutdown_connections 2024-09-21T06:41:58.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.332+0000 7f21ad97f740 1 -- 172.21.15.67:0/594632185 wait complete. 2024-09-21T06:41:58.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.332+0000 7f21ad97f740 20 client.15654 trim_cache size 0 max 16384 2024-09-21T06:41:58.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.349+0000 7f21ad97f740 1 Processor -- start 2024-09-21T06:41:58.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.349+0000 7f21ad97f740 1 -- start start 2024-09-21T06:41:58.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.349+0000 7f21ad97f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccdf91fea0 0x55ccdfd18b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:58.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.349+0000 7f21ad97f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccdf938660 0x55ccdfd2e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:58.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.349+0000 7f21ad97f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ccdf940840 0x55ccdfd2eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:58.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.349+0000 7f21ad97f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f21a01ac8a0 con 0x55ccdf940840 2024-09-21T06:41:58.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.349+0000 7f21ad97f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f21a01af120 con 0x55ccdf938660 2024-09-21T06:41:58.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.349+0000 7f21ad97f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f21a01acf10 con 0x55ccdf91fea0 2024-09-21T06:41:58.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.349+0000 7f21a59a8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ccdf940840 0x55ccdfd2eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:58.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.349+0000 7f21a59a8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ccdf940840 0x55ccdfd2eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39406/0 (socket says 172.21.15.67:39406) 2024-09-21T06:41:58.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.349+0000 7f21a59a8640 1 -- 172.21.15.67:0/334979739 learned_addr learned my addr 172.21.15.67:0/334979739 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:41:58.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.349+0000 7f21a49a6640 1 --2- 172.21.15.67:0/334979739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccdf938660 0x55ccdfd2e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:58.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.349+0000 7f21a51a7640 1 --2- 172.21.15.67:0/334979739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccdf91fea0 0x55ccdfd18b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:58.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.350+0000 7f21a59a8640 1 -- 172.21.15.67:0/334979739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccdf91fea0 msgr2=0x55ccdfd18b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:41:58.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.350+0000 7f21a59a8640 1 --2- 172.21.15.67:0/334979739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccdf91fea0 0x55ccdfd18b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.350+0000 7f21a59a8640 1 -- 172.21.15.67:0/334979739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccdf938660 msgr2=0x55ccdfd2e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:41:58.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.350+0000 7f21a59a8640 1 --2- 172.21.15.67:0/334979739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccdf938660 0x55ccdfd2e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.350+0000 7f21a59a8640 1 -- 172.21.15.67:0/334979739 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccdfa43960 con 0x55ccdf940840 2024-09-21T06:41:58.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.350+0000 7f21a59a8640 1 --2- 172.21.15.67:0/334979739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ccdf940840 0x55ccdfd2eef0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f21900e6de0 tx=0x7f2190015fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb54dbc9333a284 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:41:58.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.350+0000 7f2186ffd640 1 -- 172.21.15.67:0/334979739 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21900ac280 con 0x55ccdf940840 2024-09-21T06:41:58.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.350+0000 7f2186ffd640 1 -- 172.21.15.67:0/334979739 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21900b62a0 con 0x55ccdf940840 2024-09-21T06:41:58.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.350+0000 7f2186ffd640 1 -- 172.21.15.67:0/334979739 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21900a87d0 con 0x55ccdf940840 2024-09-21T06:41:58.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.351+0000 7f21ad97f740 1 -- 172.21.15.67:0/334979739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ccdf940840 msgr2=0x55ccdfd2eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:41:58.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.351+0000 7f21ad97f740 1 --2- 172.21.15.67:0/334979739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ccdf940840 0x55ccdfd2eef0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f21900e6de0 tx=0x7f2190015fc0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.351+0000 7f21ad97f740 1 -- 172.21.15.67:0/334979739 shutdown_connections 2024-09-21T06:41:58.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.351+0000 7f21ad97f740 1 --2- 172.21.15.67:0/334979739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccdf938660 0x55ccdfd2e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.351+0000 7f21ad97f740 1 --2- 172.21.15.67:0/334979739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ccdf940840 0x55ccdfd2eef0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.351+0000 7f21ad97f740 1 --2- 172.21.15.67:0/334979739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccdf91fea0 0x55ccdfd18b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.351+0000 7f21ad97f740 1 -- 172.21.15.67:0/334979739 >> 172.21.15.67:0/334979739 conn(0x55ccdf955910 msgr2=0x55ccdfd18430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:41:58.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.351+0000 7f21ad97f740 1 -- 172.21.15.67:0/334979739 shutdown_connections 2024-09-21T06:41:58.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.351+0000 7f21ad97f740 1 -- 172.21.15.67:0/334979739 wait complete. 2024-09-21T06:41:58.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.351+0000 7f21ad97f740 1 -- start start 2024-09-21T06:41:58.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.351+0000 7f21ad97f740 1 Processor -- start 2024-09-21T06:41:58.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.352+0000 7f21ad97f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccdf91fea0 0x55ccdfd1f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:58.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.352+0000 7f21ad97f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccdf938660 0x55ccdfd1a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:58.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.352+0000 7f21ad97f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ccdf940840 0x55ccdfd1a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:58.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.352+0000 7f21ad97f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f21a01ad5e0 con 0x55ccdf940840 2024-09-21T06:41:58.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.352+0000 7f21ad97f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f21a01aea50 con 0x55ccdf938660 2024-09-21T06:41:58.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.352+0000 7f21ad97f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f21a01adcb0 con 0x55ccdf91fea0 2024-09-21T06:41:58.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.352+0000 7f21ad97f740 10 client.0 _ll_register_callbacks cb 0x7f21900a9000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:41:58.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.352+0000 7f21a59a8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ccdf940840 0x55ccdfd1a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:58.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.352+0000 7f21a59a8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ccdf940840 0x55ccdfd1a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39410/0 (socket says 172.21.15.67:39410) 2024-09-21T06:41:58.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.352+0000 7f21a59a8640 1 -- 172.21.15.67:0/873400540 learned_addr learned my addr 172.21.15.67:0/873400540 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:41:58.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.352+0000 7f21a51a7640 1 --2- 172.21.15.67:0/873400540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccdf91fea0 0x55ccdfd1f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:58.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.352+0000 7f21a49a6640 1 --2- 172.21.15.67:0/873400540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccdf938660 0x55ccdfd1a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:58.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.352+0000 7f21a59a8640 1 -- 172.21.15.67:0/873400540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccdf91fea0 msgr2=0x55ccdfd1f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:41:58.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.352+0000 7f21a59a8640 1 --2- 172.21.15.67:0/873400540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccdf91fea0 0x55ccdfd1f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.352+0000 7f21a59a8640 1 -- 172.21.15.67:0/873400540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccdf938660 msgr2=0x55ccdfd1a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:41:58.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.352+0000 7f21a59a8640 1 --2- 172.21.15.67:0/873400540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccdf938660 0x55ccdfd1a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.352+0000 7f21a59a8640 1 -- 172.21.15.67:0/873400540 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccdfd1ae80 con 0x55ccdf940840 2024-09-21T06:41:58.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.352+0000 7f21a59a8640 1 --2- 172.21.15.67:0/873400540 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ccdf940840 0x55ccdfd1a6f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2190004a80 tx=0x7f2190004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cbb6e4d97a0e035 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:41:58.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.352+0000 7f2184ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:41:58.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.353+0000 7f2184ff9640 1 -- 172.21.15.67:0/873400540 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21900a87d0 con 0x55ccdf940840 2024-09-21T06:41:58.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.353+0000 7f2184ff9640 1 -- 172.21.15.67:0/873400540 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f21900adbb0 con 0x55ccdf940840 2024-09-21T06:41:58.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.353+0000 7f21ad97f740 1 -- 172.21.15.67:0/873400540 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21900b62a0 con 0x55ccdf940840 2024-09-21T06:41:58.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.353+0000 7f2184ff9640 1 -- 172.21.15.67:0/873400540 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f21900ac280 con 0x55ccdf940840 2024-09-21T06:41:58.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.353+0000 7f21ad97f740 10 client.15660 fetch_fsmap learned FSMap version 21 2024-09-21T06:41:58.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.353+0000 7f21ad97f740 1 -- 172.21.15.67:0/873400540 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ccdfd94c00 con 0x55ccdf940840 2024-09-21T06:41:58.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.353+0000 7f2184ff9640 1 -- 172.21.15.67:0/873400540 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21900b62a0 con 0x55ccdf940840 2024-09-21T06:41:58.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.353+0000 7f2184ff9640 1 -- 172.21.15.67:0/873400540 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f21900ce390 con 0x55ccdf940840 2024-09-21T06:41:58.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.353+0000 7f21ad97f740 10 client.15660 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:41:58.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.353+0000 7f21ad97f740 10 client.15660 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:41:58.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.353+0000 7f21ad97f740 20 client.15660 populate_metadata read hostname 'smithi067' 2024-09-21T06:41:58.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.353+0000 7f21ad97f740 1 --2- 172.21.15.67:0/873400540 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55ccdf961520 0x55ccdfd970f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:41:58.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.353+0000 7f21ad97f740 4 client.15660 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "589bb4a6", "release": true, "await": true}] 2024-09-21T06:41:58.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.353+0000 7f21ad97f740 1 -- 172.21.15.67:0/873400540 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "quiesce db", "set_id": "589bb4a6", "release": true, "await": true}) -- 0x55ccdfc7cbf0 con 0x55ccdf961520 2024-09-21T06:41:58.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.354+0000 7f21a51a7640 1 --2- 172.21.15.67:0/873400540 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55ccdf961520 0x55ccdfd970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:41:58.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.354+0000 7f2184ff9640 10 client.15660 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:41:58.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.354+0000 7f21a51a7640 1 --2- 172.21.15.67:0/873400540 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55ccdf961520 0x55ccdfd970f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b51d04b433d3600 server_cookie=7e960691dcbc664b in_seq=0 out_seq=0 2024-09-21T06:41:58.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.357+0000 7f2184ff9640 1 -- 172.21.15.67:0/873400540 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ccdfc7cbf0 con 0x55ccdf961520 2024-09-21T06:41:58.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.357+0000 7f2184ff9640 10 client.15660 handle_command_reply: tid=0 2024-09-21T06:41:58.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.357+0000 7f21ad97f740 1 client.15660 shutdown 2024-09-21T06:41:58.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.357+0000 7f21ad97f740 1 -- 172.21.15.67:0/873400540 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ccdf940840 msgr2=0x55ccdfd1a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:41:58.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.357+0000 7f21ad97f740 1 --2- 172.21.15.67:0/873400540 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ccdf940840 0x55ccdfd1a6f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2190004a80 tx=0x7f2190004b50 comp rx=0 tx=0).stop 2024-09-21T06:41:58.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.357+0000 7f21ad97f740 1 -- 172.21.15.67:0/873400540 shutdown_connections 2024-09-21T06:41:58.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.357+0000 7f21ad97f740 1 --2- 172.21.15.67:0/873400540 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55ccdf961520 0x55ccdfd970f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.357+0000 7f21ad97f740 1 --2- 172.21.15.67:0/873400540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccdf938660 0x55ccdfd1a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.357+0000 7f21ad97f740 1 --2- 172.21.15.67:0/873400540 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ccdf940840 0x55ccdfd1a6f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.357+0000 7f21ad97f740 1 --2- 172.21.15.67:0/873400540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccdf91fea0 0x55ccdfd1f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.357+0000 7f21ad97f740 1 -- 172.21.15.67:0/873400540 >> 172.21.15.67:0/873400540 conn(0x55ccdf955910 msgr2=0x55ccdfd16e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:41:58.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.357+0000 7f2184ff9640 0 client.15660 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:41:58.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.358+0000 7f21ad97f740 1 -- 172.21.15.67:0/873400540 shutdown_connections 2024-09-21T06:41:58.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.358+0000 7f21ad97f740 1 -- 172.21.15.67:0/873400540 wait complete. 2024-09-21T06:41:58.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.358+0000 7f21ad97f740 20 client.15660 trim_cache size 0 max 16384 2024-09-21T06:41:58.362 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-21T06:41:58.362 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 21, 2024-09-21T06:41:58.362 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-09-21T06:41:58.362 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 20, 2024-09-21T06:41:58.362 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-21T06:41:58.363 INFO:teuthology.orchestra.run.smithi067.stdout: "589bb4a6": { 2024-09-21T06:41:58.363 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 20, 2024-09-21T06:41:58.363 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-21T06:41:58.363 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:41:58.363 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-21T06:41:58.363 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:41:58.363 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:41:58.363 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-21T06:41:58.363 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 130.0, 2024-09-21T06:41:58.363 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-21T06:41:58.363 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-21T06:41:58.363 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-21T06:41:58.363 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:41:58.363 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-21T06:41:58.363 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:41:58.364 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:41:58.364 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:41:58.364 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:41:58.364 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:41:58.364 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:41:58.364 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-21T06:41:58.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.359+0000 7f21a740b640 1 -- 172.21.15.67:0/2585142888 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f21880b6590 msgr2=0x7f21880b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:41:58.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.359+0000 7f21a740b640 1 --2- 172.21.15.67:0/2585142888 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f21880b6590 0x7f21880b8aa0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2194031fd0 tx=0x7f219404c000 comp rx=0 tx=0).stop 2024-09-21T06:41:58.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.359+0000 7f21a740b640 1 -- 172.21.15.67:0/2585142888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21a0109e20 msgr2=0x7f21a019fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:41:58.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.359+0000 7f21a740b640 1 --2- 172.21.15.67:0/2585142888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21a0109e20 0x7f21a019fcc0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f219000b970 tx=0x7f219000be30 comp rx=0 tx=0).stop 2024-09-21T06:41:58.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.359+0000 7f21a740b640 1 -- 172.21.15.67:0/2585142888 shutdown_connections 2024-09-21T06:41:58.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.359+0000 7f21a740b640 1 --2- 172.21.15.67:0/2585142888 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f21880b6590 0x7f21880b8aa0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.359+0000 7f21a740b640 1 --2- 172.21.15.67:0/2585142888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21a0109e20 0x7f21a019fcc0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.359+0000 7f21a740b640 1 --2- 172.21.15.67:0/2585142888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21a01094d0 0x7f21a019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.359+0000 7f21a740b640 1 --2- 172.21.15.67:0/2585142888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21a0103b90 0x7f21a019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:41:58.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.359+0000 7f21a740b640 1 -- 172.21.15.67:0/2585142888 >> 172.21.15.67:0/2585142888 conn(0x7f21a00ff600 msgr2=0x7f21a0101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:41:58.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.359+0000 7f21a740b640 1 -- 172.21.15.67:0/2585142888 shutdown_connections 2024-09-21T06:41:58.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:41:58.359+0000 7f21a740b640 1 -- 172.21.15.67:0/2585142888 wait complete. 2024-09-21T06:41:59.183 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '589bb4a6', total seconds elapsed: 13.8 2024-09-21T06:41:59.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:41:59 smithi203 ceph-mon[33735]: pgmap v522: 129 pgs: 129 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 28 MiB/s wr, 446 op/s 2024-09-21T06:41:59.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:41:59 smithi067 ceph-mon[31232]: pgmap v522: 129 pgs: 129 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 28 MiB/s wr, 446 op/s 2024-09-21T06:41:59.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:41:59 smithi100 ceph-mon[33401]: pgmap v522: 129 pgs: 129 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 28 MiB/s wr, 446 op/s 2024-09-21T06:42:01.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:01 smithi203 ceph-mon[33735]: pgmap v523: 129 pgs: 129 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 17 MiB/s wr, 266 op/s 2024-09-21T06:42:01.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:01 smithi067 ceph-mon[31232]: pgmap v523: 129 pgs: 129 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 17 MiB/s wr, 266 op/s 2024-09-21T06:42:01.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:01 smithi100 ceph-mon[33401]: pgmap v523: 129 pgs: 129 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 17 MiB/s wr, 266 op/s 2024-09-21T06:42:03.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:03 smithi203 ceph-mon[33735]: pgmap v524: 129 pgs: 129 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 23 MiB/s wr, 364 op/s 2024-09-21T06:42:03.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:03 smithi067 ceph-mon[31232]: pgmap v524: 129 pgs: 129 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 23 MiB/s wr, 364 op/s 2024-09-21T06:42:03.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:03 smithi100 ceph-mon[33401]: pgmap v524: 129 pgs: 129 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 23 MiB/s wr, 364 op/s 2024-09-21T06:42:05.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:05 smithi203 ceph-mon[33735]: pgmap v525: 129 pgs: 129 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 16 MiB/s wr, 256 op/s 2024-09-21T06:42:05.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:05 smithi067 ceph-mon[31232]: pgmap v525: 129 pgs: 129 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 16 MiB/s wr, 256 op/s 2024-09-21T06:42:05.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:05 smithi100 ceph-mon[33401]: pgmap v525: 129 pgs: 129 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 16 MiB/s wr, 256 op/s 2024-09-21T06:42:07.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:07 smithi203 ceph-mon[33735]: pgmap v526: 129 pgs: 129 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 20 MiB/s wr, 323 op/s 2024-09-21T06:42:07.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:07 smithi067 ceph-mon[31232]: pgmap v526: 129 pgs: 129 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 20 MiB/s wr, 323 op/s 2024-09-21T06:42:07.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:07 smithi100 ceph-mon[33401]: pgmap v526: 129 pgs: 129 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 20 MiB/s wr, 323 op/s 2024-09-21T06:42:09.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:09 smithi203 ceph-mon[33735]: pgmap v527: 129 pgs: 129 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 31 MiB/s wr, 494 op/s 2024-09-21T06:42:09.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:09 smithi067 ceph-mon[31232]: pgmap v527: 129 pgs: 129 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 31 MiB/s wr, 494 op/s 2024-09-21T06:42:09.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:09 smithi100 ceph-mon[33401]: pgmap v527: 129 pgs: 129 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 31 MiB/s wr, 494 op/s 2024-09-21T06:42:11.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:11 smithi203 ceph-mon[33735]: pgmap v528: 129 pgs: 129 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 33 MiB/s wr, 522 op/s 2024-09-21T06:42:11.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:11 smithi067 ceph-mon[31232]: pgmap v528: 129 pgs: 129 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 33 MiB/s wr, 522 op/s 2024-09-21T06:42:11.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:11 smithi100 ceph-mon[33401]: pgmap v528: 129 pgs: 129 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 33 MiB/s wr, 522 op/s 2024-09-21T06:42:13.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:13 smithi203 ceph-mon[33735]: pgmap v529: 129 pgs: 129 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 47 MiB/s wr, 751 op/s 2024-09-21T06:42:13.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:13 smithi067 ceph-mon[31232]: pgmap v529: 129 pgs: 129 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 47 MiB/s wr, 751 op/s 2024-09-21T06:42:13.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:13 smithi100 ceph-mon[33401]: pgmap v529: 129 pgs: 129 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 47 MiB/s wr, 751 op/s 2024-09-21T06:42:15.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:15 smithi203 ceph-mon[33735]: pgmap v530: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 44 MiB/s wr, 698 op/s 2024-09-21T06:42:15.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:15 smithi067 ceph-mon[31232]: pgmap v530: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 44 MiB/s wr, 698 op/s 2024-09-21T06:42:15.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:15 smithi100 ceph-mon[33401]: pgmap v530: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 44 MiB/s wr, 698 op/s 2024-09-21T06:42:17.756 INFO:tasks.quiescer.fs.[cephfs]:Total duration (38.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-21T06:42:17.756 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.25 2024-09-21T06:42:17.756 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:42:17.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:17 smithi067 ceph-mon[31232]: pgmap v531: 129 pgs: 129 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 45 MiB/s wr, 726 op/s 2024-09-21T06:42:17.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:17 smithi100 ceph-mon[33401]: pgmap v531: 129 pgs: 129 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 45 MiB/s wr, 726 op/s 2024-09-21T06:42:17.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:17 smithi203 ceph-mon[33735]: pgmap v531: 129 pgs: 129 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 45 MiB/s wr, 726 op/s 2024-09-21T06:42:18.410 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:42:19.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:19 smithi067 ceph-mon[31232]: pgmap v532: 129 pgs: 129 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 50 MiB/s wr, 798 op/s 2024-09-21T06:42:19.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:19 smithi100 ceph-mon[33401]: pgmap v532: 129 pgs: 129 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 50 MiB/s wr, 798 op/s 2024-09-21T06:42:19.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:19 smithi203 ceph-mon[33735]: pgmap v532: 129 pgs: 129 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 50 MiB/s wr, 798 op/s 2024-09-21T06:42:21.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:21 smithi067 ceph-mon[31232]: pgmap v533: 129 pgs: 129 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 41 MiB/s wr, 653 op/s 2024-09-21T06:42:21.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:21 smithi100 ceph-mon[33401]: pgmap v533: 129 pgs: 129 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 41 MiB/s wr, 653 op/s 2024-09-21T06:42:21.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:21 smithi203 ceph-mon[33735]: pgmap v533: 129 pgs: 129 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 41 MiB/s wr, 653 op/s 2024-09-21T06:42:23.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:23 smithi067 ceph-mon[31232]: pgmap v534: 129 pgs: 129 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 52 MiB/s wr, 831 op/s 2024-09-21T06:42:23.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:23 smithi100 ceph-mon[33401]: pgmap v534: 129 pgs: 129 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 52 MiB/s wr, 831 op/s 2024-09-21T06:42:23.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:23 smithi203 ceph-mon[33735]: pgmap v534: 129 pgs: 129 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 52 MiB/s wr, 831 op/s 2024-09-21T06:42:24.702 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:42:24.702 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:42:24.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.302+0000 7fa1b3966640 1 -- 172.21.15.67:0/1552839022 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1ac10de10 msgr2=0x7fa1ac110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:24.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.302+0000 7fa1b3966640 1 --2- 172.21.15.67:0/1552839022 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1ac10de10 0x7fa1ac110300 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4009ca0 tx=0x7fa1a402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:42:24.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.302+0000 7fa1b3966640 1 -- 172.21.15.67:0/1552839022 shutdown_connections 2024-09-21T06:42:24.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.302+0000 7fa1b3966640 1 --2- 172.21.15.67:0/1552839022 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1ac10de10 0x7fa1ac110300 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:24.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.302+0000 7fa1b3966640 1 --2- 172.21.15.67:0/1552839022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1ac105300 0x7fa1ac10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:24.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.302+0000 7fa1b3966640 1 --2- 172.21.15.67:0/1552839022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1ac1049b0 0x7fa1ac104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:24.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.302+0000 7fa1b3966640 1 -- 172.21.15.67:0/1552839022 >> 172.21.15.67:0/1552839022 conn(0x7fa1ac079730 msgr2=0x7fa1ac079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:42:24.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.302+0000 7fa1b3966640 1 -- 172.21.15.67:0/1552839022 shutdown_connections 2024-09-21T06:42:24.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.302+0000 7fa1b3966640 1 -- 172.21.15.67:0/1552839022 wait complete. 2024-09-21T06:42:24.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.303+0000 7fa1b3966640 1 Processor -- start 2024-09-21T06:42:24.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.303+0000 7fa1b3966640 1 -- start start 2024-09-21T06:42:24.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.303+0000 7fa1b3966640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1ac1049b0 0x7fa1ac1026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:24.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.303+0000 7fa1b3966640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1ac105300 0x7fa1ac102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:24.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.303+0000 7fa1b3966640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1ac10de10 0x7fa1ac103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:24.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.303+0000 7fa1b3966640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ac113970 con 0x7fa1ac10de10 2024-09-21T06:42:24.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.303+0000 7fa1b3966640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ac1137f0 con 0x7fa1ac1049b0 2024-09-21T06:42:24.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.303+0000 7fa1b3966640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ac113af0 con 0x7fa1ac105300 2024-09-21T06:42:24.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.303+0000 7fa1b1f03640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1ac10de10 0x7fa1ac103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:24.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.303+0000 7fa1b1f03640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1ac10de10 0x7fa1ac103170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52796/0 (socket says 172.21.15.67:52796) 2024-09-21T06:42:24.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.303+0000 7fa1b1f03640 1 -- 172.21.15.67:0/2199280197 learned_addr learned my addr 172.21.15.67:0/2199280197 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:42:24.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.303+0000 7fa1b1702640 1 --2- 172.21.15.67:0/2199280197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1ac1049b0 0x7fa1ac1026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:24.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.303+0000 7fa1b1f03640 1 -- 172.21.15.67:0/2199280197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1ac105300 msgr2=0x7fa1ac102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:24.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.303+0000 7fa1b1f03640 1 --2- 172.21.15.67:0/2199280197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1ac105300 0x7fa1ac102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.303+0000 7fa1b1f03640 1 -- 172.21.15.67:0/2199280197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1ac1049b0 msgr2=0x7fa1ac1026f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.304+0000 7fa1b1f03640 1 --2- 172.21.15.67:0/2199280197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1ac1049b0 0x7fa1ac1026f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.304+0000 7fa1b1f03640 1 -- 172.21.15.67:0/2199280197 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ac1a8620 con 0x7fa1ac10de10 2024-09-21T06:42:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.304+0000 7fa1b1f03640 1 --2- 172.21.15.67:0/2199280197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1ac10de10 0x7fa1ac103170 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4005cc0 tx=0x7fa1a4032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:42:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.304+0000 7fa1a27fc640 1 -- 172.21.15.67:0/2199280197 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4032e00 con 0x7fa1ac10de10 2024-09-21T06:42:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.304+0000 7fa1a27fc640 1 -- 172.21.15.67:0/2199280197 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa1a40044d0 con 0x7fa1ac10de10 2024-09-21T06:42:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.304+0000 7fa1a27fc640 1 -- 172.21.15.67:0/2199280197 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4030b70 con 0x7fa1ac10de10 2024-09-21T06:42:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.304+0000 7fa1b3966640 1 -- 172.21.15.67:0/2199280197 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1ac1a88b0 con 0x7fa1ac10de10 2024-09-21T06:42:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.304+0000 7fa1b3966640 1 -- 172.21.15.67:0/2199280197 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1ac1a8df0 con 0x7fa1ac10de10 2024-09-21T06:42:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.305+0000 7fa1b3966640 1 -- 172.21.15.67:0/2199280197 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1ac104dc0 con 0x7fa1ac10de10 2024-09-21T06:42:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.307+0000 7fa1a27fc640 1 -- 172.21.15.67:0/2199280197 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa1a4030d10 con 0x7fa1ac10de10 2024-09-21T06:42:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.307+0000 7fa1a27fc640 1 --2- 172.21.15.67:0/2199280197 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fa1980b6590 0x7fa1980b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:24.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.308+0000 7fa1b1702640 1 --2- 172.21.15.67:0/2199280197 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fa1980b6590 0x7fa1980b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:24.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.308+0000 7fa1a27fc640 1 -- 172.21.15.67:0/2199280197 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fa1a4108c30 con 0x7fa1ac10de10 2024-09-21T06:42:24.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.308+0000 7fa1b1702640 1 --2- 172.21.15.67:0/2199280197 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fa1980b6590 0x7fa1980b8aa0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8005fd0 tx=0x7fa1a8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:42:24.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.311+0000 7fa1a27fc640 1 -- 172.21.15.67:0/2199280197 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa1a40cf090 con 0x7fa1ac10de10 2024-09-21T06:42:24.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.616+0000 7fa1b3966640 1 -- 172.21.15.67:0/2199280197 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1ac1039f0 con 0x7fa1ac10de10 2024-09-21T06:42:24.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.618+0000 7fa1a27fc640 1 -- 172.21.15.67:0/2199280197 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7fa1a40d3470 con 0x7fa1ac10de10 2024-09-21T06:42:24.706 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:42:24.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.622+0000 7fa1b3966640 1 -- 172.21.15.67:0/2199280197 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fa1980b6590 msgr2=0x7fa1980b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:24.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.622+0000 7fa1b3966640 1 --2- 172.21.15.67:0/2199280197 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fa1980b6590 0x7fa1980b8aa0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8005fd0 tx=0x7fa1a8005850 comp rx=0 tx=0).stop 2024-09-21T06:42:24.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.622+0000 7fa1b3966640 1 -- 172.21.15.67:0/2199280197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1ac10de10 msgr2=0x7fa1ac103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:24.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.622+0000 7fa1b3966640 1 --2- 172.21.15.67:0/2199280197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1ac10de10 0x7fa1ac103170 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4005cc0 tx=0x7fa1a4032a10 comp rx=0 tx=0).stop 2024-09-21T06:42:24.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.622+0000 7fa1b3966640 1 -- 172.21.15.67:0/2199280197 shutdown_connections 2024-09-21T06:42:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.622+0000 7fa1b3966640 1 --2- 172.21.15.67:0/2199280197 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fa1980b6590 0x7fa1980b8aa0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.622+0000 7fa1b3966640 1 --2- 172.21.15.67:0/2199280197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1ac10de10 0x7fa1ac103170 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.622+0000 7fa1b3966640 1 --2- 172.21.15.67:0/2199280197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1ac105300 0x7fa1ac102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.622+0000 7fa1b3966640 1 --2- 172.21.15.67:0/2199280197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1ac1049b0 0x7fa1ac1026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.622+0000 7fa1b3966640 1 -- 172.21.15.67:0/2199280197 >> 172.21.15.67:0/2199280197 conn(0x7fa1ac079730 msgr2=0x7fa1ac100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:42:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.622+0000 7fa1b3966640 1 -- 172.21.15.67:0/2199280197 shutdown_connections 2024-09-21T06:42:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:23.622+0000 7fa1b3966640 1 -- 172.21.15.67:0/2199280197 wait complete. 2024-09-21T06:42:24.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:24 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2199280197' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:42:24.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:24 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2199280197' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:42:24.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:24 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2199280197' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:42:25.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:25 smithi067 ceph-mon[31232]: pgmap v535: 129 pgs: 129 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 42 MiB/s wr, 667 op/s 2024-09-21T06:42:25.806 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:25 smithi100 ceph-mon[33401]: pgmap v535: 129 pgs: 129 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 42 MiB/s wr, 667 op/s 2024-09-21T06:42:25.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:25 smithi203 ceph-mon[33735]: pgmap v535: 129 pgs: 129 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 42 MiB/s wr, 667 op/s 2024-09-21T06:42:27.748 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db / --timeout 90 --expiration 139.25 --await-for 45.0' 2024-09-21T06:42:27.748 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24671 quiesce db / --timeout 90 --expiration 139.25 --await-for 45.0 2024-09-21T06:42:27.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:27 smithi067 ceph-mon[31232]: pgmap v536: 129 pgs: 129 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 42 MiB/s wr, 666 op/s 2024-09-21T06:42:27.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:27 smithi100 ceph-mon[33401]: pgmap v536: 129 pgs: 129 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 42 MiB/s wr, 666 op/s 2024-09-21T06:42:27.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:27 smithi203 ceph-mon[33735]: pgmap v536: 129 pgs: 129 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 42 MiB/s wr, 666 op/s 2024-09-21T06:42:29.069 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:42:29.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:29 smithi100 ceph-mon[33401]: pgmap v537: 129 pgs: 129 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 43 MiB/s wr, 685 op/s 2024-09-21T06:42:29.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:29 smithi203 ceph-mon[33735]: pgmap v537: 129 pgs: 129 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 43 MiB/s wr, 685 op/s 2024-09-21T06:42:30.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:29 smithi067 ceph-mon[31232]: pgmap v537: 129 pgs: 129 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 43 MiB/s wr, 685 op/s 2024-09-21T06:42:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.696+0000 7f84a306c640 1 -- 172.21.15.67:0/3047276098 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f849c10a190 msgr2=0x7f849c10a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.696+0000 7f84a306c640 1 --2- 172.21.15.67:0/3047276098 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f849c10a190 0x7f849c10a5a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f848c009900 tx=0x7f848c02a410 comp rx=0 tx=0).stop 2024-09-21T06:42:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.696+0000 7f84a306c640 1 -- 172.21.15.67:0/3047276098 shutdown_connections 2024-09-21T06:42:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.696+0000 7f84a306c640 1 --2- 172.21.15.67:0/3047276098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f849c10fad0 0x7f849c10fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.696+0000 7f84a306c640 1 --2- 172.21.15.67:0/3047276098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f849c110420 0x7f849c116c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.696+0000 7f84a306c640 1 --2- 172.21.15.67:0/3047276098 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f849c10a190 0x7f849c10a5a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.696+0000 7f84a306c640 1 -- 172.21.15.67:0/3047276098 >> 172.21.15.67:0/3047276098 conn(0x7f849c0781c0 msgr2=0x7f849c1081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:42:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.696+0000 7f84a306c640 1 -- 172.21.15.67:0/3047276098 shutdown_connections 2024-09-21T06:42:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.696+0000 7f84a306c640 1 -- 172.21.15.67:0/3047276098 wait complete. 2024-09-21T06:42:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.697+0000 7f84a306c640 1 Processor -- start 2024-09-21T06:42:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.697+0000 7f84a306c640 1 -- start start 2024-09-21T06:42:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.697+0000 7f84a306c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f849c10a190 0x7f849c1a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.697+0000 7f84a306c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f849c10fad0 0x7f849c1a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.697+0000 7f84a306c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f849c110420 0x7f849c1a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:31.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.697+0000 7f84a306c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f849c119ff0 con 0x7f849c10a190 2024-09-21T06:42:31.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.697+0000 7f84a306c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f849c119e70 con 0x7f849c110420 2024-09-21T06:42:31.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.697+0000 7f84a306c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f849c11a170 con 0x7f849c10fad0 2024-09-21T06:42:31.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.698+0000 7f84a0e08640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f849c10a190 0x7f849c1a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:31.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.698+0000 7f84a0e08640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f849c10a190 0x7f849c1a0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51142/0 (socket says 172.21.15.67:51142) 2024-09-21T06:42:31.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.698+0000 7f84a0e08640 1 -- 172.21.15.67:0/671908366 learned_addr learned my addr 172.21.15.67:0/671908366 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:42:31.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.698+0000 7f849bfff640 1 --2- 172.21.15.67:0/671908366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f849c10fad0 0x7f849c1a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:31.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.698+0000 7f84a1609640 1 --2- 172.21.15.67:0/671908366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f849c110420 0x7f849c1a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:31.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.698+0000 7f84a1609640 1 -- 172.21.15.67:0/671908366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f849c10fad0 msgr2=0x7f849c1a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:31.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.698+0000 7f84a1609640 1 --2- 172.21.15.67:0/671908366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f849c10fad0 0x7f849c1a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:31.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.698+0000 7f84a1609640 1 -- 172.21.15.67:0/671908366 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f849c10a190 msgr2=0x7f849c1a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:31.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.698+0000 7f84a1609640 1 --2- 172.21.15.67:0/671908366 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f849c10a190 0x7f849c1a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:31.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.698+0000 7f84a1609640 1 -- 172.21.15.67:0/671908366 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f849c1a6a30 con 0x7f849c110420 2024-09-21T06:42:31.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.698+0000 7f84a0e08640 1 --2- 172.21.15.67:0/671908366 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f849c10a190 0x7f849c1a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:42:31.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.698+0000 7f84a1609640 1 --2- 172.21.15.67:0/671908366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f849c110420 0x7f849c1a6300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f849400be20 tx=0x7f849400bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:42:31.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.699+0000 7f8499ffb640 1 -- 172.21.15.67:0/671908366 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8494019070 con 0x7f849c110420 2024-09-21T06:42:31.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.699+0000 7f8499ffb640 1 -- 172.21.15.67:0/671908366 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84940148d0 con 0x7f849c110420 2024-09-21T06:42:31.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.699+0000 7f84a306c640 1 -- 172.21.15.67:0/671908366 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f849c1a6d20 con 0x7f849c110420 2024-09-21T06:42:31.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.699+0000 7f8499ffb640 1 -- 172.21.15.67:0/671908366 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84940127e0 con 0x7f849c110420 2024-09-21T06:42:31.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.699+0000 7f84a306c640 1 -- 172.21.15.67:0/671908366 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f849c1ad480 con 0x7f849c110420 2024-09-21T06:42:31.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.701+0000 7f84a95e0740 1 Processor -- start 2024-09-21T06:42:31.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.702+0000 7f8499ffb640 1 -- 172.21.15.67:0/671908366 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f849401c020 con 0x7f849c110420 2024-09-21T06:42:31.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.703+0000 7f84a95e0740 1 -- start start 2024-09-21T06:42:31.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.703+0000 7f8499ffb640 1 --2- 172.21.15.67:0/671908366 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f84880b6510 0x7f84880b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:31.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.703+0000 7f849bfff640 1 --2- 172.21.15.67:0/671908366 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f84880b6510 0x7f84880b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:31.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.703+0000 7f8499ffb640 1 -- 172.21.15.67:0/671908366 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f84940e6fc0 con 0x7f849c110420 2024-09-21T06:42:31.101 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-21T06:42:31.101 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 21, 2024-09-21T06:42:31.101 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-09-21T06:42:31.101 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 22, 2024-09-21T06:42:31.101 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-21T06:42:31.101 INFO:teuthology.orchestra.run.smithi067.stdout: "79dfa6b2": { 2024-09-21T06:42:31.101 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 22, 2024-09-21T06:42:31.101 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-21T06:42:31.101 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:42:31.101 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-21T06:42:31.101 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:42:31.102 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:42:31.102 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-21T06:42:31.102 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 139.2, 2024-09-21T06:42:31.102 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-21T06:42:31.102 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-21T06:42:31.102 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-21T06:42:31.102 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:42:31.102 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-21T06:42:31.102 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:42:31.102 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:42:31.102 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:42:31.102 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:42:31.102 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:42:31.102 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:42:31.103 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-21T06:42:31.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.703+0000 7f84a95e0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bc0953850 0x556bc0cf9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:31.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.703+0000 7f84a95e0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bc092a990 0x556bc0cfa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:31.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.703+0000 7f84a0e08640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bc0953850 0x556bc0cf9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:31.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.703+0000 7f84a95e0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc0932b70 0x556bc0cfa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:31.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.703+0000 7f84a0e08640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bc0953850 0x556bc0cf9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.67:52364/0 (socket says 172.21.15.67:52364) 2024-09-21T06:42:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.703+0000 7f849bfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bc092a990 0x556bc0cfa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.703+0000 7f849bfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bc092a990 0x556bc0cfa3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.67:57732/0 (socket says 172.21.15.67:57732) 2024-09-21T06:42:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.703+0000 7f84a95e0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556bc0c70bb0 con 0x556bc0932b70 2024-09-21T06:42:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.703+0000 7f84a95e0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556bc0c5ffe0 con 0x556bc0953850 2024-09-21T06:42:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.703+0000 7f84a95e0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556bc0c63df0 con 0x556bc092a990 2024-09-21T06:42:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.703+0000 7f84a0e08640 1 -- 172.21.15.67:0/1442873796 learned_addr learned my addr 172.21.15.67:0/1442873796 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:42:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.703+0000 7f84a1609640 1 --2- 172.21.15.67:0/1442873796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc0932b70 0x556bc0cfa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.704+0000 7f849bfff640 1 --2- 172.21.15.67:0/671908366 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f84880b6510 0x7f84880b8a20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f849c10f910 tx=0x7f8484011920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:42:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.704+0000 7f84a1609640 1 -- 172.21.15.67:0/1442873796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bc092a990 msgr2=0x556bc0cfa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:42:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.704+0000 7f84a1609640 1 --2- 172.21.15.67:0/1442873796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bc092a990 0x556bc0cfa3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.704+0000 7f84a1609640 1 -- 172.21.15.67:0/1442873796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bc0953850 msgr2=0x556bc0cf9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:42:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.704+0000 7f84a1609640 1 --2- 172.21.15.67:0/1442873796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bc0953850 0x556bc0cf9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.704+0000 7f84a1609640 1 -- 172.21.15.67:0/1442873796 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bc09cdba0 con 0x556bc0932b70 2024-09-21T06:42:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.704+0000 7f849bfff640 1 --2- 172.21.15.67:0/1442873796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bc092a990 0x556bc0cfa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:42:31.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.704+0000 7f84a1609640 1 --2- 172.21.15.67:0/1442873796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc0932b70 0x556bc0cfa8f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f84940aa0a0 tx=0x7f84940ac840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdbceeba06ca062c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:42:31.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.704+0000 7f8492ffd640 1 -- 172.21.15.67:0/1442873796 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84940ac940 con 0x556bc0932b70 2024-09-21T06:42:31.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.704+0000 7f8492ffd640 1 -- 172.21.15.67:0/1442873796 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84940b1860 con 0x556bc0932b70 2024-09-21T06:42:31.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.704+0000 7f8492ffd640 1 -- 172.21.15.67:0/1442873796 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84940b4e60 con 0x556bc0932b70 2024-09-21T06:42:31.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.705+0000 7f84a95e0740 1 -- 172.21.15.67:0/1442873796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc0932b70 msgr2=0x556bc0cfa8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:31.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.705+0000 7f84a95e0740 1 --2- 172.21.15.67:0/1442873796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc0932b70 0x556bc0cfa8f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f84940aa0a0 tx=0x7f84940ac840 comp rx=0 tx=0).stop 2024-09-21T06:42:31.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.705+0000 7f84a95e0740 1 -- 172.21.15.67:0/1442873796 shutdown_connections 2024-09-21T06:42:31.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.705+0000 7f84a95e0740 1 --2- 172.21.15.67:0/1442873796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc0932b70 0x556bc0cfa8f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:31.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.705+0000 7f84a95e0740 1 --2- 172.21.15.67:0/1442873796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bc092a990 0x556bc0cfa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:31.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.705+0000 7f84a95e0740 1 --2- 172.21.15.67:0/1442873796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bc0953850 0x556bc0cf9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:31.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.705+0000 7f84a95e0740 1 -- 172.21.15.67:0/1442873796 >> 172.21.15.67:0/1442873796 conn(0x556bc0947c40 msgr2=0x556bc0cf7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:42:31.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.705+0000 7f84a95e0740 1 -- 172.21.15.67:0/1442873796 shutdown_connections 2024-09-21T06:42:31.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.705+0000 7f84a95e0740 1 -- 172.21.15.67:0/1442873796 wait complete. 2024-09-21T06:42:31.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.706+0000 7f84a95e0740 1 -- start start 2024-09-21T06:42:31.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.706+0000 7f84a95e0740 1 Processor -- start 2024-09-21T06:42:31.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.706+0000 7f84a95e0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bc092a990 0x556bc0d05af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:31.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.706+0000 7f84a95e0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc0932b70 0x556bc0d060d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:31.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.706+0000 7f849bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc0932b70 0x556bc0d060d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:31.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.706+0000 7f849bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc0932b70 0x556bc0d060d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51158/0 (socket says 172.21.15.67:51158) 2024-09-21T06:42:31.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.706+0000 7f84a95e0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bc0953850 0x556bc0d11940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:31.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.706+0000 7f84a95e0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556bc0ac17b0 con 0x556bc0932b70 2024-09-21T06:42:31.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.706+0000 7f84a95e0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556bc0bfae90 con 0x556bc0953850 2024-09-21T06:42:31.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.706+0000 7f84a95e0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556bc0d08650 con 0x556bc092a990 2024-09-21T06:42:31.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.707+0000 7f84a95e0740 10 client.0 _ll_register_callbacks cb 0x556bc0c5d710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:42:31.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.707+0000 7f84a0e08640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bc092a990 0x556bc0d05af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:31.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.707+0000 7f849bfff640 1 -- 172.21.15.67:0/2532464798 learned_addr learned my addr 172.21.15.67:0/2532464798 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:42:31.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.707+0000 7f84a1609640 1 --2- 172.21.15.67:0/2532464798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bc0953850 0x556bc0d11940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:31.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.707+0000 7f849bfff640 1 -- 172.21.15.67:0/2532464798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bc092a990 msgr2=0x556bc0d05af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:42:31.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.707+0000 7f849bfff640 1 --2- 172.21.15.67:0/2532464798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bc092a990 0x556bc0d05af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:31.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.707+0000 7f849bfff640 1 -- 172.21.15.67:0/2532464798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bc0953850 msgr2=0x556bc0d11940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:42:31.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.707+0000 7f849bfff640 1 --2- 172.21.15.67:0/2532464798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bc0953850 0x556bc0d11940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:31.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.707+0000 7f849bfff640 1 -- 172.21.15.67:0/2532464798 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bc0d11fb0 con 0x556bc0932b70 2024-09-21T06:42:31.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.707+0000 7f849bfff640 1 --2- 172.21.15.67:0/2532464798 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc0932b70 0x556bc0d060d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8484012fd0 tx=0x7f8484004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6803b27b75687d23 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:42:31.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.707+0000 7f847bfff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:42:31.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.707+0000 7f847bfff640 1 -- 172.21.15.67:0/2532464798 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848401a040 con 0x556bc0932b70 2024-09-21T06:42:31.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.707+0000 7f847bfff640 1 -- 172.21.15.67:0/2532464798 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f8484005020 con 0x556bc0932b70 2024-09-21T06:42:31.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.707+0000 7f84a95e0740 1 -- 172.21.15.67:0/2532464798 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84940b1860 con 0x556bc0932b70 2024-09-21T06:42:31.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.708+0000 7f847bfff640 1 -- 172.21.15.67:0/2532464798 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8484031050 con 0x556bc0932b70 2024-09-21T06:42:31.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.708+0000 7f84a95e0740 10 client.15678 fetch_fsmap learned FSMap version 21 2024-09-21T06:42:31.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.708+0000 7f84a95e0740 1 -- 172.21.15.67:0/2532464798 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556bc0d148e0 con 0x556bc0932b70 2024-09-21T06:42:31.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.708+0000 7f847bfff640 1 -- 172.21.15.67:0/2532464798 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84940b1860 con 0x556bc0932b70 2024-09-21T06:42:31.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.708+0000 7f847bfff640 1 -- 172.21.15.67:0/2532464798 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f84840035f0 con 0x556bc0932b70 2024-09-21T06:42:31.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.708+0000 7f84a95e0740 10 client.15678 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:42:31.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.708+0000 7f84a95e0740 10 client.15678 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:42:31.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.708+0000 7f84a95e0740 20 client.15678 populate_metadata read hostname 'smithi067' 2024-09-21T06:42:31.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.708+0000 7f84a95e0740 1 --2- 172.21.15.67:0/2532464798 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x556bc09121d0 0x556bc0d17eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:31.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.708+0000 7f84a95e0740 4 client.15678 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:42:31.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.708+0000 7f84a95e0740 1 -- 172.21.15.67:0/2532464798 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bc0c72b00 con 0x556bc09121d0 2024-09-21T06:42:31.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.708+0000 7f84a0e08640 1 --2- 172.21.15.67:0/2532464798 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x556bc09121d0 0x556bc0d17eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:31.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.709+0000 7f84a0e08640 1 --2- 172.21.15.67:0/2532464798 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x556bc09121d0 0x556bc0d17eb0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8758be65894e310 server_cookie=1d65043d305f1215 in_seq=0 out_seq=0 2024-09-21T06:42:31.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.709+0000 7f847bfff640 10 client.15678 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:42:31.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.710+0000 7f847bfff640 1 -- 172.21.15.67:0/2532464798 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556bc0c72b00 con 0x556bc09121d0 2024-09-21T06:42:31.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.710+0000 7f847bfff640 10 client.15678 handle_command_reply: tid=0 2024-09-21T06:42:31.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.710+0000 7f84a95e0740 1 client.15678 shutdown 2024-09-21T06:42:31.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.710+0000 7f84a95e0740 1 -- 172.21.15.67:0/2532464798 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc0932b70 msgr2=0x556bc0d060d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:31.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.710+0000 7f84a95e0740 1 --2- 172.21.15.67:0/2532464798 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc0932b70 0x556bc0d060d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8484012fd0 tx=0x7f8484004730 comp rx=0 tx=0).stop 2024-09-21T06:42:31.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.710+0000 7f84a95e0740 1 -- 172.21.15.67:0/2532464798 shutdown_connections 2024-09-21T06:42:31.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.710+0000 7f84a95e0740 1 --2- 172.21.15.67:0/2532464798 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x556bc09121d0 0x556bc0d17eb0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:31.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.710+0000 7f84a95e0740 1 --2- 172.21.15.67:0/2532464798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bc0953850 0x556bc0d11940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:31.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.710+0000 7f84a95e0740 1 --2- 172.21.15.67:0/2532464798 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc0932b70 0x556bc0d060d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:31.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.710+0000 7f847bfff640 0 client.15678 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:42:31.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.710+0000 7f84a95e0740 1 --2- 172.21.15.67:0/2532464798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bc092a990 0x556bc0d05af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:31.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.710+0000 7f84a95e0740 1 -- 172.21.15.67:0/2532464798 >> 172.21.15.67:0/2532464798 conn(0x556bc0947c40 msgr2=0x556bc0cfd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:42:31.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.710+0000 7f84a95e0740 1 -- 172.21.15.67:0/2532464798 shutdown_connections 2024-09-21T06:42:31.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.710+0000 7f84a95e0740 1 -- 172.21.15.67:0/2532464798 wait complete. 2024-09-21T06:42:31.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.710+0000 7f84a95e0740 20 client.15678 trim_cache size 0 max 16384 2024-09-21T06:42:31.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.727+0000 7f84a95e0740 1 Processor -- start 2024-09-21T06:42:31.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.727+0000 7f84a95e0740 1 -- start start 2024-09-21T06:42:31.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.728+0000 7f84a95e0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bc092a990 0x556bc0d11840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:31.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.728+0000 7f84a95e0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc0932b70 0x556bc0cfef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:31.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.728+0000 7f84a95e0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bc0953850 0x556bc0cff490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:31.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.728+0000 7f84a95e0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f849c1b5220 con 0x556bc0932b70 2024-09-21T06:42:31.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.728+0000 7f84a95e0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f849c1b7aa0 con 0x556bc0953850 2024-09-21T06:42:31.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.728+0000 7f84a95e0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f849c1b5890 con 0x556bc092a990 2024-09-21T06:42:31.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.728+0000 7f849bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc0932b70 0x556bc0cfef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:31.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.728+0000 7f849bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc0932b70 0x556bc0cfef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51170/0 (socket says 172.21.15.67:51170) 2024-09-21T06:42:31.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.728+0000 7f849bfff640 1 -- 172.21.15.67:0/2280402651 learned_addr learned my addr 172.21.15.67:0/2280402651 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:42:31.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.728+0000 7f84a0e08640 1 --2- 172.21.15.67:0/2280402651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bc092a990 0x556bc0d11840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:31.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.728+0000 7f84a1609640 1 --2- 172.21.15.67:0/2280402651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bc0953850 0x556bc0cff490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:31.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.728+0000 7f849bfff640 1 -- 172.21.15.67:0/2280402651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bc092a990 msgr2=0x556bc0d11840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:42:31.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.728+0000 7f849bfff640 1 --2- 172.21.15.67:0/2280402651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bc092a990 0x556bc0d11840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:31.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.728+0000 7f849bfff640 1 -- 172.21.15.67:0/2280402651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bc0953850 msgr2=0x556bc0cff490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:42:31.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.728+0000 7f849bfff640 1 --2- 172.21.15.67:0/2280402651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bc0953850 0x556bc0cff490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:31.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.728+0000 7f849bfff640 1 -- 172.21.15.67:0/2280402651 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bc0d01180 con 0x556bc0932b70 2024-09-21T06:42:31.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.728+0000 7f849bfff640 1 --2- 172.21.15.67:0/2280402651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc0932b70 0x556bc0cfef50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8484004700 tx=0x7f848400fe80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769d1f6d3081e011 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:42:31.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.728+0000 7f84927fc640 1 -- 172.21.15.67:0/2280402651 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848401a070 con 0x556bc0932b70 2024-09-21T06:42:31.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.728+0000 7f84927fc640 1 -- 172.21.15.67:0/2280402651 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8484024070 con 0x556bc0932b70 2024-09-21T06:42:31.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.728+0000 7f84927fc640 1 -- 172.21.15.67:0/2280402651 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8484020b40 con 0x556bc0932b70 2024-09-21T06:42:31.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.729+0000 7f84a95e0740 1 -- 172.21.15.67:0/2280402651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc0932b70 msgr2=0x556bc0cfef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:31.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.729+0000 7f84a95e0740 1 --2- 172.21.15.67:0/2280402651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc0932b70 0x556bc0cfef50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8484004700 tx=0x7f848400fe80 comp rx=0 tx=0).stop 2024-09-21T06:42:31.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.729+0000 7f84a95e0740 1 -- 172.21.15.67:0/2280402651 shutdown_connections 2024-09-21T06:42:31.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.729+0000 7f84a95e0740 1 --2- 172.21.15.67:0/2280402651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bc0953850 0x556bc0cff490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:31.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.729+0000 7f84a95e0740 1 --2- 172.21.15.67:0/2280402651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc0932b70 0x556bc0cfef50 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:31.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.729+0000 7f84a95e0740 1 --2- 172.21.15.67:0/2280402651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bc092a990 0x556bc0d11840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:31.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.729+0000 7f84a95e0740 1 -- 172.21.15.67:0/2280402651 >> 172.21.15.67:0/2280402651 conn(0x556bc0947c40 msgr2=0x556bc0cfcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:42:31.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.729+0000 7f84a95e0740 1 -- 172.21.15.67:0/2280402651 shutdown_connections 2024-09-21T06:42:31.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.729+0000 7f84a95e0740 1 -- 172.21.15.67:0/2280402651 wait complete. 2024-09-21T06:42:31.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.730+0000 7f84a95e0740 1 -- start start 2024-09-21T06:42:31.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.730+0000 7f84a95e0740 1 Processor -- start 2024-09-21T06:42:31.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.730+0000 7f84a95e0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bc092a990 0x556bc0d0dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:31.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.730+0000 7f84a95e0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc0932b70 0x556bc0d0c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:31.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.730+0000 7f84a95e0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bc0953850 0x556bc0d0ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:31.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.730+0000 7f84a95e0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f849c1b5f60 con 0x556bc0932b70 2024-09-21T06:42:31.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.730+0000 7f84a95e0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f849c1b73d0 con 0x556bc0953850 2024-09-21T06:42:31.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.730+0000 7f84a95e0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f849c1b6630 con 0x556bc092a990 2024-09-21T06:42:31.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.730+0000 7f84a95e0740 10 client.0 _ll_register_callbacks cb 0x7f8484009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:42:31.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.730+0000 7f849bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc0932b70 0x556bc0d0c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:31.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.730+0000 7f849bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc0932b70 0x556bc0d0c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51182/0 (socket says 172.21.15.67:51182) 2024-09-21T06:42:31.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.730+0000 7f849bfff640 1 -- 172.21.15.67:0/1093462346 learned_addr learned my addr 172.21.15.67:0/1093462346 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:42:31.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.730+0000 7f84a1609640 1 --2- 172.21.15.67:0/1093462346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bc0953850 0x556bc0d0ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:31.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.730+0000 7f84a0e08640 1 --2- 172.21.15.67:0/1093462346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bc092a990 0x556bc0d0dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:31.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.731+0000 7f849bfff640 1 -- 172.21.15.67:0/1093462346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bc092a990 msgr2=0x556bc0d0dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:42:31.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.731+0000 7f849bfff640 1 --2- 172.21.15.67:0/1093462346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bc092a990 0x556bc0d0dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:31.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.731+0000 7f849bfff640 1 -- 172.21.15.67:0/1093462346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bc0953850 msgr2=0x556bc0d0ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:42:31.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.731+0000 7f849bfff640 1 --2- 172.21.15.67:0/1093462346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bc0953850 0x556bc0d0ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:31.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.731+0000 7f849bfff640 1 -- 172.21.15.67:0/1093462346 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bc0d0d360 con 0x556bc0932b70 2024-09-21T06:42:31.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.731+0000 7f849bfff640 1 --2- 172.21.15.67:0/1093462346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc0932b70 0x556bc0d0c8e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8484009220 tx=0x7f8484020c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1ea878c513b30f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:42:31.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.731+0000 7f847bfff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:42:31.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.731+0000 7f847bfff640 1 -- 172.21.15.67:0/1093462346 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848401a070 con 0x556bc0932b70 2024-09-21T06:42:31.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.731+0000 7f847bfff640 1 -- 172.21.15.67:0/1093462346 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f8484005230 con 0x556bc0932b70 2024-09-21T06:42:31.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.731+0000 7f84a95e0740 1 -- 172.21.15.67:0/1093462346 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8484024070 con 0x556bc0932b70 2024-09-21T06:42:31.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.731+0000 7f847bfff640 1 -- 172.21.15.67:0/1093462346 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8484047050 con 0x556bc0932b70 2024-09-21T06:42:31.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.731+0000 7f84a95e0740 10 client.15684 fetch_fsmap learned FSMap version 21 2024-09-21T06:42:31.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.731+0000 7f84a95e0740 1 -- 172.21.15.67:0/1093462346 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556bc0d834c0 con 0x556bc0932b70 2024-09-21T06:42:31.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.732+0000 7f847bfff640 1 -- 172.21.15.67:0/1093462346 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8484024070 con 0x556bc0932b70 2024-09-21T06:42:31.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.732+0000 7f847bfff640 1 -- 172.21.15.67:0/1093462346 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f848403c020 con 0x556bc0932b70 2024-09-21T06:42:31.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.732+0000 7f84a95e0740 10 client.15684 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:42:31.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.732+0000 7f84a95e0740 10 client.15684 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:42:31.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.732+0000 7f84a95e0740 20 client.15684 populate_metadata read hostname 'smithi067' 2024-09-21T06:42:31.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.732+0000 7f84a95e0740 1 --2- 172.21.15.67:0/1093462346 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x556bc08e7580 0x556bc0d85ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:31.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.732+0000 7f84a95e0740 4 client.15684 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.25, "await_for": 45.0}] 2024-09-21T06:42:31.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.732+0000 7f84a95e0740 1 -- 172.21.15.67:0/1093462346 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.25, "await_for": 45.0}) -- 0x556bc0c6e5b0 con 0x556bc08e7580 2024-09-21T06:42:31.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.732+0000 7f84a0e08640 1 --2- 172.21.15.67:0/1093462346 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x556bc08e7580 0x556bc0d85ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:31.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.732+0000 7f847bfff640 10 client.15684 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:42:31.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.732+0000 7f84a0e08640 1 --2- 172.21.15.67:0/1093462346 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x556bc08e7580 0x556bc0d85ba0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b16704687f0b82de server_cookie=fadb8e855d69568b in_seq=0 out_seq=0 2024-09-21T06:42:31.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.935+0000 7f847bfff640 1 -- 172.21.15.67:0/1093462346 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556bc0c6e5b0 con 0x556bc08e7580 2024-09-21T06:42:31.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.935+0000 7f847bfff640 10 client.15684 handle_command_reply: tid=0 2024-09-21T06:42:31.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.935+0000 7f84a95e0740 1 client.15684 shutdown 2024-09-21T06:42:31.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.935+0000 7f84a95e0740 1 -- 172.21.15.67:0/1093462346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc0932b70 msgr2=0x556bc0d0c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:31.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.935+0000 7f84a95e0740 1 --2- 172.21.15.67:0/1093462346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc0932b70 0x556bc0d0c8e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8484009220 tx=0x7f8484020c60 comp rx=0 tx=0).stop 2024-09-21T06:42:31.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.935+0000 7f84a95e0740 1 -- 172.21.15.67:0/1093462346 shutdown_connections 2024-09-21T06:42:31.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.935+0000 7f84a95e0740 1 --2- 172.21.15.67:0/1093462346 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x556bc08e7580 0x556bc0d85ba0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:31.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.935+0000 7f84a95e0740 1 --2- 172.21.15.67:0/1093462346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bc0953850 0x556bc0d0ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:31.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.935+0000 7f84a95e0740 1 --2- 172.21.15.67:0/1093462346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc0932b70 0x556bc0d0c8e0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:31.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.935+0000 7f84a95e0740 1 --2- 172.21.15.67:0/1093462346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bc092a990 0x556bc0d0dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:31.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.935+0000 7f847bfff640 0 client.15684 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:42:31.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.935+0000 7f84a95e0740 1 -- 172.21.15.67:0/1093462346 >> 172.21.15.67:0/1093462346 conn(0x556bc0947c40 msgr2=0x556bc0cf7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:42:31.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.935+0000 7f84a95e0740 1 -- 172.21.15.67:0/1093462346 shutdown_connections 2024-09-21T06:42:31.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.935+0000 7f84a95e0740 1 -- 172.21.15.67:0/1093462346 wait complete. 2024-09-21T06:42:31.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.935+0000 7f84a95e0740 20 client.15684 trim_cache size 0 max 16384 2024-09-21T06:42:31.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.936+0000 7f84a306c640 1 -- 172.21.15.67:0/671908366 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f84880b6510 msgr2=0x7f84880b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:31.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.936+0000 7f84a306c640 1 --2- 172.21.15.67:0/671908366 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f84880b6510 0x7f84880b8a20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f849c10f910 tx=0x7f8484011920 comp rx=0 tx=0).stop 2024-09-21T06:42:31.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.936+0000 7f84a306c640 1 -- 172.21.15.67:0/671908366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f849c110420 msgr2=0x7f849c1a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:31.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.936+0000 7f84a306c640 1 --2- 172.21.15.67:0/671908366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f849c110420 0x7f849c1a6300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f849400be20 tx=0x7f849400bf10 comp rx=0 tx=0).stop 2024-09-21T06:42:31.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.936+0000 7f84a306c640 1 -- 172.21.15.67:0/671908366 shutdown_connections 2024-09-21T06:42:31.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.936+0000 7f84a306c640 1 --2- 172.21.15.67:0/671908366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f849c110420 0x7f849c1a6300 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:31.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.936+0000 7f84a306c640 1 --2- 172.21.15.67:0/671908366 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f84880b6510 0x7f84880b8a20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:31.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.936+0000 7f84a306c640 1 --2- 172.21.15.67:0/671908366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f849c10fad0 0x7f849c1a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:31.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.936+0000 7f84a306c640 1 --2- 172.21.15.67:0/671908366 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f849c10a190 0x7f849c1a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:31.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.936+0000 7f84a306c640 1 -- 172.21.15.67:0/671908366 >> 172.21.15.67:0/671908366 conn(0x7f849c0781c0 msgr2=0x7f849c108860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:42:31.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.936+0000 7f84a306c640 1 -- 172.21.15.67:0/671908366 shutdown_connections 2024-09-21T06:42:31.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:30.936+0000 7f84a306c640 1 -- 172.21.15.67:0/671908366 wait complete. 2024-09-21T06:42:31.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:31 smithi100 ceph-mon[33401]: pgmap v538: 129 pgs: 129 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 33 MiB/s wr, 529 op/s 2024-09-21T06:42:31.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:31 smithi203 ceph-mon[33735]: pgmap v538: 129 pgs: 129 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 33 MiB/s wr, 529 op/s 2024-09-21T06:42:32.040 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:31 smithi067 ceph-mon[31232]: pgmap v538: 129 pgs: 129 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 33 MiB/s wr, 529 op/s 2024-09-21T06:42:32.074 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '79dfa6b2', quiesce took 14.3 seconds. Will release after: 4.949999999999999 2024-09-21T06:42:33.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:33 smithi203 ceph-mon[33735]: pgmap v539: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 43 MiB/s wr, 685 op/s 2024-09-21T06:42:34.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:33 smithi067 ceph-mon[31232]: pgmap v539: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 43 MiB/s wr, 685 op/s 2024-09-21T06:42:34.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:33 smithi100 ceph-mon[33401]: pgmap v539: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 43 MiB/s wr, 685 op/s 2024-09-21T06:42:35.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:35 smithi203 ceph-mon[33735]: pgmap v540: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 32 MiB/s wr, 513 op/s 2024-09-21T06:42:36.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:35 smithi067 ceph-mon[31232]: pgmap v540: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 32 MiB/s wr, 513 op/s 2024-09-21T06:42:36.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:35 smithi100 ceph-mon[33401]: pgmap v540: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 32 MiB/s wr, 513 op/s 2024-09-21T06:42:37.025 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:42:37.383 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:42:37.778 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:37 smithi067 ceph-mon[31232]: pgmap v541: 129 pgs: 129 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 28 MiB/s wr, 456 op/s 2024-09-21T06:42:37.778 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:37 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:42:37.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.868+0000 7f28c98f4640 1 -- 172.21.15.67:0/3328109700 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28c41083a0 msgr2=0x7f28c41087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:37.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.868+0000 7f28c98f4640 1 --2- 172.21.15.67:0/3328109700 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28c41083a0 0x7f28c41087b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f28ac009900 tx=0x7f28ac02a410 comp rx=0 tx=0).stop 2024-09-21T06:42:37.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.868+0000 7f28c98f4640 1 -- 172.21.15.67:0/3328109700 shutdown_connections 2024-09-21T06:42:37.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.868+0000 7f28c98f4640 1 --2- 172.21.15.67:0/3328109700 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28c4108cf0 0x7f28c410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:37.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.868+0000 7f28c98f4640 1 --2- 172.21.15.67:0/3328109700 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28c41083a0 0x7f28c41087b0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:37.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.868+0000 7f28c98f4640 1 --2- 172.21.15.67:0/3328109700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28c4102a60 0x7f28c4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:37.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.868+0000 7f28c98f4640 1 -- 172.21.15.67:0/3328109700 >> 172.21.15.67:0/3328109700 conn(0x7f28c4079730 msgr2=0x7f28c4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:42:37.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.868+0000 7f28c98f4640 1 -- 172.21.15.67:0/3328109700 shutdown_connections 2024-09-21T06:42:37.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.869+0000 7f28c98f4640 1 -- 172.21.15.67:0/3328109700 wait complete. 2024-09-21T06:42:37.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.869+0000 7f28c98f4640 1 Processor -- start 2024-09-21T06:42:37.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.869+0000 7f28c98f4640 1 -- start start 2024-09-21T06:42:37.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.870+0000 7f28c98f4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28c4102a60 0x7f28c419a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:37.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.870+0000 7f28c98f4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28c41083a0 0x7f28c419a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:37.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.870+0000 7f28c98f4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28c4108cf0 0x7f28c419fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:37.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.870+0000 7f28c98f4640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f28c4112810 con 0x7f28c4108cf0 2024-09-21T06:42:37.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.870+0000 7f28c98f4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f28c4112690 con 0x7f28c4102a60 2024-09-21T06:42:37.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.870+0000 7f28c98f4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f28c4112990 con 0x7f28c41083a0 2024-09-21T06:42:37.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.870+0000 7f28c37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28c4108cf0 0x7f28c419fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:37.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.870+0000 7f28c2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28c4102a60 0x7f28c419a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:37.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.870+0000 7f28c2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28c4102a60 0x7f28c419a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.67:51550/0 (socket says 172.21.15.67:51550) 2024-09-21T06:42:37.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.870+0000 7f28c37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28c4108cf0 0x7f28c419fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36388/0 (socket says 172.21.15.67:36388) 2024-09-21T06:42:37.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.870+0000 7f28c2ffd640 1 -- 172.21.15.67:0/2890450523 learned_addr learned my addr 172.21.15.67:0/2890450523 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:42:37.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.870+0000 7f28c27fc640 1 --2- 172.21.15.67:0/2890450523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28c41083a0 0x7f28c419a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:37.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.870+0000 7f28c2ffd640 1 -- 172.21.15.67:0/2890450523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28c41083a0 msgr2=0x7f28c419a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:37.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.870+0000 7f28c2ffd640 1 --2- 172.21.15.67:0/2890450523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28c41083a0 0x7f28c419a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:37.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.870+0000 7f28c2ffd640 1 -- 172.21.15.67:0/2890450523 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28c4108cf0 msgr2=0x7f28c419fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:37.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.870+0000 7f28c2ffd640 1 --2- 172.21.15.67:0/2890450523 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28c4108cf0 0x7f28c419fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:37.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.870+0000 7f28c2ffd640 1 -- 172.21.15.67:0/2890450523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c41a0300 con 0x7f28c4102a60 2024-09-21T06:42:37.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.870+0000 7f28c37fe640 1 --2- 172.21.15.67:0/2890450523 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28c4108cf0 0x7f28c419fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:42:37.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.870+0000 7f28c2ffd640 1 --2- 172.21.15.67:0/2890450523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28c4102a60 0x7f28c419a390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f28b400ea60 tx=0x7f28b400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:42:37.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.871+0000 7f28c88f2640 1 -- 172.21.15.67:0/2890450523 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b400cde0 con 0x7f28c4102a60 2024-09-21T06:42:37.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.871+0000 7f28c98f4640 1 -- 172.21.15.67:0/2890450523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28c41a05f0 con 0x7f28c4102a60 2024-09-21T06:42:37.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.871+0000 7f28c98f4640 1 -- 172.21.15.67:0/2890450523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28c41a6d50 con 0x7f28c4102a60 2024-09-21T06:42:37.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.871+0000 7f28c88f2640 1 -- 172.21.15.67:0/2890450523 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28b40035f0 con 0x7f28c4102a60 2024-09-21T06:42:37.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.871+0000 7f28c88f2640 1 -- 172.21.15.67:0/2890450523 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b4018670 con 0x7f28c4102a60 2024-09-21T06:42:37.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.872+0000 7f28c98f4640 1 -- 172.21.15.67:0/2890450523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28c4102e70 con 0x7f28c4102a60 2024-09-21T06:42:37.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.873+0000 7f28c88f2640 1 -- 172.21.15.67:0/2890450523 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f28b4003a10 con 0x7f28c4102a60 2024-09-21T06:42:37.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.874+0000 7f28c88f2640 1 --2- 172.21.15.67:0/2890450523 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f28b00b6510 0x7f28b00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:37.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.874+0000 7f28c27fc640 1 --2- 172.21.15.67:0/2890450523 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f28b00b6510 0x7f28b00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:37.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.874+0000 7f28c88f2640 1 -- 172.21.15.67:0/2890450523 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f28b40e7a90 con 0x7f28c4102a60 2024-09-21T06:42:37.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.875+0000 7f28c27fc640 1 --2- 172.21.15.67:0/2890450523 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f28b00b6510 0x7f28b00b8a20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f28ac006010 tx=0x7f28ac032ba0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:42:37.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:37.879+0000 7f28c88f2640 1 -- 172.21.15.67:0/2890450523 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f28b40adf50 con 0x7f28c4102a60 2024-09-21T06:42:37.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:37 smithi203 ceph-mon[33735]: pgmap v541: 129 pgs: 129 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 28 MiB/s wr, 456 op/s 2024-09-21T06:42:37.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:37 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:42:38.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:37 smithi100 ceph-mon[33401]: pgmap v541: 129 pgs: 129 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 28 MiB/s wr, 456 op/s 2024-09-21T06:42:38.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:37 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:42:38.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:38.114+0000 7f28c98f4640 1 -- 172.21.15.67:0/2890450523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28c419b4c0 con 0x7f28c4102a60 2024-09-21T06:42:38.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:38.115+0000 7f28c88f2640 1 -- 172.21.15.67:0/2890450523 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7f28b40b2330 con 0x7f28c4102a60 2024-09-21T06:42:38.116 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:42:38.117 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:42:38.117 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:42:38.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:38.117+0000 7f28c98f4640 1 -- 172.21.15.67:0/2890450523 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f28b00b6510 msgr2=0x7f28b00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:38.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:38.117+0000 7f28c98f4640 1 --2- 172.21.15.67:0/2890450523 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f28b00b6510 0x7f28b00b8a20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f28ac006010 tx=0x7f28ac032ba0 comp rx=0 tx=0).stop 2024-09-21T06:42:38.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:38.117+0000 7f28c98f4640 1 -- 172.21.15.67:0/2890450523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28c4102a60 msgr2=0x7f28c419a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:38.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:38.117+0000 7f28c98f4640 1 --2- 172.21.15.67:0/2890450523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28c4102a60 0x7f28c419a390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f28b400ea60 tx=0x7f28b400ef20 comp rx=0 tx=0).stop 2024-09-21T06:42:38.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:38.117+0000 7f28c98f4640 1 -- 172.21.15.67:0/2890450523 shutdown_connections 2024-09-21T06:42:38.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:38.117+0000 7f28c98f4640 1 --2- 172.21.15.67:0/2890450523 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f28b00b6510 0x7f28b00b8a20 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:38.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:38.117+0000 7f28c98f4640 1 --2- 172.21.15.67:0/2890450523 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28c4108cf0 0x7f28c419fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:38.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:38.117+0000 7f28c98f4640 1 --2- 172.21.15.67:0/2890450523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28c41083a0 0x7f28c419a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:38.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:38.117+0000 7f28c98f4640 1 --2- 172.21.15.67:0/2890450523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28c4102a60 0x7f28c419a390 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:38.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:38.117+0000 7f28c98f4640 1 -- 172.21.15.67:0/2890450523 >> 172.21.15.67:0/2890450523 conn(0x7f28c4079730 msgr2=0x7f28c406b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:42:38.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:38.117+0000 7f28c98f4640 1 -- 172.21.15.67:0/2890450523 shutdown_connections 2024-09-21T06:42:38.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:38.117+0000 7f28c98f4640 1 -- 172.21.15.67:0/2890450523 wait complete. 2024-09-21T06:42:38.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:38 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:42:38.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:38 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:42:38.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:38 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:42:38.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:38 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2890450523' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:42:38.962 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db --set-id 79dfa6b2 --release --await' 2024-09-21T06:42:38.962 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24671 quiesce db --set-id 79dfa6b2 --release --await 2024-09-21T06:42:38.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:38 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:42:38.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:38 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:42:38.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:38 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:42:38.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:38 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2890450523' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:42:39.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:38 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:42:39.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:38 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:42:39.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:38 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:42:39.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:38 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2890450523' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:42:39.225 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:42:39.614 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:39 smithi067 ceph-mon[31232]: pgmap v542: 129 pgs: 129 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 24 MiB/s wr, 382 op/s 2024-09-21T06:42:39.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.713+0000 7f9e33420640 1 -- 172.21.15.67:0/1037211952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e2c1083d0 msgr2=0x7f9e2c1087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:39.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.713+0000 7f9e33420640 1 --2- 172.21.15.67:0/1037211952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e2c1083d0 0x7f9e2c1087e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c009900 tx=0x7f9e1c02a410 comp rx=0 tx=0).stop 2024-09-21T06:42:39.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.713+0000 7f9e33420640 1 -- 172.21.15.67:0/1037211952 shutdown_connections 2024-09-21T06:42:39.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.713+0000 7f9e33420640 1 --2- 172.21.15.67:0/1037211952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e2c108d20 0x7f9e2c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.713+0000 7f9e33420640 1 --2- 172.21.15.67:0/1037211952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e2c1083d0 0x7f9e2c1087e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.713+0000 7f9e33420640 1 --2- 172.21.15.67:0/1037211952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e2c102ae0 0x7f9e2c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.713+0000 7f9e33420640 1 -- 172.21.15.67:0/1037211952 >> 172.21.15.67:0/1037211952 conn(0x7f9e2c079760 msgr2=0x7f9e2c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:42:39.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.713+0000 7f9e33420640 1 -- 172.21.15.67:0/1037211952 shutdown_connections 2024-09-21T06:42:39.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.713+0000 7f9e33420640 1 -- 172.21.15.67:0/1037211952 wait complete. 2024-09-21T06:42:39.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.714+0000 7f9e33420640 1 Processor -- start 2024-09-21T06:42:39.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.714+0000 7f9e33420640 1 -- start start 2024-09-21T06:42:39.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.714+0000 7f9e33420640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e2c102ae0 0x7f9e2c0ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:39.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.714+0000 7f9e33420640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e2c1083d0 0x7f9e2c0ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:39.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.714+0000 7f9e33420640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e2c108d20 0x7f9e2c1000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:39.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.714+0000 7f9e33420640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c112840 con 0x7f9e2c1083d0 2024-09-21T06:42:39.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.714+0000 7f9e33420640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c1126c0 con 0x7f9e2c108d20 2024-09-21T06:42:39.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.714+0000 7f9e33420640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c1129c0 con 0x7f9e2c102ae0 2024-09-21T06:42:39.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.714+0000 7f9e309bb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e2c1083d0 0x7f9e2c0ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:39.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.715+0000 7f9e309bb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e2c1083d0 0x7f9e2c0ffbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36396/0 (socket says 172.21.15.67:36396) 2024-09-21T06:42:39.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.715+0000 7f9e309bb640 1 -- 172.21.15.67:0/3342114595 learned_addr learned my addr 172.21.15.67:0/3342114595 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:42:39.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.715+0000 7f9e319bd640 1 --2- 172.21.15.67:0/3342114595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e2c108d20 0x7f9e2c1000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:39.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.715+0000 7f9e311bc640 1 --2- 172.21.15.67:0/3342114595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e2c102ae0 0x7f9e2c0ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:39.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.715+0000 7f9e309bb640 1 -- 172.21.15.67:0/3342114595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e2c102ae0 msgr2=0x7f9e2c0ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:39.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.715+0000 7f9e309bb640 1 --2- 172.21.15.67:0/3342114595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e2c102ae0 0x7f9e2c0ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.715+0000 7f9e309bb640 1 -- 172.21.15.67:0/3342114595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e2c108d20 msgr2=0x7f9e2c1000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:39.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.715+0000 7f9e309bb640 1 --2- 172.21.15.67:0/3342114595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e2c108d20 0x7f9e2c1000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.715+0000 7f9e309bb640 1 -- 172.21.15.67:0/3342114595 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e2c1a86e0 con 0x7f9e2c1083d0 2024-09-21T06:42:39.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.715+0000 7f9e311bc640 1 --2- 172.21.15.67:0/3342114595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e2c102ae0 0x7f9e2c0ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:42:39.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.715+0000 7f9e309bb640 1 --2- 172.21.15.67:0/3342114595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e2c1083d0 0x7f9e2c0ffbb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c005940 tx=0x7f9e1c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:42:39.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.715+0000 7f9e227fc640 1 -- 172.21.15.67:0/3342114595 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c03f030 con 0x7f9e2c1083d0 2024-09-21T06:42:39.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.715+0000 7f9e227fc640 1 -- 172.21.15.67:0/3342114595 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9e1c003c90 con 0x7f9e2c1083d0 2024-09-21T06:42:39.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.715+0000 7f9e33420640 1 -- 172.21.15.67:0/3342114595 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e2c1a8970 con 0x7f9e2c1083d0 2024-09-21T06:42:39.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.715+0000 7f9e227fc640 1 -- 172.21.15.67:0/3342114595 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c035070 con 0x7f9e2c1083d0 2024-09-21T06:42:39.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.715+0000 7f9e33420640 1 -- 172.21.15.67:0/3342114595 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e2c1a8e50 con 0x7f9e2c1083d0 2024-09-21T06:42:39.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.718+0000 7f9e39994740 1 Processor -- start 2024-09-21T06:42:39.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.718+0000 7f9e39994740 1 -- start start 2024-09-21T06:42:39.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.718+0000 7f9e227fc640 1 -- 172.21.15.67:0/3342114595 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f9e1c042030 con 0x7f9e2c1083d0 2024-09-21T06:42:39.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.719+0000 7f9e39994740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55653429b520 0x556534641510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:39.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.719+0000 7f9e39994740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556534272660 0x556534641a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:39.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.719+0000 7f9e39994740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55653427a840 0x556534641f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:39.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.719+0000 7f9e39994740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55653452cdc0 con 0x55653427a840 2024-09-21T06:42:39.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.719+0000 7f9e39994740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5565345b91f0 con 0x55653429b520 2024-09-21T06:42:39.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.719+0000 7f9e39994740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5565345a8620 con 0x556534272660 2024-09-21T06:42:39.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.719+0000 7f9e309bb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55653427a840 0x556534641f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:39.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.719+0000 7f9e309bb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55653427a840 0x556534641f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36402/0 (socket says 172.21.15.67:36402) 2024-09-21T06:42:39.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.719+0000 7f9e309bb640 1 -- 172.21.15.67:0/1026010121 learned_addr learned my addr 172.21.15.67:0/1026010121 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:42:39.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.719+0000 7f9e319bd640 1 --2- 172.21.15.67:0/1026010121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55653429b520 0x556534641510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:39.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.719+0000 7f9e311bc640 1 --2- 172.21.15.67:0/1026010121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556534272660 0x556534641a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:39.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.719+0000 7f9e309bb640 1 -- 172.21.15.67:0/1026010121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556534272660 msgr2=0x556534641a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:42:39.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.719+0000 7f9e309bb640 1 --2- 172.21.15.67:0/1026010121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556534272660 0x556534641a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.719+0000 7f9e309bb640 1 -- 172.21.15.67:0/1026010121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55653429b520 msgr2=0x556534641510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:42:39.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.719+0000 7f9e309bb640 1 --2- 172.21.15.67:0/1026010121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55653429b520 0x556534641510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.719+0000 7f9e309bb640 1 -- 172.21.15.67:0/1026010121 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556534570fd0 con 0x55653427a840 2024-09-21T06:42:39.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.719+0000 7f9e319bd640 1 --2- 172.21.15.67:0/1026010121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55653429b520 0x556534641510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:42:39.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.719+0000 7f9e309bb640 1 --2- 172.21.15.67:0/1026010121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55653427a840 0x556534641f90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c10be80 tx=0x7f9e1c10bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f70d0dd91a5778 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:42:39.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.719+0000 7f9e227fc640 1 --2- 172.21.15.67:0/3342114595 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9e180b6510 0x7f9e180b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:39.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.720+0000 7f9e177fe640 1 -- 172.21.15.67:0/1026010121 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c0ca0f0 con 0x55653427a840 2024-09-21T06:42:39.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.720+0000 7f9e319bd640 1 --2- 172.21.15.67:0/3342114595 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9e180b6510 0x7f9e180b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:39.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.720+0000 7f9e227fc640 1 -- 172.21.15.67:0/3342114595 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f9e1c107ce0 con 0x7f9e2c1083d0 2024-09-21T06:42:39.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.720+0000 7f9e177fe640 1 -- 172.21.15.67:0/1026010121 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9e1c115070 con 0x55653427a840 2024-09-21T06:42:39.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.720+0000 7f9e177fe640 1 -- 172.21.15.67:0/1026010121 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c0cd210 con 0x55653427a840 2024-09-21T06:42:39.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.721+0000 7f9e39994740 1 -- 172.21.15.67:0/1026010121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55653427a840 msgr2=0x556534641f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:39.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.721+0000 7f9e39994740 1 --2- 172.21.15.67:0/1026010121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55653427a840 0x556534641f90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c10be80 tx=0x7f9e1c10bf70 comp rx=0 tx=0).stop 2024-09-21T06:42:39.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.721+0000 7f9e319bd640 1 --2- 172.21.15.67:0/3342114595 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9e180b6510 0x7f9e180b8a20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9e240092f0 tx=0x7f9e24009280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:42:39.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.721+0000 7f9e39994740 1 -- 172.21.15.67:0/1026010121 shutdown_connections 2024-09-21T06:42:39.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.721+0000 7f9e39994740 1 --2- 172.21.15.67:0/1026010121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55653427a840 0x556534641f90 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.721+0000 7f9e39994740 1 --2- 172.21.15.67:0/1026010121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556534272660 0x556534641a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.721+0000 7f9e39994740 1 --2- 172.21.15.67:0/1026010121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55653429b520 0x556534641510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.721+0000 7f9e39994740 1 -- 172.21.15.67:0/1026010121 >> 172.21.15.67:0/1026010121 conn(0x55653428f910 msgr2=0x55653463f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:42:39.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.721+0000 7f9e39994740 1 -- 172.21.15.67:0/1026010121 shutdown_connections 2024-09-21T06:42:39.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.721+0000 7f9e39994740 1 -- 172.21.15.67:0/1026010121 wait complete. 2024-09-21T06:42:39.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.721+0000 7f9e39994740 1 -- start start 2024-09-21T06:42:39.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.722+0000 7f9e39994740 1 Processor -- start 2024-09-21T06:42:39.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.722+0000 7f9e39994740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556534272660 0x55653464cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:39.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.722+0000 7f9e39994740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55653427a840 0x55653464d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:39.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.722+0000 7f9e39994740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55653429b520 0x556534658b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:39.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.722+0000 7f9e39994740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556534543300 con 0x55653427a840 2024-09-21T06:42:39.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.722+0000 7f9e39994740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5565345ac430 con 0x55653429b520 2024-09-21T06:42:39.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.722+0000 7f9e39994740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556534409bb0 con 0x556534272660 2024-09-21T06:42:39.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.722+0000 7f9e39994740 10 client.0 _ll_register_callbacks cb 0x55653455d120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:42:39.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.722+0000 7f9e309bb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55653427a840 0x55653464d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:39.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.722+0000 7f9e309bb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55653427a840 0x55653464d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36412/0 (socket says 172.21.15.67:36412) 2024-09-21T06:42:39.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.722+0000 7f9e309bb640 1 -- 172.21.15.67:0/3215892866 learned_addr learned my addr 172.21.15.67:0/3215892866 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:42:39.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.722+0000 7f9e311bc640 1 --2- 172.21.15.67:0/3215892866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556534272660 0x55653464cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:39.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.722+0000 7f9e319bd640 1 --2- 172.21.15.67:0/3215892866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55653429b520 0x556534658b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:39.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.723+0000 7f9e309bb640 1 -- 172.21.15.67:0/3215892866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556534272660 msgr2=0x55653464cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:42:39.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.723+0000 7f9e309bb640 1 --2- 172.21.15.67:0/3215892866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556534272660 0x55653464cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.723+0000 7f9e309bb640 1 -- 172.21.15.67:0/3215892866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55653429b520 msgr2=0x556534658b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:42:39.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.723+0000 7f9e309bb640 1 --2- 172.21.15.67:0/3215892866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55653429b520 0x556534658b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.723+0000 7f9e309bb640 1 -- 172.21.15.67:0/3215892866 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556534532780 con 0x55653427a840 2024-09-21T06:42:39.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.723+0000 7f9e309bb640 1 --2- 172.21.15.67:0/3215892866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55653427a840 0x55653464d0b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c005760 tx=0x7f9e1c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef65fd786433b608 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:42:39.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.723+0000 7f9e14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:42:39.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.723+0000 7f9e14ff9640 1 -- 172.21.15.67:0/3215892866 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c049050 con 0x55653427a840 2024-09-21T06:42:39.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.723+0000 7f9e14ff9640 1 -- 172.21.15.67:0/3215892866 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f9e1c10ba00 con 0x55653427a840 2024-09-21T06:42:39.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.723+0000 7f9e39994740 1 -- 172.21.15.67:0/3215892866 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e1c115070 con 0x55653427a840 2024-09-21T06:42:39.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.723+0000 7f9e14ff9640 1 -- 172.21.15.67:0/3215892866 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9e1c0cd3f0 con 0x55653427a840 2024-09-21T06:42:39.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.723+0000 7f9e39994740 10 client.15702 fetch_fsmap learned FSMap version 21 2024-09-21T06:42:39.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.723+0000 7f9e39994740 1 -- 172.21.15.67:0/3215892866 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556534306b00 con 0x55653427a840 2024-09-21T06:42:39.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.723+0000 7f9e14ff9640 1 -- 172.21.15.67:0/3215892866 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9e1c115070 con 0x55653427a840 2024-09-21T06:42:39.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.724+0000 7f9e14ff9640 1 -- 172.21.15.67:0/3215892866 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f9e1c0c9d10 con 0x55653427a840 2024-09-21T06:42:39.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.724+0000 7f9e39994740 10 client.15702 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:42:39.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.724+0000 7f9e39994740 10 client.15702 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:42:39.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.724+0000 7f9e39994740 20 client.15702 populate_metadata read hostname 'smithi067' 2024-09-21T06:42:39.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.724+0000 7f9e39994740 1 --2- 172.21.15.67:0/3215892866 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x556534259ea0 0x55653465ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:39.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.724+0000 7f9e39994740 4 client.15702 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:42:39.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.724+0000 7f9e39994740 1 -- 172.21.15.67:0/3215892866 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565345bb140 con 0x556534259ea0 2024-09-21T06:42:39.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.724+0000 7f9e311bc640 1 --2- 172.21.15.67:0/3215892866 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x556534259ea0 0x55653465ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:39.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.725+0000 7f9e311bc640 1 --2- 172.21.15.67:0/3215892866 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x556534259ea0 0x55653465ea70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a949d8f439604d5e server_cookie=aacadb5dace6fde8 in_seq=0 out_seq=0 2024-09-21T06:42:39.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.725+0000 7f9e14ff9640 10 client.15702 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:42:39.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.725+0000 7f9e14ff9640 1 -- 172.21.15.67:0/3215892866 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5565345bb140 con 0x556534259ea0 2024-09-21T06:42:39.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.725+0000 7f9e14ff9640 10 client.15702 handle_command_reply: tid=0 2024-09-21T06:42:39.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.725+0000 7f9e39994740 1 client.15702 shutdown 2024-09-21T06:42:39.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.726+0000 7f9e39994740 1 -- 172.21.15.67:0/3215892866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55653427a840 msgr2=0x55653464d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:39.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.726+0000 7f9e39994740 1 --2- 172.21.15.67:0/3215892866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55653427a840 0x55653464d0b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c005760 tx=0x7f9e1c009260 comp rx=0 tx=0).stop 2024-09-21T06:42:39.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.726+0000 7f9e39994740 1 -- 172.21.15.67:0/3215892866 shutdown_connections 2024-09-21T06:42:39.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.726+0000 7f9e39994740 1 --2- 172.21.15.67:0/3215892866 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x556534259ea0 0x55653465ea70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.726+0000 7f9e39994740 1 --2- 172.21.15.67:0/3215892866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55653429b520 0x556534658b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.726+0000 7f9e14ff9640 0 client.15702 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:42:39.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.726+0000 7f9e39994740 1 --2- 172.21.15.67:0/3215892866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55653427a840 0x55653464d0b0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.726+0000 7f9e39994740 1 --2- 172.21.15.67:0/3215892866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556534272660 0x55653464cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.726+0000 7f9e39994740 1 -- 172.21.15.67:0/3215892866 >> 172.21.15.67:0/3215892866 conn(0x55653428f910 msgr2=0x5565346446a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:42:39.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.726+0000 7f9e39994740 1 -- 172.21.15.67:0/3215892866 shutdown_connections 2024-09-21T06:42:39.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.726+0000 7f9e39994740 1 -- 172.21.15.67:0/3215892866 wait complete. 2024-09-21T06:42:39.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.726+0000 7f9e39994740 20 client.15702 trim_cache size 0 max 16384 2024-09-21T06:42:39.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.744+0000 7f9e39994740 1 Processor -- start 2024-09-21T06:42:39.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.744+0000 7f9e39994740 1 -- start start 2024-09-21T06:42:39.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.744+0000 7f9e39994740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556534259ea0 0x556534658b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:39.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.744+0000 7f9e39994740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556534272660 0x556534642f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:39.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.744+0000 7f9e39994740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55653427a840 0x55653464e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:39.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.744+0000 7f9e39994740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c1b0d20 con 0x556534272660 2024-09-21T06:42:39.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.744+0000 7f9e39994740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c1b35a0 con 0x55653427a840 2024-09-21T06:42:39.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.744+0000 7f9e39994740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c1b1390 con 0x556534259ea0 2024-09-21T06:42:39.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.744+0000 7f9e309bb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556534272660 0x556534642f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:39.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.744+0000 7f9e309bb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556534272660 0x556534642f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36420/0 (socket says 172.21.15.67:36420) 2024-09-21T06:42:39.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.744+0000 7f9e309bb640 1 -- 172.21.15.67:0/148076865 learned_addr learned my addr 172.21.15.67:0/148076865 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:42:39.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.745+0000 7f9e319bd640 1 --2- 172.21.15.67:0/148076865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55653427a840 0x55653464e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:39.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.745+0000 7f9e311bc640 1 --2- 172.21.15.67:0/148076865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556534259ea0 0x556534658b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:39.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.745+0000 7f9e309bb640 1 -- 172.21.15.67:0/148076865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556534259ea0 msgr2=0x556534658b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:42:39.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.745+0000 7f9e309bb640 1 --2- 172.21.15.67:0/148076865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556534259ea0 0x556534658b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.745+0000 7f9e309bb640 1 -- 172.21.15.67:0/148076865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55653427a840 msgr2=0x55653464e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:42:39.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.745+0000 7f9e309bb640 1 --2- 172.21.15.67:0/148076865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55653427a840 0x55653464e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.745+0000 7f9e309bb640 1 -- 172.21.15.67:0/148076865 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55653437d960 con 0x556534272660 2024-09-21T06:42:39.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.745+0000 7f9e319bd640 1 --2- 172.21.15.67:0/148076865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55653427a840 0x55653464e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:42:39.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.745+0000 7f9e309bb640 1 --2- 172.21.15.67:0/148076865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556534272660 0x556534642f00 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c0ca070 tx=0x7f9e1c10bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1555d67d17924eab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:42:39.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.745+0000 7f9e16ffd640 1 -- 172.21.15.67:0/148076865 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c049070 con 0x556534272660 2024-09-21T06:42:39.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.745+0000 7f9e16ffd640 1 -- 172.21.15.67:0/148076865 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9e1c115030 con 0x556534272660 2024-09-21T06:42:39.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.745+0000 7f9e16ffd640 1 -- 172.21.15.67:0/148076865 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c0cd5e0 con 0x556534272660 2024-09-21T06:42:39.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.746+0000 7f9e39994740 1 -- 172.21.15.67:0/148076865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556534272660 msgr2=0x556534642f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:39.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.746+0000 7f9e39994740 1 --2- 172.21.15.67:0/148076865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556534272660 0x556534642f00 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c0ca070 tx=0x7f9e1c10bf70 comp rx=0 tx=0).stop 2024-09-21T06:42:39.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.746+0000 7f9e39994740 1 -- 172.21.15.67:0/148076865 shutdown_connections 2024-09-21T06:42:39.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.746+0000 7f9e39994740 1 --2- 172.21.15.67:0/148076865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55653427a840 0x55653464e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.746+0000 7f9e39994740 1 --2- 172.21.15.67:0/148076865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556534272660 0x556534642f00 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.746+0000 7f9e39994740 1 --2- 172.21.15.67:0/148076865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556534259ea0 0x556534658b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.746+0000 7f9e39994740 1 -- 172.21.15.67:0/148076865 >> 172.21.15.67:0/148076865 conn(0x55653428f910 msgr2=0x55653464e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:42:39.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.746+0000 7f9e39994740 1 -- 172.21.15.67:0/148076865 shutdown_connections 2024-09-21T06:42:39.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.746+0000 7f9e39994740 1 -- 172.21.15.67:0/148076865 wait complete. 2024-09-21T06:42:39.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.746+0000 7f9e39994740 1 -- start start 2024-09-21T06:42:39.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.747+0000 7f9e39994740 1 Processor -- start 2024-09-21T06:42:39.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.747+0000 7f9e39994740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556534259ea0 0x5565346bd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:39.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.747+0000 7f9e39994740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556534272660 0x5565346bd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:39.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.747+0000 7f9e39994740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55653427a840 0x5565346c8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:39.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.747+0000 7f9e39994740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c1b1a60 con 0x556534259ea0 2024-09-21T06:42:39.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.747+0000 7f9e39994740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c1b2ed0 con 0x55653427a840 2024-09-21T06:42:39.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.747+0000 7f9e39994740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c1b2130 con 0x556534272660 2024-09-21T06:42:39.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.747+0000 7f9e39994740 10 client.0 _ll_register_callbacks cb 0x7f9e1c10bd00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:42:39.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.747+0000 7f9e311bc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556534259ea0 0x5565346bd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:39.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.747+0000 7f9e311bc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556534259ea0 0x5565346bd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36422/0 (socket says 172.21.15.67:36422) 2024-09-21T06:42:39.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.747+0000 7f9e311bc640 1 -- 172.21.15.67:0/202081022 learned_addr learned my addr 172.21.15.67:0/202081022 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:42:39.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.747+0000 7f9e309bb640 1 --2- 172.21.15.67:0/202081022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556534272660 0x5565346bd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:39.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.747+0000 7f9e319bd640 1 --2- 172.21.15.67:0/202081022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55653427a840 0x5565346c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.748+0000 7f9e311bc640 1 -- 172.21.15.67:0/202081022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556534272660 msgr2=0x5565346bd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:42:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.748+0000 7f9e311bc640 1 --2- 172.21.15.67:0/202081022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556534272660 0x5565346bd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.748+0000 7f9e311bc640 1 -- 172.21.15.67:0/202081022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55653427a840 msgr2=0x5565346c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:42:39.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.748+0000 7f9e311bc640 1 --2- 172.21.15.67:0/202081022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55653427a840 0x5565346c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.748+0000 7f9e311bc640 1 -- 172.21.15.67:0/202081022 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565346bded0 con 0x556534259ea0 2024-09-21T06:42:39.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.748+0000 7f9e311bc640 1 --2- 172.21.15.67:0/202081022 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556534259ea0 0x5565346bd180 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9e28005d00 tx=0x7f9e280054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f55ea0545973a5b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:42:39.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.748+0000 7f9e14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:42:39.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.748+0000 7f9e14ff9640 1 -- 172.21.15.67:0/202081022 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e28004020 con 0x556534259ea0 2024-09-21T06:42:39.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.748+0000 7f9e14ff9640 1 -- 172.21.15.67:0/202081022 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f9e280041c0 con 0x556534259ea0 2024-09-21T06:42:39.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.748+0000 7f9e39994740 1 -- 172.21.15.67:0/202081022 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e1c115030 con 0x556534259ea0 2024-09-21T06:42:39.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.748+0000 7f9e14ff9640 1 -- 172.21.15.67:0/202081022 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9e280043c0 con 0x556534259ea0 2024-09-21T06:42:39.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.748+0000 7f9e39994740 10 client.15708 fetch_fsmap learned FSMap version 21 2024-09-21T06:42:39.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.748+0000 7f9e39994740 1 -- 172.21.15.67:0/202081022 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5565346c97d0 con 0x556534259ea0 2024-09-21T06:42:39.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.748+0000 7f9e14ff9640 1 -- 172.21.15.67:0/202081022 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9e1c115030 con 0x556534259ea0 2024-09-21T06:42:39.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.749+0000 7f9e14ff9640 1 -- 172.21.15.67:0/202081022 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f9e2800f040 con 0x556534259ea0 2024-09-21T06:42:39.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.749+0000 7f9e39994740 10 client.15708 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:42:39.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.749+0000 7f9e39994740 10 client.15708 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:42:39.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.749+0000 7f9e39994740 20 client.15708 populate_metadata read hostname 'smithi067' 2024-09-21T06:42:39.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.749+0000 7f9e39994740 1 --2- 172.21.15.67:0/202081022 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55653429b520 0x5565346cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:39.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.749+0000 7f9e39994740 4 client.15708 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "79dfa6b2", "release": true, "await": true}] 2024-09-21T06:42:39.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.749+0000 7f9e39994740 1 -- 172.21.15.67:0/202081022 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "quiesce db", "set_id": "79dfa6b2", "release": true, "await": true}) -- 0x5565345b6bf0 con 0x55653429b520 2024-09-21T06:42:39.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.749+0000 7f9e309bb640 1 --2- 172.21.15.67:0/202081022 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55653429b520 0x5565346cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:39.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.749+0000 7f9e309bb640 1 --2- 172.21.15.67:0/202081022 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55653429b520 0x5565346cbfc0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89535c70da68a26c server_cookie=751930d3bd4659a4 in_seq=0 out_seq=0 2024-09-21T06:42:39.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.749+0000 7f9e14ff9640 10 client.15708 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:42:39.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.755+0000 7f9e14ff9640 1 -- 172.21.15.67:0/202081022 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5565345b6bf0 con 0x55653429b520 2024-09-21T06:42:39.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.755+0000 7f9e14ff9640 10 client.15708 handle_command_reply: tid=0 2024-09-21T06:42:39.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.755+0000 7f9e39994740 1 client.15708 shutdown 2024-09-21T06:42:39.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.755+0000 7f9e39994740 1 -- 172.21.15.67:0/202081022 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556534259ea0 msgr2=0x5565346bd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:39.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.755+0000 7f9e39994740 1 --2- 172.21.15.67:0/202081022 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556534259ea0 0x5565346bd180 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9e28005d00 tx=0x7f9e280054d0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.755+0000 7f9e39994740 1 -- 172.21.15.67:0/202081022 shutdown_connections 2024-09-21T06:42:39.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.755+0000 7f9e39994740 1 --2- 172.21.15.67:0/202081022 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55653429b520 0x5565346cbfc0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.755+0000 7f9e14ff9640 0 client.15708 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:42:39.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.755+0000 7f9e39994740 1 --2- 172.21.15.67:0/202081022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55653427a840 0x5565346c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.755+0000 7f9e39994740 1 --2- 172.21.15.67:0/202081022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556534272660 0x5565346bd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.755+0000 7f9e39994740 1 --2- 172.21.15.67:0/202081022 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556534259ea0 0x5565346bd180 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.755+0000 7f9e39994740 1 -- 172.21.15.67:0/202081022 >> 172.21.15.67:0/202081022 conn(0x55653428f910 msgr2=0x55653464dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:42:39.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.755+0000 7f9e39994740 1 -- 172.21.15.67:0/202081022 shutdown_connections 2024-09-21T06:42:39.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.755+0000 7f9e39994740 1 -- 172.21.15.67:0/202081022 wait complete. 2024-09-21T06:42:39.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.755+0000 7f9e39994740 20 client.15708 trim_cache size 0 max 16384 2024-09-21T06:42:39.761 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-21T06:42:39.761 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 21, 2024-09-21T06:42:39.761 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-09-21T06:42:39.761 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 24, 2024-09-21T06:42:39.761 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-21T06:42:39.761 INFO:teuthology.orchestra.run.smithi067.stdout: "79dfa6b2": { 2024-09-21T06:42:39.761 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 24, 2024-09-21T06:42:39.761 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-21T06:42:39.762 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:42:39.762 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-21T06:42:39.762 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:42:39.762 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:42:39.762 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-21T06:42:39.762 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 139.2, 2024-09-21T06:42:39.762 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-21T06:42:39.763 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-21T06:42:39.763 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-21T06:42:39.763 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:42:39.763 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-21T06:42:39.763 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:42:39.763 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:42:39.764 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:42:39.764 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:42:39.764 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:42:39.764 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:42:39.764 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-21T06:42:39.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.756+0000 7f9e33420640 1 -- 172.21.15.67:0/3342114595 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9e180b6510 msgr2=0x7f9e180b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:39.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.756+0000 7f9e33420640 1 --2- 172.21.15.67:0/3342114595 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9e180b6510 0x7f9e180b8a20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9e240092f0 tx=0x7f9e24009280 comp rx=0 tx=0).stop 2024-09-21T06:42:39.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.756+0000 7f9e33420640 1 -- 172.21.15.67:0/3342114595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e2c1083d0 msgr2=0x7f9e2c0ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:39.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.756+0000 7f9e33420640 1 --2- 172.21.15.67:0/3342114595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e2c1083d0 0x7f9e2c0ffbb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c005940 tx=0x7f9e1c0042f0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.756+0000 7f9e33420640 1 -- 172.21.15.67:0/3342114595 shutdown_connections 2024-09-21T06:42:39.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.756+0000 7f9e33420640 1 --2- 172.21.15.67:0/3342114595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e2c108d20 0x7f9e2c1000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.756+0000 7f9e33420640 1 --2- 172.21.15.67:0/3342114595 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9e180b6510 0x7f9e180b8a20 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.756+0000 7f9e33420640 1 --2- 172.21.15.67:0/3342114595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e2c1083d0 0x7f9e2c0ffbb0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.756+0000 7f9e33420640 1 --2- 172.21.15.67:0/3342114595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e2c102ae0 0x7f9e2c0ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:39.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.756+0000 7f9e33420640 1 -- 172.21.15.67:0/3342114595 >> 172.21.15.67:0/3342114595 conn(0x7f9e2c079760 msgr2=0x7f9e2c101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:42:39.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.757+0000 7f9e33420640 1 -- 172.21.15.67:0/3342114595 shutdown_connections 2024-09-21T06:42:39.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:39.757+0000 7f9e33420640 1 -- 172.21.15.67:0/3342114595 wait complete. 2024-09-21T06:42:39.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:39 smithi203 ceph-mon[33735]: pgmap v542: 129 pgs: 129 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 24 MiB/s wr, 382 op/s 2024-09-21T06:42:40.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:39 smithi100 ceph-mon[33401]: pgmap v542: 129 pgs: 129 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 24 MiB/s wr, 382 op/s 2024-09-21T06:42:40.634 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '79dfa6b2', total seconds elapsed: 22.9 2024-09-21T06:42:40.634 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.25 2024-09-21T06:42:40.634 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:42:41.227 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:42:41.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:41 smithi067 ceph-mon[31232]: pgmap v543: 129 pgs: 129 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 16 MiB/s wr, 261 op/s 2024-09-21T06:42:41.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.903+0000 7fded0c73640 1 -- 172.21.15.67:0/598678976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdecc111cd0 msgr2=0x7fdecc10d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:41.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.903+0000 7fded0c73640 1 --2- 172.21.15.67:0/598678976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdecc111cd0 0x7fdecc10d6b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdebc009900 tx=0x7fdebc02a440 comp rx=0 tx=0).stop 2024-09-21T06:42:41.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.904+0000 7fded0c73640 1 -- 172.21.15.67:0/598678976 shutdown_connections 2024-09-21T06:42:41.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.904+0000 7fded0c73640 1 --2- 172.21.15.67:0/598678976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdecc11cba0 0x7fdecc11efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:41.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.904+0000 7fded0c73640 1 --2- 172.21.15.67:0/598678976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdecc10dbf0 0x7fdecc10e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:41.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.904+0000 7fded0c73640 1 --2- 172.21.15.67:0/598678976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdecc111cd0 0x7fdecc10d6b0 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdebc009900 tx=0x7fdebc02a440 comp rx=0 tx=0).stop 2024-09-21T06:42:41.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.904+0000 7fded0c73640 1 -- 172.21.15.67:0/598678976 >> 172.21.15.67:0/598678976 conn(0x7fdecc06e9f0 msgr2=0x7fdecc06ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:42:41.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.907+0000 7fded0c73640 1 -- 172.21.15.67:0/598678976 shutdown_connections 2024-09-21T06:42:41.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.909+0000 7fded0c73640 1 -- 172.21.15.67:0/598678976 wait complete. 2024-09-21T06:42:41.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.910+0000 7fded0c73640 1 Processor -- start 2024-09-21T06:42:41.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.910+0000 7fded0c73640 1 -- start start 2024-09-21T06:42:41.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.910+0000 7fded0c73640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdecc11cba0 0x7fdecc1a9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:41.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.910+0000 7fded0c73640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdecc1a9950 0x7fdecc1aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:41.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.910+0000 7fded0c73640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdecc1af0d0 0x7fdecc1af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:41.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.910+0000 7fded0c73640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc1226c0 con 0x7fdecc1af0d0 2024-09-21T06:42:41.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.910+0000 7fded0c73640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc122540 con 0x7fdecc1a9950 2024-09-21T06:42:41.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.910+0000 7fded0c73640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc122840 con 0x7fdecc11cba0 2024-09-21T06:42:41.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.911+0000 7fdecb7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdecc11cba0 0x7fdecc1a9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:41.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.911+0000 7fdecbfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdecc1af0d0 0x7fdecc1af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:41.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.911+0000 7fdecb7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdecc11cba0 0x7fdecc1a9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.67:49252/0 (socket says 172.21.15.67:49252) 2024-09-21T06:42:41.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.911+0000 7fdecb7fe640 1 -- 172.21.15.67:0/4285989404 learned_addr learned my addr 172.21.15.67:0/4285989404 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:42:41.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.911+0000 7fdecaffd640 1 --2- 172.21.15.67:0/4285989404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdecc1a9950 0x7fdecc1aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:41.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.913+0000 7fdecaffd640 1 -- 172.21.15.67:0/4285989404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdecc11cba0 msgr2=0x7fdecc1a9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:41.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.913+0000 7fdecaffd640 1 --2- 172.21.15.67:0/4285989404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdecc11cba0 0x7fdecc1a9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:41.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.913+0000 7fdecaffd640 1 -- 172.21.15.67:0/4285989404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdecc1af0d0 msgr2=0x7fdecc1af4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:41.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.913+0000 7fdecaffd640 1 --2- 172.21.15.67:0/4285989404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdecc1af0d0 0x7fdecc1af4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:41.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.913+0000 7fdecaffd640 1 -- 172.21.15.67:0/4285989404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdecc112400 con 0x7fdecc1a9950 2024-09-21T06:42:41.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.913+0000 7fdecaffd640 1 --2- 172.21.15.67:0/4285989404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdecc1a9950 0x7fdecc1aeb90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdecc06c680 tx=0x7fdec000b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:42:41.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.913+0000 7fdec8ff9640 1 -- 172.21.15.67:0/4285989404 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec0015020 con 0x7fdecc1a9950 2024-09-21T06:42:41.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.913+0000 7fdec8ff9640 1 -- 172.21.15.67:0/4285989404 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdec0011cc0 con 0x7fdecc1a9950 2024-09-21T06:42:41.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.914+0000 7fdec8ff9640 1 -- 172.21.15.67:0/4285989404 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec0010cd0 con 0x7fdecc1a9950 2024-09-21T06:42:41.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.914+0000 7fded0c73640 1 -- 172.21.15.67:0/4285989404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdecc1126f0 con 0x7fdecc1a9950 2024-09-21T06:42:41.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.914+0000 7fded0c73640 1 -- 172.21.15.67:0/4285989404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdecc112c60 con 0x7fdecc1a9950 2024-09-21T06:42:41.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.917+0000 7fded0c73640 1 -- 172.21.15.67:0/4285989404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdecc10e000 con 0x7fdecc1a9950 2024-09-21T06:42:41.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.918+0000 7fdec8ff9640 1 -- 172.21.15.67:0/4285989404 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fdec0018020 con 0x7fdecc1a9950 2024-09-21T06:42:41.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.920+0000 7fdec8ff9640 1 --2- 172.21.15.67:0/4285989404 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fdeb40b6650 0x7fdeb40b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:41.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.920+0000 7fdec8ff9640 1 -- 172.21.15.67:0/4285989404 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fdec00e6cc0 con 0x7fdecc1a9950 2024-09-21T06:42:41.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.921+0000 7fdecb7fe640 1 --2- 172.21.15.67:0/4285989404 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fdeb40b6650 0x7fdeb40b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:41.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.926+0000 7fdec8ff9640 1 -- 172.21.15.67:0/4285989404 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdec001f020 con 0x7fdecc1a9950 2024-09-21T06:42:41.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:41.926+0000 7fdecb7fe640 1 --2- 172.21.15.67:0/4285989404 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fdeb40b6650 0x7fdeb40b8b60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdebc0057c0 tx=0x7fdebc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:42:41.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:41 smithi203 ceph-mon[33735]: pgmap v543: 129 pgs: 129 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 16 MiB/s wr, 261 op/s 2024-09-21T06:42:42.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:41 smithi100 ceph-mon[33401]: pgmap v543: 129 pgs: 129 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 16 MiB/s wr, 261 op/s 2024-09-21T06:42:42.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:42.385+0000 7fded0c73640 1 -- 172.21.15.67:0/4285989404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdecc112ff0 con 0x7fdecc1a9950 2024-09-21T06:42:42.391 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:42:42.391 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:42:42.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:42.386+0000 7fdec8ff9640 1 -- 172.21.15.67:0/4285989404 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7fdec00ad120 con 0x7fdecc1a9950 2024-09-21T06:42:42.392 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:42:42.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:42.393+0000 7fded0c73640 1 -- 172.21.15.67:0/4285989404 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fdeb40b6650 msgr2=0x7fdeb40b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:42.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:42.393+0000 7fded0c73640 1 --2- 172.21.15.67:0/4285989404 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fdeb40b6650 0x7fdeb40b8b60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdebc0057c0 tx=0x7fdebc005fd0 comp rx=0 tx=0).stop 2024-09-21T06:42:42.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:42.393+0000 7fded0c73640 1 -- 172.21.15.67:0/4285989404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdecc1a9950 msgr2=0x7fdecc1aeb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:42.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:42.393+0000 7fded0c73640 1 --2- 172.21.15.67:0/4285989404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdecc1a9950 0x7fdecc1aeb90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdecc06c680 tx=0x7fdec000b870 comp rx=0 tx=0).stop 2024-09-21T06:42:42.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:42.394+0000 7fded0c73640 1 -- 172.21.15.67:0/4285989404 shutdown_connections 2024-09-21T06:42:42.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:42.394+0000 7fded0c73640 1 --2- 172.21.15.67:0/4285989404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdecc1a9950 0x7fdecc1aeb90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:42.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:42.394+0000 7fded0c73640 1 --2- 172.21.15.67:0/4285989404 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fdeb40b6650 0x7fdeb40b8b60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:42.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:42.394+0000 7fded0c73640 1 --2- 172.21.15.67:0/4285989404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdecc1af0d0 0x7fdecc1af4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:42.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:42.394+0000 7fded0c73640 1 --2- 172.21.15.67:0/4285989404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdecc11cba0 0x7fdecc1a9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:42.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:42.394+0000 7fded0c73640 1 -- 172.21.15.67:0/4285989404 >> 172.21.15.67:0/4285989404 conn(0x7fdecc06e9f0 msgr2=0x7fdecc10cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:42:42.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:42.394+0000 7fded0c73640 1 -- 172.21.15.67:0/4285989404 shutdown_connections 2024-09-21T06:42:42.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:42.394+0000 7fded0c73640 1 -- 172.21.15.67:0/4285989404 wait complete. 2024-09-21T06:42:42.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:42 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/4285989404' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:42:42.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:42 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/4285989404' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:42:43.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:42 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/4285989404' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:42:43.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:43 smithi067 ceph-mon[31232]: pgmap v544: 129 pgs: 129 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 19 MiB/s wr, 300 op/s 2024-09-21T06:42:43.963 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db / --timeout 90 --expiration 139.25 --await-for 45.0' 2024-09-21T06:42:43.963 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24671 quiesce db / --timeout 90 --expiration 139.25 --await-for 45.0 2024-09-21T06:42:43.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:43 smithi203 ceph-mon[33735]: pgmap v544: 129 pgs: 129 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 19 MiB/s wr, 300 op/s 2024-09-21T06:42:44.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:43 smithi100 ceph-mon[33401]: pgmap v544: 129 pgs: 129 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 19 MiB/s wr, 300 op/s 2024-09-21T06:42:44.506 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:42:45.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.731+0000 7f7acf2d0640 1 -- 172.21.15.67:0/45643424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ac810a190 msgr2=0x7f7ac810a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:45.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.731+0000 7f7acf2d0640 1 --2- 172.21.15.67:0/45643424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ac810a190 0x7f7ac810a5a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7ac401a910 tx=0x7f7ac403b450 comp rx=0 tx=0).stop 2024-09-21T06:42:45.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.732+0000 7f7acf2d0640 1 -- 172.21.15.67:0/45643424 shutdown_connections 2024-09-21T06:42:45.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.732+0000 7f7acf2d0640 1 --2- 172.21.15.67:0/45643424 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ac8110420 0x7f7ac8116c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.732+0000 7f7acf2d0640 1 --2- 172.21.15.67:0/45643424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ac810fad0 0x7f7ac810fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.732+0000 7f7acf2d0640 1 --2- 172.21.15.67:0/45643424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ac810a190 0x7f7ac810a5a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.732+0000 7f7acf2d0640 1 -- 172.21.15.67:0/45643424 >> 172.21.15.67:0/45643424 conn(0x7f7ac80781c0 msgr2=0x7f7ac81081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:42:45.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.734+0000 7f7acf2d0640 1 -- 172.21.15.67:0/45643424 shutdown_connections 2024-09-21T06:42:45.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.734+0000 7f7acf2d0640 1 -- 172.21.15.67:0/45643424 wait complete. 2024-09-21T06:42:45.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.735+0000 7f7acf2d0640 1 Processor -- start 2024-09-21T06:42:45.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.738+0000 7f7acf2d0640 1 -- start start 2024-09-21T06:42:45.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.738+0000 7f7acf2d0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ac810a190 0x7f7ac8116960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:45.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.738+0000 7f7acf2d0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ac810fad0 0x7f7ac8112c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:45.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.739+0000 7f7acd06c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ac810a190 0x7f7ac8116960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:45.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.739+0000 7f7acd06c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ac810a190 0x7f7ac8116960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.67:49268/0 (socket says 172.21.15.67:49268) 2024-09-21T06:42:45.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.739+0000 7f7acd06c640 1 -- 172.21.15.67:0/734437681 learned_addr learned my addr 172.21.15.67:0/734437681 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:42:45.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.739+0000 7f7acf2d0640 1 --2- 172.21.15.67:0/734437681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ac8110420 0x7f7ac8083c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:45.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.739+0000 7f7acf2d0640 1 -- 172.21.15.67:0/734437681 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac806f5a0 con 0x7f7ac810fad0 2024-09-21T06:42:45.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.739+0000 7f7acf2d0640 1 -- 172.21.15.67:0/734437681 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac806f420 con 0x7f7ac8110420 2024-09-21T06:42:45.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.739+0000 7f7acf2d0640 1 -- 172.21.15.67:0/734437681 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac806f720 con 0x7f7ac810a190 2024-09-21T06:42:45.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.739+0000 7f7acd86d640 1 --2- 172.21.15.67:0/734437681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ac8110420 0x7f7ac8083c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:45.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.739+0000 7f7acc86b640 1 --2- 172.21.15.67:0/734437681 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ac810fad0 0x7f7ac8112c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:45.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.740+0000 7f7acd06c640 1 -- 172.21.15.67:0/734437681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ac8110420 msgr2=0x7f7ac8083c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:45.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.740+0000 7f7acd06c640 1 --2- 172.21.15.67:0/734437681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ac8110420 0x7f7ac8083c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.740+0000 7f7acd06c640 1 -- 172.21.15.67:0/734437681 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ac810fad0 msgr2=0x7f7ac8112c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:45.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.740+0000 7f7acd06c640 1 --2- 172.21.15.67:0/734437681 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ac810fad0 0x7f7ac8112c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.740+0000 7f7acd06c640 1 -- 172.21.15.67:0/734437681 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac8113180 con 0x7f7ac810a190 2024-09-21T06:42:45.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.741+0000 7f7acd06c640 1 --2- 172.21.15.67:0/734437681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ac810a190 0x7f7ac8116960 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7ac403ba40 tx=0x7f7ac4005000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:42:45.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.743+0000 7f7abe7fc640 1 -- 172.21.15.67:0/734437681 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac404f020 con 0x7f7ac810a190 2024-09-21T06:42:45.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.743+0000 7f7abe7fc640 1 -- 172.21.15.67:0/734437681 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ac40035f0 con 0x7f7ac810a190 2024-09-21T06:42:45.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.743+0000 7f7abe7fc640 1 -- 172.21.15.67:0/734437681 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4046070 con 0x7f7ac810a190 2024-09-21T06:42:45.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.743+0000 7f7acf2d0640 1 -- 172.21.15.67:0/734437681 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ac8113410 con 0x7f7ac810a190 2024-09-21T06:42:45.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.743+0000 7f7acf2d0640 1 -- 172.21.15.67:0/734437681 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ac8113920 con 0x7f7ac810a190 2024-09-21T06:42:45.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.747+0000 7f7abe7fc640 1 -- 172.21.15.67:0/734437681 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f7ac4052020 con 0x7f7ac810a190 2024-09-21T06:42:45.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.753+0000 7f7abe7fc640 1 --2- 172.21.15.67:0/734437681 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f7ab40b6660 0x7f7ab40b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:45.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.755+0000 7f7acc86b640 1 --2- 172.21.15.67:0/734437681 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f7ab40b6660 0x7f7ab40b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:45.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.756+0000 7f7ad5844740 1 Processor -- start 2024-09-21T06:42:45.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.756+0000 7f7abe7fc640 1 -- 172.21.15.67:0/734437681 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f7ac4005440 con 0x7f7ac810a190 2024-09-21T06:42:45.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.757+0000 7f7ad5844740 1 -- start start 2024-09-21T06:42:45.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.757+0000 7f7ad5844740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555729c83850 0x55572a029e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:45.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.757+0000 7f7ad5844740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555729c5a990 0x55572a02a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:45.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.758+0000 7f7acd06c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555729c83850 0x55572a029e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:45.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.758+0000 7f7acd06c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555729c83850 0x55572a029e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.67:49276/0 (socket says 172.21.15.67:49276) 2024-09-21T06:42:45.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.758+0000 7f7acd06c640 1 -- 172.21.15.67:0/819556530 learned_addr learned my addr 172.21.15.67:0/819556530 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:42:45.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.758+0000 7f7ad5844740 1 --2- 172.21.15.67:0/819556530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555729c62b70 0x55572a02a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:45.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.758+0000 7f7ad5844740 1 -- 172.21.15.67:0/819556530 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x555729fa0bb0 con 0x555729c5a990 2024-09-21T06:42:45.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.758+0000 7f7ad5844740 1 -- 172.21.15.67:0/819556530 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555729f8ffe0 con 0x555729c62b70 2024-09-21T06:42:45.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.758+0000 7f7ad5844740 1 -- 172.21.15.67:0/819556530 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x555729f93df0 con 0x555729c83850 2024-09-21T06:42:45.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.758+0000 7f7acc86b640 1 --2- 172.21.15.67:0/734437681 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f7ab40b6660 0x7f7ab40b8b70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7ac810f820 tx=0x7f7ab8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:42:45.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.758+0000 7f7acc86b640 1 --2- 172.21.15.67:0/819556530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555729c5a990 0x55572a02a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:45.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.759+0000 7f7acd06c640 1 -- 172.21.15.67:0/819556530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555729c62b70 msgr2=0x55572a02a8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:42:45.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.759+0000 7f7acd06c640 1 --2- 172.21.15.67:0/819556530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555729c62b70 0x55572a02a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.759+0000 7f7acd06c640 1 -- 172.21.15.67:0/819556530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555729c5a990 msgr2=0x55572a02a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:42:45.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.759+0000 7f7acd06c640 1 --2- 172.21.15.67:0/819556530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555729c5a990 0x55572a02a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.759+0000 7f7acd06c640 1 -- 172.21.15.67:0/819556530 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555729cfdba0 con 0x555729c83850 2024-09-21T06:42:45.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.759+0000 7f7acd06c640 1 --2- 172.21.15.67:0/819556530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555729c83850 0x55572a029e70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40de540 tx=0x7f7ac40de190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bad9ac5a6620ee83 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:42:45.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.760+0000 7f7ab37fe640 1 -- 172.21.15.67:0/819556530 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac40def40 con 0x555729c83850 2024-09-21T06:42:45.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.760+0000 7f7ab37fe640 1 -- 172.21.15.67:0/819556530 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ac40e3320 con 0x555729c83850 2024-09-21T06:42:45.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.760+0000 7f7ab37fe640 1 -- 172.21.15.67:0/819556530 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4003a60 con 0x555729c83850 2024-09-21T06:42:45.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.761+0000 7f7ad5844740 1 -- 172.21.15.67:0/819556530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555729c83850 msgr2=0x55572a029e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:45.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.761+0000 7f7ad5844740 1 --2- 172.21.15.67:0/819556530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555729c83850 0x55572a029e70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40de540 tx=0x7f7ac40de190 comp rx=0 tx=0).stop 2024-09-21T06:42:45.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.762+0000 7f7ad5844740 1 -- 172.21.15.67:0/819556530 shutdown_connections 2024-09-21T06:42:45.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.763+0000 7f7ad5844740 1 --2- 172.21.15.67:0/819556530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555729c62b70 0x55572a02a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.763+0000 7f7ad5844740 1 --2- 172.21.15.67:0/819556530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555729c5a990 0x55572a02a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.763+0000 7f7ad5844740 1 --2- 172.21.15.67:0/819556530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555729c83850 0x55572a029e70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.763+0000 7f7ad5844740 1 -- 172.21.15.67:0/819556530 >> 172.21.15.67:0/819556530 conn(0x555729c77c40 msgr2=0x55572a027d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:42:45.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.763+0000 7f7ad5844740 1 -- 172.21.15.67:0/819556530 shutdown_connections 2024-09-21T06:42:45.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.764+0000 7f7ad5844740 1 -- 172.21.15.67:0/819556530 wait complete. 2024-09-21T06:42:45.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.765+0000 7f7ad5844740 1 -- start start 2024-09-21T06:42:45.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.765+0000 7f7ad5844740 1 Processor -- start 2024-09-21T06:42:45.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.766+0000 7f7ad5844740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555729c5a990 0x55572a035a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:45.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.766+0000 7f7ad5844740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555729c62b70 0x55572a035fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:45.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.766+0000 7f7acd06c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555729c5a990 0x55572a035a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:45.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.766+0000 7f7acd06c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555729c5a990 0x55572a035a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.67:49278/0 (socket says 172.21.15.67:49278) 2024-09-21T06:42:45.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.766+0000 7f7acd06c640 1 -- 172.21.15.67:0/2902625226 learned_addr learned my addr 172.21.15.67:0/2902625226 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:42:45.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.766+0000 7f7ad5844740 1 --2- 172.21.15.67:0/2902625226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555729c83850 0x55572a041670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:45.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.767+0000 7f7ad5844740 1 -- 172.21.15.67:0/2902625226 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x555729df17b0 con 0x555729c62b70 2024-09-21T06:42:45.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.767+0000 7f7ad5844740 1 -- 172.21.15.67:0/2902625226 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555729f2ae90 con 0x555729c83850 2024-09-21T06:42:45.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.767+0000 7f7acc86b640 1 --2- 172.21.15.67:0/2902625226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555729c62b70 0x55572a035fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:45.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.767+0000 7f7acd86d640 1 --2- 172.21.15.67:0/2902625226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555729c83850 0x55572a041670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:45.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.767+0000 7f7ad5844740 1 -- 172.21.15.67:0/2902625226 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55572a038410 con 0x555729c5a990 2024-09-21T06:42:45.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.767+0000 7f7ad5844740 10 client.0 _ll_register_callbacks cb 0x555729f8d710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:42:45.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.767+0000 7f7acd06c640 1 -- 172.21.15.67:0/2902625226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555729c83850 msgr2=0x55572a041670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:42:45.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.767+0000 7f7acd06c640 1 --2- 172.21.15.67:0/2902625226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555729c83850 0x55572a041670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.768+0000 7f7acd06c640 1 -- 172.21.15.67:0/2902625226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555729c62b70 msgr2=0x55572a035fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:42:45.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.768+0000 7f7acd06c640 1 --2- 172.21.15.67:0/2902625226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555729c62b70 0x55572a035fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.768+0000 7f7acd06c640 1 -- 172.21.15.67:0/2902625226 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55572a041c80 con 0x555729c5a990 2024-09-21T06:42:45.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.768+0000 7f7ad5844740 1 -- 172.21.15.67:0/2902625226 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ac40e3320 con 0x555729c5a990 2024-09-21T06:42:45.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.768+0000 7f7acd06c640 1 --2- 172.21.15.67:0/2902625226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555729c5a990 0x55572a035a60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40e8290 tx=0x7f7ac40e82c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=396970168cd6a831 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:42:45.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.769+0000 7f7ab0ff9640 10 client.25396 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T06:42:45.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.769+0000 7f7ab0ff9640 1 -- 172.21.15.67:0/2902625226 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac40daab0 con 0x555729c5a990 2024-09-21T06:42:45.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.769+0000 7f7ab0ff9640 1 -- 172.21.15.67:0/2902625226 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f7ac4003a60 con 0x555729c5a990 2024-09-21T06:42:45.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.769+0000 7f7ab0ff9640 1 -- 172.21.15.67:0/2902625226 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7ac40dac50 con 0x555729c5a990 2024-09-21T06:42:45.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.769+0000 7f7ad5844740 10 client.25396 fetch_fsmap learned FSMap version 21 2024-09-21T06:42:45.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.769+0000 7f7ad5844740 1 -- 172.21.15.67:0/2902625226 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({fsmap=21}) -- 0x55572a044410 con 0x555729c5a990 2024-09-21T06:42:45.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.771+0000 7f7ab0ff9640 1 -- 172.21.15.67:0/2902625226 <== mon.1 v2:172.21.15.203:3300/0 4 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f7ac40dfb70 con 0x555729c5a990 2024-09-21T06:42:45.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.772+0000 7f7ad5844740 10 client.25396 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:42:45.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.772+0000 7f7ad5844740 10 client.25396 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:42:45.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.772+0000 7f7ad5844740 20 client.25396 populate_metadata read hostname 'smithi067' 2024-09-21T06:42:45.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.772+0000 7f7ad5844740 1 --2- 172.21.15.67:0/2902625226 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x555729c421d0 0x55572a0475a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:45.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.772+0000 7f7ad5844740 4 client.25396 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:42:45.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.772+0000 7f7ad5844740 1 -- 172.21.15.67:0/2902625226 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7ac40e3320 con 0x555729c421d0 2024-09-21T06:42:45.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.773+0000 7f7acc86b640 1 --2- 172.21.15.67:0/2902625226 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x555729c421d0 0x55572a0475a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:45.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.774+0000 7f7acc86b640 1 --2- 172.21.15.67:0/2902625226 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x555729c421d0 0x55572a0475a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfc7a65f99dc09a9 server_cookie=870af3eb23fe18fe in_seq=0 out_seq=0 2024-09-21T06:42:45.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.774+0000 7f7ab0ff9640 10 client.25396 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:42:45.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.776+0000 7f7ab0ff9640 1 -- 172.21.15.67:0/2902625226 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f7ac40e3320 con 0x555729c421d0 2024-09-21T06:42:45.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.776+0000 7f7ab0ff9640 10 client.25396 handle_command_reply: tid=0 2024-09-21T06:42:45.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.776+0000 7f7ad5844740 1 client.25396 shutdown 2024-09-21T06:42:45.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.779+0000 7f7ad5844740 1 -- 172.21.15.67:0/2902625226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555729c5a990 msgr2=0x55572a035a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:45.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.779+0000 7f7ad5844740 1 --2- 172.21.15.67:0/2902625226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555729c5a990 0x55572a035a60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40e8290 tx=0x7f7ac40e82c0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.781+0000 7f7ad5844740 1 -- 172.21.15.67:0/2902625226 shutdown_connections 2024-09-21T06:42:45.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.781+0000 7f7ad5844740 1 --2- 172.21.15.67:0/2902625226 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x555729c421d0 0x55572a0475a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.781+0000 7f7ad5844740 1 --2- 172.21.15.67:0/2902625226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555729c83850 0x55572a041670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.781+0000 7f7ad5844740 1 --2- 172.21.15.67:0/2902625226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555729c62b70 0x55572a035fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.781+0000 7f7ad5844740 1 --2- 172.21.15.67:0/2902625226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555729c5a990 0x55572a035a60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.781+0000 7f7ab0ff9640 0 client.25396 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:42:45.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.781+0000 7f7ad5844740 1 -- 172.21.15.67:0/2902625226 >> 172.21.15.67:0/2902625226 conn(0x555729c77c40 msgr2=0x55572a02d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:42:45.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.781+0000 7f7ad5844740 1 -- 172.21.15.67:0/2902625226 shutdown_connections 2024-09-21T06:42:45.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.784+0000 7f7ad5844740 1 -- 172.21.15.67:0/2902625226 wait complete. 2024-09-21T06:42:45.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.784+0000 7f7ad5844740 20 client.25396 trim_cache size 0 max 16384 2024-09-21T06:42:45.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.819+0000 7f7ad5844740 1 Processor -- start 2024-09-21T06:42:45.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.819+0000 7f7ad5844740 1 -- start start 2024-09-21T06:42:45.823 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:45 smithi067 ceph-mon[31232]: pgmap v545: 129 pgs: 129 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 9.9 MiB/s wr, 160 op/s 2024-09-21T06:42:45.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.819+0000 7f7ad5844740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555729c5a990 0x55572a0414b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:45.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.820+0000 7f7ad5844740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555729c62b70 0x55572a037490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:45.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.820+0000 7f7acd06c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555729c5a990 0x55572a0414b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:45.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.820+0000 7f7acd06c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555729c5a990 0x55572a0414b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.67:49286/0 (socket says 172.21.15.67:49286) 2024-09-21T06:42:45.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.820+0000 7f7acd06c640 1 -- 172.21.15.67:0/1651589531 learned_addr learned my addr 172.21.15.67:0/1651589531 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:42:45.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.820+0000 7f7ad5844740 1 --2- 172.21.15.67:0/1651589531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555729c83850 0x55572a032a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:45.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.820+0000 7f7ad5844740 1 -- 172.21.15.67:0/1651589531 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81b52d0 con 0x555729c83850 2024-09-21T06:42:45.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.820+0000 7f7ad5844740 1 -- 172.21.15.67:0/1651589531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81b7b50 con 0x555729c62b70 2024-09-21T06:42:45.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.820+0000 7f7ad5844740 1 -- 172.21.15.67:0/1651589531 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81b5940 con 0x555729c5a990 2024-09-21T06:42:45.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.821+0000 7f7acd86d640 1 --2- 172.21.15.67:0/1651589531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555729c83850 0x55572a032a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:45.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.821+0000 7f7acd06c640 1 -- 172.21.15.67:0/1651589531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555729c62b70 msgr2=0x55572a037490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:42:45.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.821+0000 7f7acd06c640 1 --2- 172.21.15.67:0/1651589531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555729c62b70 0x55572a037490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.821+0000 7f7acd06c640 1 -- 172.21.15.67:0/1651589531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555729c83850 msgr2=0x55572a032a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:42:45.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.821+0000 7f7acd06c640 1 --2- 172.21.15.67:0/1651589531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555729c83850 0x55572a032a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.821+0000 7f7acd06c640 1 -- 172.21.15.67:0/1651589531 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55572a02c700 con 0x555729c5a990 2024-09-21T06:42:45.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.822+0000 7f7acd06c640 1 --2- 172.21.15.67:0/1651589531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555729c5a990 0x55572a0414b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40de540 tx=0x7f7ac408c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62f2b8f0cece34c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:42:45.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.822+0000 7f7ab2ffd640 1 -- 172.21.15.67:0/1651589531 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac40e3320 con 0x555729c5a990 2024-09-21T06:42:45.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.822+0000 7f7ab2ffd640 1 -- 172.21.15.67:0/1651589531 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ac40dfb70 con 0x555729c5a990 2024-09-21T06:42:45.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.824+0000 7f7ad5844740 1 -- 172.21.15.67:0/1651589531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555729c5a990 msgr2=0x55572a0414b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:45.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.824+0000 7f7ad5844740 1 --2- 172.21.15.67:0/1651589531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555729c5a990 0x55572a0414b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40de540 tx=0x7f7ac408c000 comp rx=0 tx=0).stop 2024-09-21T06:42:45.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.825+0000 7f7ad5844740 1 -- 172.21.15.67:0/1651589531 shutdown_connections 2024-09-21T06:42:45.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.825+0000 7f7ad5844740 1 --2- 172.21.15.67:0/1651589531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555729c62b70 0x55572a037490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.825+0000 7f7ad5844740 1 --2- 172.21.15.67:0/1651589531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555729c83850 0x55572a032a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.825+0000 7f7ad5844740 1 --2- 172.21.15.67:0/1651589531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555729c5a990 0x55572a0414b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.825+0000 7f7ad5844740 1 -- 172.21.15.67:0/1651589531 >> 172.21.15.67:0/1651589531 conn(0x555729c77c40 msgr2=0x55572a036d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:42:45.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.825+0000 7f7ad5844740 1 -- 172.21.15.67:0/1651589531 shutdown_connections 2024-09-21T06:42:45.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.827+0000 7f7ad5844740 1 -- 172.21.15.67:0/1651589531 wait complete. 2024-09-21T06:42:45.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.827+0000 7f7ad5844740 1 -- start start 2024-09-21T06:42:45.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.827+0000 7f7ad5844740 1 Processor -- start 2024-09-21T06:42:45.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.828+0000 7f7ad5844740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555729c5a990 0x55572a03dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:45.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.828+0000 7f7ad5844740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555729c62b70 0x55572a03a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:45.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.828+0000 7f7ad5844740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555729c83850 0x55572a03a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:45.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.828+0000 7f7ad5844740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81b6010 con 0x555729c5a990 2024-09-21T06:42:45.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.828+0000 7f7ad5844740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81b7480 con 0x555729c83850 2024-09-21T06:42:45.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.828+0000 7f7ad5844740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81b66e0 con 0x555729c62b70 2024-09-21T06:42:45.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.828+0000 7f7ad5844740 10 client.0 _ll_register_callbacks cb 0x7f7ac40c8070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:42:45.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.828+0000 7f7acd86d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555729c83850 0x55572a03a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:45.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.828+0000 7f7acd86d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555729c83850 0x55572a03a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.67:51690/0 (socket says 172.21.15.67:51690) 2024-09-21T06:42:45.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.828+0000 7f7acd86d640 1 -- 172.21.15.67:0/3040754311 learned_addr learned my addr 172.21.15.67:0/3040754311 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:42:45.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.829+0000 7f7acd06c640 1 --2- 172.21.15.67:0/3040754311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555729c5a990 0x55572a03dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:45.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.830+0000 7f7acd86d640 1 -- 172.21.15.67:0/3040754311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555729c62b70 msgr2=0x55572a03a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:42:45.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.830+0000 7f7acd86d640 1 --2- 172.21.15.67:0/3040754311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555729c62b70 0x55572a03a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.830+0000 7f7acd86d640 1 -- 172.21.15.67:0/3040754311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555729c5a990 msgr2=0x55572a03dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:42:45.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.830+0000 7f7acd86d640 1 --2- 172.21.15.67:0/3040754311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555729c5a990 0x55572a03dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.830+0000 7f7acd86d640 1 -- 172.21.15.67:0/3040754311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55572a03ac30 con 0x555729c83850 2024-09-21T06:42:45.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.830+0000 7f7acd86d640 1 --2- 172.21.15.67:0/3040754311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555729c83850 0x55572a03a5c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7ac000be90 tx=0x7f7ac000a7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f06ecc3a3292e8c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:42:45.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.830+0000 7f7ab0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-21T06:42:45.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.830+0000 7f7ab0ff9640 1 -- 172.21.15.67:0/3040754311 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0017620 con 0x555729c83850 2024-09-21T06:42:45.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.830+0000 7f7ab0ff9640 1 -- 172.21.15.67:0/3040754311 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f7ac00177c0 con 0x555729c83850 2024-09-21T06:42:45.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.830+0000 7f7ad5844740 1 -- 172.21.15.67:0/3040754311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555729fa2b00 con 0x555729c83850 2024-09-21T06:42:45.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.832+0000 7f7ab0ff9640 1 -- 172.21.15.67:0/3040754311 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7ac00179c0 con 0x555729c83850 2024-09-21T06:42:45.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.832+0000 7f7ad5844740 10 client.25451 fetch_fsmap learned FSMap version 21 2024-09-21T06:42:45.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.832+0000 7f7ad5844740 1 -- 172.21.15.67:0/3040754311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55572a0b2ca0 con 0x555729c83850 2024-09-21T06:42:45.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.835+0000 7f7ab0ff9640 1 -- 172.21.15.67:0/3040754311 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x555729fa2b00 con 0x555729c83850 2024-09-21T06:42:45.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.835+0000 7f7ab0ff9640 1 -- 172.21.15.67:0/3040754311 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f7ac0016c30 con 0x555729c83850 2024-09-21T06:42:45.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.835+0000 7f7ad5844740 10 client.25451 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:42:45.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.835+0000 7f7ad5844740 10 client.25451 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:42:45.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.835+0000 7f7ad5844740 20 client.25451 populate_metadata read hostname 'smithi067' 2024-09-21T06:42:45.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.835+0000 7f7ad5844740 1 --2- 172.21.15.67:0/3040754311 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x555729c17580 0x55572a0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:42:45.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.835+0000 7f7ad5844740 4 client.25451 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.25, "await_for": 45.0}] 2024-09-21T06:42:45.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.835+0000 7f7ad5844740 1 -- 172.21.15.67:0/3040754311 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.25, "await_for": 45.0}) -- 0x555729f9e5b0 con 0x555729c17580 2024-09-21T06:42:45.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.836+0000 7f7acd06c640 1 --2- 172.21.15.67:0/3040754311 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x555729c17580 0x55572a0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:42:45.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.840+0000 7f7acd06c640 1 --2- 172.21.15.67:0/3040754311 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x555729c17580 0x55572a0b5450 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8b20168da2f9bef server_cookie=fa8821f914c01018 in_seq=0 out_seq=0 2024-09-21T06:42:45.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.840+0000 7f7ab0ff9640 10 client.25451 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:42:45.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.878+0000 7f7ab0ff9640 1 -- 172.21.15.67:0/3040754311 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555729f9e5b0 con 0x555729c17580 2024-09-21T06:42:45.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.878+0000 7f7ab0ff9640 10 client.25451 handle_command_reply: tid=0 2024-09-21T06:42:45.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.878+0000 7f7ad5844740 1 client.25451 shutdown 2024-09-21T06:42:45.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.878+0000 7f7ad5844740 1 -- 172.21.15.67:0/3040754311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555729c83850 msgr2=0x55572a03a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:45.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.878+0000 7f7ad5844740 1 --2- 172.21.15.67:0/3040754311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555729c83850 0x55572a03a5c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7ac000be90 tx=0x7f7ac000a7b0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.878+0000 7f7ad5844740 1 -- 172.21.15.67:0/3040754311 shutdown_connections 2024-09-21T06:42:45.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.878+0000 7f7ad5844740 1 --2- 172.21.15.67:0/3040754311 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x555729c17580 0x55572a0b5450 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.878+0000 7f7ad5844740 1 --2- 172.21.15.67:0/3040754311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555729c83850 0x55572a03a5c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.878+0000 7f7ad5844740 1 --2- 172.21.15.67:0/3040754311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555729c62b70 0x55572a03a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.878+0000 7f7ad5844740 1 --2- 172.21.15.67:0/3040754311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555729c5a990 0x55572a03dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.878+0000 7f7ad5844740 1 -- 172.21.15.67:0/3040754311 >> 172.21.15.67:0/3040754311 conn(0x555729c77c40 msgr2=0x55572a0376a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:42:45.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.878+0000 7f7ab0ff9640 0 client.25451 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:42:45.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.878+0000 7f7ad5844740 1 -- 172.21.15.67:0/3040754311 shutdown_connections 2024-09-21T06:42:45.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.879+0000 7f7ad5844740 1 -- 172.21.15.67:0/3040754311 wait complete. 2024-09-21T06:42:45.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.879+0000 7f7ad5844740 20 client.25451 trim_cache size 0 max 16384 2024-09-21T06:42:45.882 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-21T06:42:45.883 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 21, 2024-09-21T06:42:45.883 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-09-21T06:42:45.883 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 26, 2024-09-21T06:42:45.883 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-21T06:42:45.883 INFO:teuthology.orchestra.run.smithi067.stdout: "5162919c": { 2024-09-21T06:42:45.883 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 26, 2024-09-21T06:42:45.883 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-21T06:42:45.883 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:42:45.883 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-21T06:42:45.883 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:42:45.883 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:42:45.883 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-21T06:42:45.883 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 139.2, 2024-09-21T06:42:45.884 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-21T06:42:45.884 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-21T06:42:45.884 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-21T06:42:45.884 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:42:45.884 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-21T06:42:45.884 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:42:45.884 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:42:45.884 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:42:45.884 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:42:45.885 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:42:45.885 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:42:45.885 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-21T06:42:45.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.879+0000 7f7acf2d0640 1 -- 172.21.15.67:0/734437681 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f7ab40b6660 msgr2=0x7f7ab40b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:45.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.879+0000 7f7acf2d0640 1 --2- 172.21.15.67:0/734437681 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f7ab40b6660 0x7f7ab40b8b70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7ac810f820 tx=0x7f7ab8005f90 comp rx=0 tx=0).stop 2024-09-21T06:42:45.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.879+0000 7f7acf2d0640 1 -- 172.21.15.67:0/734437681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ac810a190 msgr2=0x7f7ac8116960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:42:45.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.879+0000 7f7acf2d0640 1 --2- 172.21.15.67:0/734437681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ac810a190 0x7f7ac8116960 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7ac403ba40 tx=0x7f7ac4005000 comp rx=0 tx=0).stop 2024-09-21T06:42:45.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.879+0000 7f7acf2d0640 1 -- 172.21.15.67:0/734437681 shutdown_connections 2024-09-21T06:42:45.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.879+0000 7f7acf2d0640 1 --2- 172.21.15.67:0/734437681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ac8110420 0x7f7ac8083c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.879+0000 7f7acf2d0640 1 --2- 172.21.15.67:0/734437681 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f7ab40b6660 0x7f7ab40b8b70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.879+0000 7f7acf2d0640 1 --2- 172.21.15.67:0/734437681 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ac810fad0 0x7f7ac8112c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.879+0000 7f7acf2d0640 1 --2- 172.21.15.67:0/734437681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ac810a190 0x7f7ac8116960 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:42:45.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.879+0000 7f7acf2d0640 1 -- 172.21.15.67:0/734437681 >> 172.21.15.67:0/734437681 conn(0x7f7ac80781c0 msgr2=0x7f7ac8108660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:42:45.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.880+0000 7f7acf2d0640 1 -- 172.21.15.67:0/734437681 shutdown_connections 2024-09-21T06:42:45.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:42:45.880+0000 7f7acf2d0640 1 -- 172.21.15.67:0/734437681 wait complete. 2024-09-21T06:42:45.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:45 smithi203 ceph-mon[33735]: pgmap v545: 129 pgs: 129 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 9.9 MiB/s wr, 160 op/s 2024-09-21T06:42:46.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:45 smithi100 ceph-mon[33401]: pgmap v545: 129 pgs: 129 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 9.9 MiB/s wr, 160 op/s 2024-09-21T06:42:46.618 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '5162919c', quiesce took 6.0 seconds. Will release after: 13.25 2024-09-21T06:42:47.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:47 smithi203 ceph-mon[33735]: pgmap v546: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 14 MiB/s wr, 220 op/s 2024-09-21T06:42:48.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:47 smithi067 ceph-mon[31232]: pgmap v546: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 14 MiB/s wr, 220 op/s 2024-09-21T06:42:48.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:47 smithi100 ceph-mon[33401]: pgmap v546: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 14 MiB/s wr, 220 op/s 2024-09-21T06:42:49.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:49 smithi203 ceph-mon[33735]: pgmap v547: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 21 MiB/s wr, 339 op/s 2024-09-21T06:42:50.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:49 smithi067 ceph-mon[31232]: pgmap v547: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 21 MiB/s wr, 339 op/s 2024-09-21T06:42:50.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:49 smithi100 ceph-mon[33401]: pgmap v547: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 21 MiB/s wr, 339 op/s 2024-09-21T06:42:51.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:51 smithi203 ceph-mon[33735]: pgmap v548: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 22 MiB/s wr, 352 op/s 2024-09-21T06:42:52.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:51 smithi067 ceph-mon[31232]: pgmap v548: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 22 MiB/s wr, 352 op/s 2024-09-21T06:42:52.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:51 smithi100 ceph-mon[33401]: pgmap v548: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 22 MiB/s wr, 352 op/s 2024-09-21T06:42:53.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:53 smithi203 ceph-mon[33735]: pgmap v549: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 23 MiB/s wr, 363 op/s 2024-09-21T06:42:54.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:53 smithi067 ceph-mon[31232]: pgmap v549: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 23 MiB/s wr, 363 op/s 2024-09-21T06:42:54.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:53 smithi100 ceph-mon[33401]: pgmap v549: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 23 MiB/s wr, 363 op/s 2024-09-21T06:42:55.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:55 smithi203 ceph-mon[33735]: pgmap v550: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 19 MiB/s wr, 302 op/s 2024-09-21T06:42:56.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:55 smithi067 ceph-mon[31232]: pgmap v550: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 19 MiB/s wr, 302 op/s 2024-09-21T06:42:56.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:55 smithi100 ceph-mon[33401]: pgmap v550: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 19 MiB/s wr, 302 op/s 2024-09-21T06:42:57.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:57 smithi203 ceph-mon[33735]: pgmap v551: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 16 MiB/s wr, 259 op/s 2024-09-21T06:42:58.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:57 smithi067 ceph-mon[31232]: pgmap v551: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 16 MiB/s wr, 259 op/s 2024-09-21T06:42:58.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:57 smithi100 ceph-mon[33401]: pgmap v551: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 16 MiB/s wr, 259 op/s 2024-09-21T06:42:59.868 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:42:59.897 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:42:59 smithi067 ceph-mon[31232]: pgmap v552: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 9.9 MiB/s wr, 160 op/s 2024-09-21T06:42:59.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:42:59 smithi203 ceph-mon[33735]: pgmap v552: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 9.9 MiB/s wr, 160 op/s 2024-09-21T06:43:00.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:42:59 smithi100 ceph-mon[33401]: pgmap v552: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 9.9 MiB/s wr, 160 op/s 2024-09-21T06:43:00.143 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:00.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.604+0000 7f58baa48640 1 -- 172.21.15.67:0/444568305 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58b4108d20 msgr2=0x7f58b410f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:00.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.604+0000 7f58baa48640 1 --2- 172.21.15.67:0/444568305 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58b4108d20 0x7f58b410f550 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009ca0 tx=0x7f58ac02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:43:00.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.604+0000 7f58baa48640 1 -- 172.21.15.67:0/444568305 shutdown_connections 2024-09-21T06:43:00.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.604+0000 7f58baa48640 1 --2- 172.21.15.67:0/444568305 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58b4108d20 0x7f58b410f550 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:00.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.604+0000 7f58baa48640 1 --2- 172.21.15.67:0/444568305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58b41083d0 0x7f58b41087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:00.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.604+0000 7f58baa48640 1 --2- 172.21.15.67:0/444568305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58b4102ae0 0x7f58b4102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:00.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.604+0000 7f58baa48640 1 -- 172.21.15.67:0/444568305 >> 172.21.15.67:0/444568305 conn(0x7f58b4079760 msgr2=0x7f58b4079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:00.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.604+0000 7f58baa48640 1 -- 172.21.15.67:0/444568305 shutdown_connections 2024-09-21T06:43:00.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.604+0000 7f58baa48640 1 -- 172.21.15.67:0/444568305 wait complete. 2024-09-21T06:43:00.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.605+0000 7f58baa48640 1 Processor -- start 2024-09-21T06:43:00.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.605+0000 7f58baa48640 1 -- start start 2024-09-21T06:43:00.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.605+0000 7f58baa48640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58b4102ae0 0x7f58b410ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:00.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.606+0000 7f58baa48640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58b41083d0 0x7f58b4110520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:00.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.606+0000 7f58baa48640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58b4108d20 0x7f58b4110a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:00.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.606+0000 7f58baa48640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f58b4113990 con 0x7f58b4108d20 2024-09-21T06:43:00.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.606+0000 7f58baa48640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f58b4113810 con 0x7f58b41083d0 2024-09-21T06:43:00.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.606+0000 7f58baa48640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f58b4113b10 con 0x7f58b4102ae0 2024-09-21T06:43:00.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.606+0000 7f58b8fe5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58b4108d20 0x7f58b4110a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:00.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.606+0000 7f58b8fe5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58b4108d20 0x7f58b4110a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51626/0 (socket says 172.21.15.67:51626) 2024-09-21T06:43:00.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.606+0000 7f58b8fe5640 1 -- 172.21.15.67:0/54903128 learned_addr learned my addr 172.21.15.67:0/54903128 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:43:00.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.606+0000 7f58b3fff640 1 --2- 172.21.15.67:0/54903128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58b4102ae0 0x7f58b410ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:00.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.606+0000 7f58b37fe640 1 --2- 172.21.15.67:0/54903128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58b41083d0 0x7f58b4110520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:00.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.606+0000 7f58b8fe5640 1 -- 172.21.15.67:0/54903128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58b4102ae0 msgr2=0x7f58b410ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:00.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.606+0000 7f58b8fe5640 1 --2- 172.21.15.67:0/54903128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58b4102ae0 0x7f58b410ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:00.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.606+0000 7f58b8fe5640 1 -- 172.21.15.67:0/54903128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58b41083d0 msgr2=0x7f58b4110520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:00.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.606+0000 7f58b8fe5640 1 --2- 172.21.15.67:0/54903128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58b41083d0 0x7f58b4110520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:00.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.606+0000 7f58b8fe5640 1 -- 172.21.15.67:0/54903128 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b41a84f0 con 0x7f58b4108d20 2024-09-21T06:43:00.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.606+0000 7f58b8fe5640 1 --2- 172.21.15.67:0/54903128 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58b4108d20 0x7f58b4110a60 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f58ac031040 tx=0x7f58ac032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:00.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.606+0000 7f58b17fa640 1 -- 172.21.15.67:0/54903128 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac004040 con 0x7f58b4108d20 2024-09-21T06:43:00.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.606+0000 7f58baa48640 1 -- 172.21.15.67:0/54903128 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58b41a8780 con 0x7f58b4108d20 2024-09-21T06:43:00.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.606+0000 7f58b17fa640 1 -- 172.21.15.67:0/54903128 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f58ac004670 con 0x7f58b4108d20 2024-09-21T06:43:00.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.607+0000 7f58b17fa640 1 -- 172.21.15.67:0/54903128 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac030d50 con 0x7f58b4108d20 2024-09-21T06:43:00.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.607+0000 7f58baa48640 1 -- 172.21.15.67:0/54903128 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58b41a8d10 con 0x7f58b4108d20 2024-09-21T06:43:00.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.607+0000 7f58baa48640 1 -- 172.21.15.67:0/54903128 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58b4103d10 con 0x7f58b4108d20 2024-09-21T06:43:00.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.609+0000 7f58b17fa640 1 -- 172.21.15.67:0/54903128 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f58ac0305c0 con 0x7f58b4108d20 2024-09-21T06:43:00.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.610+0000 7f58b17fa640 1 --2- 172.21.15.67:0/54903128 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f589c0b6510 0x7f589c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:00.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.611+0000 7f58b3fff640 1 --2- 172.21.15.67:0/54903128 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f589c0b6510 0x7f589c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:00.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.611+0000 7f58b17fa640 1 -- 172.21.15.67:0/54903128 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f58ac108e80 con 0x7f58b4108d20 2024-09-21T06:43:00.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.611+0000 7f58b3fff640 1 --2- 172.21.15.67:0/54903128 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f589c0b6510 0x7f589c0b8a20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f58a4005e10 tx=0x7f58a4005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:00.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.614+0000 7f58b17fa640 1 -- 172.21.15.67:0/54903128 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f58ac0098b0 con 0x7f58b4108d20 2024-09-21T06:43:00.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.845+0000 7f58baa48640 1 -- 172.21.15.67:0/54903128 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58b410ab20 con 0x7f58b4108d20 2024-09-21T06:43:00.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.846+0000 7f58b17fa640 1 -- 172.21.15.67:0/54903128 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7f58ac0cf340 con 0x7f58b4108d20 2024-09-21T06:43:00.849 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:43:00.849 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:43:00.849 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:43:00.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.849+0000 7f58baa48640 1 -- 172.21.15.67:0/54903128 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f589c0b6510 msgr2=0x7f589c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:00.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.849+0000 7f58baa48640 1 --2- 172.21.15.67:0/54903128 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f589c0b6510 0x7f589c0b8a20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f58a4005e10 tx=0x7f58a4005da0 comp rx=0 tx=0).stop 2024-09-21T06:43:00.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.849+0000 7f58baa48640 1 -- 172.21.15.67:0/54903128 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58b4108d20 msgr2=0x7f58b4110a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:00.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.849+0000 7f58baa48640 1 --2- 172.21.15.67:0/54903128 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58b4108d20 0x7f58b4110a60 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f58ac031040 tx=0x7f58ac032bc0 comp rx=0 tx=0).stop 2024-09-21T06:43:00.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.849+0000 7f58baa48640 1 -- 172.21.15.67:0/54903128 shutdown_connections 2024-09-21T06:43:00.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.849+0000 7f58baa48640 1 --2- 172.21.15.67:0/54903128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58b41083d0 0x7f58b4110520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:00.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.849+0000 7f58baa48640 1 --2- 172.21.15.67:0/54903128 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f589c0b6510 0x7f589c0b8a20 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:00.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.849+0000 7f58baa48640 1 --2- 172.21.15.67:0/54903128 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58b4108d20 0x7f58b4110a60 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:00.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.849+0000 7f58baa48640 1 --2- 172.21.15.67:0/54903128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58b4102ae0 0x7f58b410ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:00.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.849+0000 7f58baa48640 1 -- 172.21.15.67:0/54903128 >> 172.21.15.67:0/54903128 conn(0x7f58b4079760 msgr2=0x7f58b41043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:00.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.849+0000 7f58baa48640 1 -- 172.21.15.67:0/54903128 shutdown_connections 2024-09-21T06:43:00.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:00.849+0000 7f58baa48640 1 -- 172.21.15.67:0/54903128 wait complete. 2024-09-21T06:43:01.758 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db --set-id 5162919c --release --await' 2024-09-21T06:43:01.759 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24671 quiesce db --set-id 5162919c --release --await 2024-09-21T06:43:01.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:01 smithi203 ceph-mon[33735]: pgmap v553: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 MiB/s wr, 33 op/s 2024-09-21T06:43:01.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:01 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/54903128' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:43:02.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:01 smithi067 ceph-mon[31232]: pgmap v553: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 MiB/s wr, 33 op/s 2024-09-21T06:43:02.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:01 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/54903128' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:43:02.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:01 smithi100 ceph-mon[33401]: pgmap v553: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 MiB/s wr, 33 op/s 2024-09-21T06:43:02.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:01 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/54903128' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:43:02.070 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:02.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.582+0000 7f667d1d8640 1 -- 172.21.15.67:0/2335546363 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6678105380 msgr2=0x7f667810d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:02.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.582+0000 7f667d1d8640 1 --2- 172.21.15.67:0/2335546363 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6678105380 0x7f667810d8d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f666c009900 tx=0x7f666c02a410 comp rx=0 tx=0).stop 2024-09-21T06:43:02.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.582+0000 7f667d1d8640 1 -- 172.21.15.67:0/2335546363 shutdown_connections 2024-09-21T06:43:02.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.582+0000 7f667d1d8640 1 --2- 172.21.15.67:0/2335546363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f667810de60 0x7f6678110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.582+0000 7f667d1d8640 1 --2- 172.21.15.67:0/2335546363 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6678105380 0x7f667810d8d0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.582+0000 7f667d1d8640 1 --2- 172.21.15.67:0/2335546363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6678104a30 0x7f6678104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.582+0000 7f667d1d8640 1 -- 172.21.15.67:0/2335546363 >> 172.21.15.67:0/2335546363 conn(0x7f6678079760 msgr2=0x7f6678079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:02.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.582+0000 7f667d1d8640 1 -- 172.21.15.67:0/2335546363 shutdown_connections 2024-09-21T06:43:02.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.582+0000 7f667d1d8640 1 -- 172.21.15.67:0/2335546363 wait complete. 2024-09-21T06:43:02.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.583+0000 7f667d1d8640 1 Processor -- start 2024-09-21T06:43:02.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.583+0000 7f667d1d8640 1 -- start start 2024-09-21T06:43:02.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.583+0000 7f667d1d8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6678104a30 0x7f667819c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:02.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.583+0000 7f667d1d8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6678105380 0x7f667819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:02.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.583+0000 7f667d1d8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f667810de60 0x7f66781a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:02.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.583+0000 7f667d1d8640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f66781139b0 con 0x7f6678104a30 2024-09-21T06:43:02.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.583+0000 7f667d1d8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6678113830 con 0x7f667810de60 2024-09-21T06:43:02.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.583+0000 7f667d1d8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6678113b30 con 0x7f6678105380 2024-09-21T06:43:02.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.583+0000 7f6676d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6678104a30 0x7f667819c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:02.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.583+0000 7f6676d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6678104a30 0x7f667819c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51648/0 (socket says 172.21.15.67:51648) 2024-09-21T06:43:02.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.583+0000 7f6676d9d640 1 -- 172.21.15.67:0/3446372786 learned_addr learned my addr 172.21.15.67:0/3446372786 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:43:02.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.584+0000 7f667759e640 1 --2- 172.21.15.67:0/3446372786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f667810de60 0x7f66781a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:02.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.584+0000 7f667659c640 1 --2- 172.21.15.67:0/3446372786 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6678105380 0x7f667819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:02.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.584+0000 7f6676d9d640 1 -- 172.21.15.67:0/3446372786 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6678105380 msgr2=0x7f667819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:02.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.584+0000 7f6676d9d640 1 --2- 172.21.15.67:0/3446372786 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6678105380 0x7f667819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.584+0000 7f6676d9d640 1 -- 172.21.15.67:0/3446372786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f667810de60 msgr2=0x7f66781a1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:02.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.584+0000 7f6676d9d640 1 --2- 172.21.15.67:0/3446372786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f667810de60 0x7f66781a1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.584+0000 7f6676d9d640 1 -- 172.21.15.67:0/3446372786 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66781a26b0 con 0x7f6678104a30 2024-09-21T06:43:02.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.584+0000 7f667659c640 1 --2- 172.21.15.67:0/3446372786 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6678105380 0x7f667819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:43:02.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.584+0000 7f667759e640 1 --2- 172.21.15.67:0/3446372786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f667810de60 0x7f66781a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:43:02.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.584+0000 7f6676d9d640 1 --2- 172.21.15.67:0/3446372786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6678104a30 0x7f667819c740 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f666800d770 tx=0x7f666800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:02.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.584+0000 7f665ffff640 1 -- 172.21.15.67:0/3446372786 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668017070 con 0x7f6678104a30 2024-09-21T06:43:02.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.584+0000 7f667d1d8640 1 -- 172.21.15.67:0/3446372786 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66781a29a0 con 0x7f6678104a30 2024-09-21T06:43:02.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.584+0000 7f665ffff640 1 -- 172.21.15.67:0/3446372786 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6668009c30 con 0x7f6678104a30 2024-09-21T06:43:02.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.584+0000 7f665ffff640 1 -- 172.21.15.67:0/3446372786 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668003c50 con 0x7f6678104a30 2024-09-21T06:43:02.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.584+0000 7f667d1d8640 1 -- 172.21.15.67:0/3446372786 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66781a9100 con 0x7f6678104a30 2024-09-21T06:43:02.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.587+0000 7f668374c740 1 Processor -- start 2024-09-21T06:43:02.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.587+0000 7f668374c740 1 -- start start 2024-09-21T06:43:02.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.587+0000 7f665ffff640 1 -- 172.21.15.67:0/3446372786 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f6668003df0 con 0x7f6678104a30 2024-09-21T06:43:02.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.587+0000 7f668374c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561be8d4520 0x5561bec7a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:02.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.587+0000 7f668374c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5561be8ab660 0x5561bec7aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:02.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.587+0000 7f668374c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561be8b3840 0x5561bec7af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:02.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.587+0000 7f668374c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5561beb65dc0 con 0x5561be8ab660 2024-09-21T06:43:02.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.587+0000 7f668374c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5561bebf21f0 con 0x5561be8b3840 2024-09-21T06:43:02.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.587+0000 7f668374c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5561bebe1620 con 0x5561be8d4520 2024-09-21T06:43:02.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.587+0000 7f6676d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5561be8ab660 0x5561bec7aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:02.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.587+0000 7f6676d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5561be8ab660 0x5561bec7aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51658/0 (socket says 172.21.15.67:51658) 2024-09-21T06:43:02.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.587+0000 7f6676d9d640 1 -- 172.21.15.67:0/2891021942 learned_addr learned my addr 172.21.15.67:0/2891021942 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:43:02.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.587+0000 7f667659c640 1 --2- 172.21.15.67:0/2891021942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561be8b3840 0x5561bec7af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:02.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.587+0000 7f667759e640 1 --2- 172.21.15.67:0/2891021942 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561be8d4520 0x5561bec7a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:02.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.588+0000 7f6676d9d640 1 -- 172.21.15.67:0/2891021942 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561be8d4520 msgr2=0x5561bec7a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:43:02.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.588+0000 7f6676d9d640 1 --2- 172.21.15.67:0/2891021942 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561be8d4520 0x5561bec7a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.588+0000 7f6676d9d640 1 -- 172.21.15.67:0/2891021942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561be8b3840 msgr2=0x5561bec7af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:43:02.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.588+0000 7f6676d9d640 1 --2- 172.21.15.67:0/2891021942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561be8b3840 0x5561bec7af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.588+0000 7f6676d9d640 1 -- 172.21.15.67:0/2891021942 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561beba9fd0 con 0x5561be8ab660 2024-09-21T06:43:02.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.588+0000 7f665ffff640 1 --2- 172.21.15.67:0/3446372786 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f66600b6510 0x7f66600b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:02.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.588+0000 7f667659c640 1 --2- 172.21.15.67:0/2891021942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561be8b3840 0x5561bec7af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:43:02.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.588+0000 7f667759e640 1 --2- 172.21.15.67:0/3446372786 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f66600b6510 0x7f66600b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:02.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.588+0000 7f6676d9d640 1 --2- 172.21.15.67:0/2891021942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5561be8ab660 0x5561bec7aa50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f66680ef480 tx=0x7f66680ef940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa0942d504009b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:02.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.588+0000 7f665ffff640 1 -- 172.21.15.67:0/3446372786 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f66680e71b0 con 0x7f6678104a30 2024-09-21T06:43:02.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.588+0000 7f665d7fa640 1 -- 172.21.15.67:0/2891021942 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668020070 con 0x5561be8ab660 2024-09-21T06:43:02.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.588+0000 7f665d7fa640 1 -- 172.21.15.67:0/2891021942 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f66680aca80 con 0x5561be8ab660 2024-09-21T06:43:02.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.588+0000 7f665d7fa640 1 -- 172.21.15.67:0/2891021942 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66680a8e10 con 0x5561be8ab660 2024-09-21T06:43:02.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.589+0000 7f668374c740 1 -- 172.21.15.67:0/2891021942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5561be8ab660 msgr2=0x5561bec7aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:02.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.589+0000 7f668374c740 1 --2- 172.21.15.67:0/2891021942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5561be8ab660 0x5561bec7aa50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f66680ef480 tx=0x7f66680ef940 comp rx=0 tx=0).stop 2024-09-21T06:43:02.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.589+0000 7f667759e640 1 --2- 172.21.15.67:0/3446372786 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f66600b6510 0x7f66600b8a20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f667000a450 tx=0x7f6670009650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:02.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.589+0000 7f668374c740 1 -- 172.21.15.67:0/2891021942 shutdown_connections 2024-09-21T06:43:02.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.589+0000 7f668374c740 1 --2- 172.21.15.67:0/2891021942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561be8b3840 0x5561bec7af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.589+0000 7f668374c740 1 --2- 172.21.15.67:0/2891021942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5561be8ab660 0x5561bec7aa50 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.589+0000 7f668374c740 1 --2- 172.21.15.67:0/2891021942 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561be8d4520 0x5561bec7a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.589+0000 7f668374c740 1 -- 172.21.15.67:0/2891021942 >> 172.21.15.67:0/2891021942 conn(0x5561be8c8910 msgr2=0x5561bec78400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:02.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.589+0000 7f668374c740 1 -- 172.21.15.67:0/2891021942 shutdown_connections 2024-09-21T06:43:02.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.589+0000 7f668374c740 1 -- 172.21.15.67:0/2891021942 wait complete. 2024-09-21T06:43:02.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.590+0000 7f668374c740 1 -- start start 2024-09-21T06:43:02.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.590+0000 7f668374c740 1 Processor -- start 2024-09-21T06:43:02.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.590+0000 7f668374c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561be8ab660 0x5561bec85ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:02.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.590+0000 7f668374c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5561be8b3840 0x5561bec860e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:02.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.590+0000 7f668374c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561be8d4520 0x5561bec86660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:02.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.590+0000 7f668374c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5561beb7c300 con 0x5561be8b3840 2024-09-21T06:43:02.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.590+0000 7f668374c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5561bebe5430 con 0x5561be8d4520 2024-09-21T06:43:02.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.590+0000 7f668374c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5561bea42bb0 con 0x5561be8ab660 2024-09-21T06:43:02.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.590+0000 7f668374c740 10 client.0 _ll_register_callbacks cb 0x5561beb96120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:43:02.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.590+0000 7f667659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5561be8b3840 0x5561bec860e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:02.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.590+0000 7f667659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5561be8b3840 0x5561bec860e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51664/0 (socket says 172.21.15.67:51664) 2024-09-21T06:43:02.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.590+0000 7f667659c640 1 -- 172.21.15.67:0/3106571365 learned_addr learned my addr 172.21.15.67:0/3106571365 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:43:02.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.590+0000 7f6676d9d640 1 --2- 172.21.15.67:0/3106571365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561be8ab660 0x5561bec85ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:02.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.591+0000 7f667759e640 1 --2- 172.21.15.67:0/3106571365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561be8d4520 0x5561bec86660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:02.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.591+0000 7f667659c640 1 -- 172.21.15.67:0/3106571365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561be8ab660 msgr2=0x5561bec85ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:43:02.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.591+0000 7f667659c640 1 --2- 172.21.15.67:0/3106571365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561be8ab660 0x5561bec85ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.591+0000 7f667659c640 1 -- 172.21.15.67:0/3106571365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561be8d4520 msgr2=0x5561bec86660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:43:02.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.591+0000 7f667659c640 1 --2- 172.21.15.67:0/3106571365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561be8d4520 0x5561bec86660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.591+0000 7f667659c640 1 -- 172.21.15.67:0/3106571365 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561beb6b780 con 0x5561be8b3840 2024-09-21T06:43:02.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.591+0000 7f667659c640 1 --2- 172.21.15.67:0/3106571365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5561be8b3840 0x5561bec860e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f667806b910 tx=0x7f666c031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68726ee63fd573dd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:02.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.591+0000 7f665aff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:43:02.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.591+0000 7f665aff5640 1 -- 172.21.15.67:0/3106571365 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c03f070 con 0x5561be8b3840 2024-09-21T06:43:02.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.591+0000 7f665aff5640 1 -- 172.21.15.67:0/3106571365 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f666c004250 con 0x5561be8b3840 2024-09-21T06:43:02.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.591+0000 7f668374c740 1 -- 172.21.15.67:0/3106571365 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66680aca80 con 0x5561be8b3840 2024-09-21T06:43:02.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.591+0000 7f665aff5640 1 -- 172.21.15.67:0/3106571365 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f666c035070 con 0x5561be8b3840 2024-09-21T06:43:02.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.591+0000 7f668374c740 10 client.15747 fetch_fsmap learned FSMap version 21 2024-09-21T06:43:02.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.591+0000 7f668374c740 1 -- 172.21.15.67:0/3106571365 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5561be93fb00 con 0x5561be8b3840 2024-09-21T06:43:02.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.591+0000 7f665aff5640 1 -- 172.21.15.67:0/3106571365 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f66680aca80 con 0x5561be8b3840 2024-09-21T06:43:02.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.592+0000 7f665aff5640 1 -- 172.21.15.67:0/3106571365 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f666c004b50 con 0x5561be8b3840 2024-09-21T06:43:02.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.592+0000 7f668374c740 10 client.15747 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:43:02.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.592+0000 7f668374c740 10 client.15747 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:43:02.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.592+0000 7f668374c740 20 client.15747 populate_metadata read hostname 'smithi067' 2024-09-21T06:43:02.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.592+0000 7f668374c740 1 --2- 172.21.15.67:0/3106571365 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x5561be892ea0 0x5561bec97f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:02.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.592+0000 7f668374c740 4 client.15747 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:43:02.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.592+0000 7f668374c740 1 -- 172.21.15.67:0/3106571365 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561bebf4140 con 0x5561be892ea0 2024-09-21T06:43:02.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.592+0000 7f6676d9d640 1 --2- 172.21.15.67:0/3106571365 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x5561be892ea0 0x5561bec97f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:02.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.593+0000 7f6676d9d640 1 --2- 172.21.15.67:0/3106571365 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x5561be892ea0 0x5561bec97f30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=821d63caa691c90b server_cookie=8e76e92687b0764e in_seq=0 out_seq=0 2024-09-21T06:43:02.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.593+0000 7f665aff5640 10 client.15747 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:43:02.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.594+0000 7f665aff5640 1 -- 172.21.15.67:0/3106571365 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5561bebf4140 con 0x5561be892ea0 2024-09-21T06:43:02.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.594+0000 7f665aff5640 10 client.15747 handle_command_reply: tid=0 2024-09-21T06:43:02.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.594+0000 7f668374c740 1 client.15747 shutdown 2024-09-21T06:43:02.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.594+0000 7f668374c740 1 -- 172.21.15.67:0/3106571365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5561be8b3840 msgr2=0x5561bec860e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:02.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.594+0000 7f668374c740 1 --2- 172.21.15.67:0/3106571365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5561be8b3840 0x5561bec860e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f667806b910 tx=0x7f666c031630 comp rx=0 tx=0).stop 2024-09-21T06:43:02.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.594+0000 7f668374c740 1 -- 172.21.15.67:0/3106571365 shutdown_connections 2024-09-21T06:43:02.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.594+0000 7f668374c740 1 --2- 172.21.15.67:0/3106571365 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x5561be892ea0 0x5561bec97f30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.594+0000 7f665aff5640 0 client.15747 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:43:02.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.594+0000 7f668374c740 1 --2- 172.21.15.67:0/3106571365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561be8d4520 0x5561bec86660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.594+0000 7f668374c740 1 --2- 172.21.15.67:0/3106571365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5561be8b3840 0x5561bec860e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.594+0000 7f668374c740 1 --2- 172.21.15.67:0/3106571365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561be8ab660 0x5561bec85ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.594+0000 7f668374c740 1 -- 172.21.15.67:0/3106571365 >> 172.21.15.67:0/3106571365 conn(0x5561be8c8910 msgr2=0x5561bec7d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:02.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.594+0000 7f668374c740 1 -- 172.21.15.67:0/3106571365 shutdown_connections 2024-09-21T06:43:02.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.594+0000 7f668374c740 1 -- 172.21.15.67:0/3106571365 wait complete. 2024-09-21T06:43:02.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.594+0000 7f668374c740 20 client.15747 trim_cache size 0 max 16384 2024-09-21T06:43:02.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.611+0000 7f668374c740 1 Processor -- start 2024-09-21T06:43:02.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.611+0000 7f668374c740 1 -- start start 2024-09-21T06:43:02.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.612+0000 7f668374c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561be892ea0 0x5561bec78430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:02.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.612+0000 7f668374c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561be8ab660 0x5561bec7f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:02.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.612+0000 7f668374c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5561be8b3840 0x5561bec7fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:02.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.612+0000 7f668374c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f66781b0ed0 con 0x5561be8b3840 2024-09-21T06:43:02.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.612+0000 7f668374c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f66781b3750 con 0x5561be8ab660 2024-09-21T06:43:02.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.612+0000 7f668374c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f66781b1540 con 0x5561be892ea0 2024-09-21T06:43:02.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.612+0000 7f667759e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5561be8b3840 0x5561bec7fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:02.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.612+0000 7f667759e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5561be8b3840 0x5561bec7fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51672/0 (socket says 172.21.15.67:51672) 2024-09-21T06:43:02.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.612+0000 7f667759e640 1 -- 172.21.15.67:0/3519848621 learned_addr learned my addr 172.21.15.67:0/3519848621 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:43:02.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.612+0000 7f667659c640 1 --2- 172.21.15.67:0/3519848621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561be8ab660 0x5561bec7f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:02.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.612+0000 7f6676d9d640 1 --2- 172.21.15.67:0/3519848621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561be892ea0 0x5561bec78430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:02.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.612+0000 7f667759e640 1 -- 172.21.15.67:0/3519848621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561be892ea0 msgr2=0x5561bec78430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:43:02.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.612+0000 7f667759e640 1 --2- 172.21.15.67:0/3519848621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561be892ea0 0x5561bec78430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.612+0000 7f667759e640 1 -- 172.21.15.67:0/3519848621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561be8ab660 msgr2=0x5561bec7f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:43:02.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.612+0000 7f667759e640 1 --2- 172.21.15.67:0/3519848621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561be8ab660 0x5561bec7f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.612+0000 7f667759e640 1 -- 172.21.15.67:0/3519848621 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561be9b6960 con 0x5561be8b3840 2024-09-21T06:43:02.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.612+0000 7f667759e640 1 --2- 172.21.15.67:0/3519848621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5561be8b3840 0x5561bec7fae0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6670007900 tx=0x7f6670013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291e81627f077b73 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:02.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.613+0000 7f665cff9640 1 -- 172.21.15.67:0/3519848621 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6670029070 con 0x5561be8b3840 2024-09-21T06:43:02.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.613+0000 7f665cff9640 1 -- 172.21.15.67:0/3519848621 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6670018cb0 con 0x5561be8b3840 2024-09-21T06:43:02.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.613+0000 7f665cff9640 1 -- 172.21.15.67:0/3519848621 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6670024d60 con 0x5561be8b3840 2024-09-21T06:43:02.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.613+0000 7f668374c740 1 -- 172.21.15.67:0/3519848621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5561be8b3840 msgr2=0x5561bec7fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:02.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.613+0000 7f668374c740 1 --2- 172.21.15.67:0/3519848621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5561be8b3840 0x5561bec7fae0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6670007900 tx=0x7f6670013000 comp rx=0 tx=0).stop 2024-09-21T06:43:02.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.614+0000 7f668374c740 1 -- 172.21.15.67:0/3519848621 shutdown_connections 2024-09-21T06:43:02.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.614+0000 7f668374c740 1 --2- 172.21.15.67:0/3519848621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561be8ab660 0x5561bec7f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.614+0000 7f668374c740 1 --2- 172.21.15.67:0/3519848621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5561be8b3840 0x5561bec7fae0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.614+0000 7f668374c740 1 --2- 172.21.15.67:0/3519848621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561be892ea0 0x5561bec78430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.614+0000 7f668374c740 1 -- 172.21.15.67:0/3519848621 >> 172.21.15.67:0/3519848621 conn(0x5561be8c8910 msgr2=0x5561bec77d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:02.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.614+0000 7f668374c740 1 -- 172.21.15.67:0/3519848621 shutdown_connections 2024-09-21T06:43:02.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.614+0000 7f668374c740 1 -- 172.21.15.67:0/3519848621 wait complete. 2024-09-21T06:43:02.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.614+0000 7f668374c740 1 -- start start 2024-09-21T06:43:02.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.614+0000 7f668374c740 1 Processor -- start 2024-09-21T06:43:02.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.614+0000 7f668374c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561be892ea0 0x5561bec8e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:02.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.614+0000 7f668374c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561be8ab660 0x5561bec8e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:02.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.614+0000 7f668374c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5561be8b3840 0x5561bec88de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:02.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.615+0000 7f668374c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f66781b1c10 con 0x5561be8b3840 2024-09-21T06:43:02.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.615+0000 7f668374c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f66781b3080 con 0x5561be892ea0 2024-09-21T06:43:02.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.615+0000 7f668374c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f66781b22e0 con 0x5561be8ab660 2024-09-21T06:43:02.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.615+0000 7f667759e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5561be8b3840 0x5561bec88de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:02.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.615+0000 7f667759e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5561be8b3840 0x5561bec88de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51682/0 (socket says 172.21.15.67:51682) 2024-09-21T06:43:02.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.615+0000 7f668374c740 10 client.0 _ll_register_callbacks cb 0x7f666c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:43:02.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.615+0000 7f667759e640 1 -- 172.21.15.67:0/2226489270 learned_addr learned my addr 172.21.15.67:0/2226489270 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:43:02.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.615+0000 7f6676d9d640 1 --2- 172.21.15.67:0/2226489270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561be892ea0 0x5561bec8e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:02.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.615+0000 7f667659c640 1 --2- 172.21.15.67:0/2226489270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561be8ab660 0x5561bec8e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:02.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.615+0000 7f667759e640 1 -- 172.21.15.67:0/2226489270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561be8ab660 msgr2=0x5561bec8e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:43:02.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.615+0000 7f667759e640 1 --2- 172.21.15.67:0/2226489270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561be8ab660 0x5561bec8e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.615+0000 7f667759e640 1 -- 172.21.15.67:0/2226489270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561be892ea0 msgr2=0x5561bec8e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:43:02.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.615+0000 7f667759e640 1 --2- 172.21.15.67:0/2226489270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561be892ea0 0x5561bec8e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.615+0000 7f667759e640 1 -- 172.21.15.67:0/2226489270 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561bec89420 con 0x5561be8b3840 2024-09-21T06:43:02.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.615+0000 7f667759e640 1 --2- 172.21.15.67:0/2226489270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5561be8b3840 0x5561bec88de0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6670024a00 tx=0x7f6670024cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3601ebae982926a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:02.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.615+0000 7f665aff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:43:02.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.615+0000 7f665aff5640 1 -- 172.21.15.67:0/2226489270 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6670029070 con 0x5561be8b3840 2024-09-21T06:43:02.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.616+0000 7f665aff5640 1 -- 172.21.15.67:0/2226489270 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f6670018a30 con 0x5561be8b3840 2024-09-21T06:43:02.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.616+0000 7f668374c740 1 -- 172.21.15.67:0/2226489270 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6670018cb0 con 0x5561be8b3840 2024-09-21T06:43:02.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.616+0000 7f665aff5640 1 -- 172.21.15.67:0/2226489270 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6670018e60 con 0x5561be8b3840 2024-09-21T06:43:02.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.616+0000 7f668374c740 10 client.15753 fetch_fsmap learned FSMap version 21 2024-09-21T06:43:02.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.616+0000 7f668374c740 1 -- 172.21.15.67:0/2226489270 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5561bed031d0 con 0x5561be8b3840 2024-09-21T06:43:02.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.616+0000 7f665aff5640 1 -- 172.21.15.67:0/2226489270 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6670018cb0 con 0x5561be8b3840 2024-09-21T06:43:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.616+0000 7f665aff5640 1 -- 172.21.15.67:0/2226489270 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6670023070 con 0x5561be8b3840 2024-09-21T06:43:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.616+0000 7f668374c740 10 client.15753 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:43:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.616+0000 7f668374c740 10 client.15753 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:43:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.616+0000 7f668374c740 20 client.15753 populate_metadata read hostname 'smithi067' 2024-09-21T06:43:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.616+0000 7f668374c740 1 --2- 172.21.15.67:0/2226489270 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x5561be8d4520 0x5561bed058a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.616+0000 7f668374c740 4 client.15753 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "5162919c", "release": true, "await": true}] 2024-09-21T06:43:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.616+0000 7f668374c740 1 -- 172.21.15.67:0/2226489270 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "quiesce db", "set_id": "5162919c", "release": true, "await": true}) -- 0x5561bebefbf0 con 0x5561be8d4520 2024-09-21T06:43:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.617+0000 7f6676d9d640 1 --2- 172.21.15.67:0/2226489270 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x5561be8d4520 0x5561bed058a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.617+0000 7f665aff5640 10 client.15753 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:43:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.617+0000 7f6676d9d640 1 --2- 172.21.15.67:0/2226489270 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x5561be8d4520 0x5561bed058a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e944ca842187d791 server_cookie=c01f79d6a4755110 in_seq=0 out_seq=0 2024-09-21T06:43:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.620+0000 7f665aff5640 1 -- 172.21.15.67:0/2226489270 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5561bebefbf0 con 0x5561be8d4520 2024-09-21T06:43:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.620+0000 7f665aff5640 10 client.15753 handle_command_reply: tid=0 2024-09-21T06:43:02.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.620+0000 7f668374c740 1 client.15753 shutdown 2024-09-21T06:43:02.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.621+0000 7f668374c740 1 -- 172.21.15.67:0/2226489270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5561be8b3840 msgr2=0x5561bec88de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:02.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.621+0000 7f668374c740 1 --2- 172.21.15.67:0/2226489270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5561be8b3840 0x5561bec88de0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6670024a00 tx=0x7f6670024cd0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.621+0000 7f668374c740 1 -- 172.21.15.67:0/2226489270 shutdown_connections 2024-09-21T06:43:02.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.621+0000 7f668374c740 1 --2- 172.21.15.67:0/2226489270 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x5561be8d4520 0x5561bed058a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.621+0000 7f668374c740 1 --2- 172.21.15.67:0/2226489270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5561be8b3840 0x5561bec88de0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.621+0000 7f668374c740 1 --2- 172.21.15.67:0/2226489270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561be8ab660 0x5561bec8e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.621+0000 7f668374c740 1 --2- 172.21.15.67:0/2226489270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561be892ea0 0x5561bec8e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.621+0000 7f668374c740 1 -- 172.21.15.67:0/2226489270 >> 172.21.15.67:0/2226489270 conn(0x5561be8c8910 msgr2=0x5561bec7b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:02.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.621+0000 7f665aff5640 0 client.15753 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:43:02.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.621+0000 7f668374c740 1 -- 172.21.15.67:0/2226489270 shutdown_connections 2024-09-21T06:43:02.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.621+0000 7f668374c740 1 -- 172.21.15.67:0/2226489270 wait complete. 2024-09-21T06:43:02.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.621+0000 7f668374c740 20 client.15753 trim_cache size 0 max 16384 2024-09-21T06:43:02.635 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-21T06:43:02.635 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 21, 2024-09-21T06:43:02.635 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-09-21T06:43:02.635 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 28, 2024-09-21T06:43:02.635 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-21T06:43:02.635 INFO:teuthology.orchestra.run.smithi067.stdout: "5162919c": { 2024-09-21T06:43:02.635 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 28, 2024-09-21T06:43:02.635 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-21T06:43:02.635 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:43:02.635 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-21T06:43:02.635 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:43:02.635 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:43:02.636 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-21T06:43:02.636 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 139.2, 2024-09-21T06:43:02.636 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-21T06:43:02.636 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-21T06:43:02.636 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-21T06:43:02.636 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:43:02.636 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-21T06:43:02.636 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:43:02.636 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:43:02.636 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:43:02.636 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:43:02.636 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:43:02.636 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:43:02.636 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-21T06:43:02.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.622+0000 7f667d1d8640 1 -- 172.21.15.67:0/3446372786 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f66600b6510 msgr2=0x7f66600b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:02.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.622+0000 7f667d1d8640 1 --2- 172.21.15.67:0/3446372786 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f66600b6510 0x7f66600b8a20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f667000a450 tx=0x7f6670009650 comp rx=0 tx=0).stop 2024-09-21T06:43:02.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.622+0000 7f667d1d8640 1 -- 172.21.15.67:0/3446372786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6678104a30 msgr2=0x7f667819c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:02.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.622+0000 7f667d1d8640 1 --2- 172.21.15.67:0/3446372786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6678104a30 0x7f667819c740 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f666800d770 tx=0x7f666800dc30 comp rx=0 tx=0).stop 2024-09-21T06:43:02.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.622+0000 7f667d1d8640 1 -- 172.21.15.67:0/3446372786 shutdown_connections 2024-09-21T06:43:02.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.622+0000 7f667d1d8640 1 --2- 172.21.15.67:0/3446372786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f667810de60 0x7f66781a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.622+0000 7f667d1d8640 1 --2- 172.21.15.67:0/3446372786 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f66600b6510 0x7f66600b8a20 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.622+0000 7f667d1d8640 1 --2- 172.21.15.67:0/3446372786 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6678105380 0x7f667819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.622+0000 7f667d1d8640 1 --2- 172.21.15.67:0/3446372786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6678104a30 0x7f667819c740 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:02.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.622+0000 7f667d1d8640 1 -- 172.21.15.67:0/3446372786 >> 172.21.15.67:0/3446372786 conn(0x7f6678079760 msgr2=0x7f6678109e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:02.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.622+0000 7f667d1d8640 1 -- 172.21.15.67:0/3446372786 shutdown_connections 2024-09-21T06:43:02.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:43:02.622+0000 7f667d1d8640 1 -- 172.21.15.67:0/3446372786 wait complete. 2024-09-21T06:43:03.380 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '5162919c', total seconds elapsed: 22.7 2024-09-21T06:43:03.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:03 smithi203 ceph-mon[33735]: pgmap v554: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1010 KiB/s wr, 18 op/s 2024-09-21T06:43:04.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:03 smithi067 ceph-mon[31232]: pgmap v554: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1010 KiB/s wr, 18 op/s 2024-09-21T06:43:04.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:03 smithi100 ceph-mon[33401]: pgmap v554: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1010 KiB/s wr, 18 op/s 2024-09-21T06:43:05.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:05 smithi203 ceph-mon[33735]: pgmap v555: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 555 KiB/s wr, 10 op/s 2024-09-21T06:43:06.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:05 smithi067 ceph-mon[31232]: pgmap v555: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 555 KiB/s wr, 10 op/s 2024-09-21T06:43:06.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:05 smithi100 ceph-mon[33401]: pgmap v555: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 555 KiB/s wr, 10 op/s 2024-09-21T06:43:07.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:07 smithi203 ceph-mon[33735]: pgmap v556: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3.9 MiB/s wr, 65 op/s 2024-09-21T06:43:08.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:07 smithi067 ceph-mon[31232]: pgmap v556: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3.9 MiB/s wr, 65 op/s 2024-09-21T06:43:08.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:07 smithi100 ceph-mon[33401]: pgmap v556: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3.9 MiB/s wr, 65 op/s 2024-09-21T06:43:09.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:09 smithi203 ceph-mon[33735]: pgmap v557: 129 pgs: 129 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 13 MiB/s wr, 205 op/s 2024-09-21T06:43:10.043 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:09 smithi067 ceph-mon[31232]: pgmap v557: 129 pgs: 129 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 13 MiB/s wr, 205 op/s 2024-09-21T06:43:10.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:09 smithi100 ceph-mon[33401]: pgmap v557: 129 pgs: 129 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 13 MiB/s wr, 205 op/s 2024-09-21T06:43:11.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:11 smithi203 ceph-mon[33735]: pgmap v558: 129 pgs: 129 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 15 MiB/s wr, 234 op/s 2024-09-21T06:43:12.041 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:11 smithi067 ceph-mon[31232]: pgmap v558: 129 pgs: 129 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 15 MiB/s wr, 234 op/s 2024-09-21T06:43:12.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:11 smithi100 ceph-mon[33401]: pgmap v558: 129 pgs: 129 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 15 MiB/s wr, 234 op/s 2024-09-21T06:43:13.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:13 smithi203 ceph-mon[33735]: pgmap v559: 129 pgs: 129 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 28 MiB/s wr, 448 op/s 2024-09-21T06:43:14.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:13 smithi067 ceph-mon[31232]: pgmap v559: 129 pgs: 129 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 28 MiB/s wr, 448 op/s 2024-09-21T06:43:14.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:13 smithi100 ceph-mon[33401]: pgmap v559: 129 pgs: 129 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 28 MiB/s wr, 448 op/s 2024-09-21T06:43:15.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:15 smithi203 ceph-mon[33735]: pgmap v560: 129 pgs: 129 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 32 MiB/s wr, 516 op/s 2024-09-21T06:43:16.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:15 smithi067 ceph-mon[31232]: pgmap v560: 129 pgs: 129 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 32 MiB/s wr, 516 op/s 2024-09-21T06:43:16.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:15 smithi100 ceph-mon[33401]: pgmap v560: 129 pgs: 129 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 32 MiB/s wr, 516 op/s 2024-09-21T06:43:17.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:17 smithi203 ceph-mon[33735]: pgmap v561: 129 pgs: 129 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 36 MiB/s wr, 574 op/s 2024-09-21T06:43:18.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:17 smithi067 ceph-mon[31232]: pgmap v561: 129 pgs: 129 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 36 MiB/s wr, 574 op/s 2024-09-21T06:43:18.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:17 smithi100 ceph-mon[33401]: pgmap v561: 129 pgs: 129 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 36 MiB/s wr, 574 op/s 2024-09-21T06:43:19.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:19 smithi203 ceph-mon[33735]: pgmap v562: 129 pgs: 129 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 46 MiB/s wr, 731 op/s 2024-09-21T06:43:20.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:19 smithi067 ceph-mon[31232]: pgmap v562: 129 pgs: 129 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 46 MiB/s wr, 731 op/s 2024-09-21T06:43:20.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:19 smithi100 ceph-mon[33401]: pgmap v562: 129 pgs: 129 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 46 MiB/s wr, 731 op/s 2024-09-21T06:43:21.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:21 smithi203 ceph-mon[33735]: pgmap v563: 129 pgs: 129 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 39 MiB/s wr, 618 op/s 2024-09-21T06:43:22.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:21 smithi067 ceph-mon[31232]: pgmap v563: 129 pgs: 129 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 39 MiB/s wr, 618 op/s 2024-09-21T06:43:22.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:21 smithi100 ceph-mon[33401]: pgmap v563: 129 pgs: 129 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 39 MiB/s wr, 618 op/s 2024-09-21T06:43:23.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:23 smithi203 ceph-mon[33735]: pgmap v564: 129 pgs: 129 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 51 MiB/s wr, 812 op/s 2024-09-21T06:43:24.043 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:23 smithi067 ceph-mon[31232]: pgmap v564: 129 pgs: 129 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 51 MiB/s wr, 812 op/s 2024-09-21T06:43:24.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:23 smithi100 ceph-mon[33401]: pgmap v564: 129 pgs: 129 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 51 MiB/s wr, 812 op/s 2024-09-21T06:43:25.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:24 smithi067 ceph-mon[31232]: pgmap v565: 129 pgs: 129 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 42 MiB/s wr, 673 op/s 2024-09-21T06:43:25.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:24 smithi100 ceph-mon[33401]: pgmap v565: 129 pgs: 129 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 42 MiB/s wr, 673 op/s 2024-09-21T06:43:25.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:24 smithi203 ceph-mon[33735]: pgmap v565: 129 pgs: 129 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 42 MiB/s wr, 673 op/s 2024-09-21T06:43:27.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:27 smithi203 ceph-mon[33735]: pgmap v566: 129 pgs: 129 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 42 MiB/s wr, 665 op/s 2024-09-21T06:43:27.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:27 smithi067 ceph-mon[31232]: pgmap v566: 129 pgs: 129 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 42 MiB/s wr, 665 op/s 2024-09-21T06:43:27.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:27 smithi100 ceph-mon[33401]: pgmap v566: 129 pgs: 129 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 42 MiB/s wr, 665 op/s 2024-09-21T06:43:29.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:29 smithi203 ceph-mon[33735]: pgmap v567: 129 pgs: 129 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 50 MiB/s wr, 806 op/s 2024-09-21T06:43:29.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:29 smithi067 ceph-mon[31232]: pgmap v567: 129 pgs: 129 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 50 MiB/s wr, 806 op/s 2024-09-21T06:43:29.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:29 smithi100 ceph-mon[33401]: pgmap v567: 129 pgs: 129 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 50 MiB/s wr, 806 op/s 2024-09-21T06:43:31.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:31 smithi203 ceph-mon[33735]: pgmap v568: 129 pgs: 129 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 39 MiB/s wr, 623 op/s 2024-09-21T06:43:31.541 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:31 smithi067 ceph-mon[31232]: pgmap v568: 129 pgs: 129 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 39 MiB/s wr, 623 op/s 2024-09-21T06:43:31.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:31 smithi100 ceph-mon[33401]: pgmap v568: 129 pgs: 129 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 39 MiB/s wr, 623 op/s 2024-09-21T06:43:33.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:33 smithi203 ceph-mon[33735]: pgmap v569: 129 pgs: 129 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 51 MiB/s wr, 810 op/s 2024-09-21T06:43:33.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:33 smithi067 ceph-mon[31232]: pgmap v569: 129 pgs: 129 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 51 MiB/s wr, 810 op/s 2024-09-21T06:43:33.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:33 smithi100 ceph-mon[33401]: pgmap v569: 129 pgs: 129 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 51 MiB/s wr, 810 op/s 2024-09-21T06:43:35.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:35 smithi203 ceph-mon[33735]: pgmap v570: 129 pgs: 129 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 41 MiB/s wr, 659 op/s 2024-09-21T06:43:35.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:35 smithi067 ceph-mon[31232]: pgmap v570: 129 pgs: 129 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 41 MiB/s wr, 659 op/s 2024-09-21T06:43:35.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:35 smithi100 ceph-mon[33401]: pgmap v570: 129 pgs: 129 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 41 MiB/s wr, 659 op/s 2024-09-21T06:43:37.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:37 smithi203 ceph-mon[33735]: pgmap v571: 129 pgs: 129 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 40 MiB/s wr, 645 op/s 2024-09-21T06:43:37.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:37 smithi067 ceph-mon[31232]: pgmap v571: 129 pgs: 129 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 40 MiB/s wr, 645 op/s 2024-09-21T06:43:37.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:37 smithi100 ceph-mon[33401]: pgmap v571: 129 pgs: 129 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 40 MiB/s wr, 645 op/s 2024-09-21T06:43:39.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:39 smithi203 ceph-mon[33735]: pgmap v572: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 49 MiB/s wr, 782 op/s 2024-09-21T06:43:39.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:39 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:43:39.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:39 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:39.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:39 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:39.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:39 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:39.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:39 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:39.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:39 smithi203 ceph-mon[33735]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-09-21T06:43:39.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:39 smithi203 ceph-mon[33735]: Unable to set osd_memory_target on smithi100 to 184520806: error parsing value: Value '184520806' is below minimum 939524096 2024-09-21T06:43:39.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:39 smithi067 ceph-mon[31232]: pgmap v572: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 49 MiB/s wr, 782 op/s 2024-09-21T06:43:39.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:39 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:43:39.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:39 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:39.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:39 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:39.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:39 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:39.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:39 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:39.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:39 smithi067 ceph-mon[31232]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-09-21T06:43:39.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:39 smithi067 ceph-mon[31232]: Unable to set osd_memory_target on smithi100 to 184520806: error parsing value: Value '184520806' is below minimum 939524096 2024-09-21T06:43:39.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:39 smithi100 ceph-mon[33401]: pgmap v572: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 49 MiB/s wr, 782 op/s 2024-09-21T06:43:39.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:39 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:43:39.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:39 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:39.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:39 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:39.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:39 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:39.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:39 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:39.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:39 smithi100 ceph-mon[33401]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-09-21T06:43:39.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:39 smithi100 ceph-mon[33401]: Unable to set osd_memory_target on smithi100 to 184520806: error parsing value: Value '184520806' is below minimum 939524096 2024-09-21T06:43:41.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:41 smithi203 ceph-mon[33735]: pgmap v573: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 MiB/s wr, 606 op/s 2024-09-21T06:43:41.546 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:41 smithi067 ceph-mon[31232]: pgmap v573: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 MiB/s wr, 606 op/s 2024-09-21T06:43:41.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:41 smithi100 ceph-mon[33401]: pgmap v573: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 MiB/s wr, 606 op/s 2024-09-21T06:43:42.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:42 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:42.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:42 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:42.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:42 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:42.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:42 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:42.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:42 smithi203 ceph-mon[33735]: Adjusting osd_memory_target on smithi067 to 176.0M 2024-09-21T06:43:42.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:42 smithi203 ceph-mon[33735]: Unable to set osd_memory_target on smithi067 to 184584601: error parsing value: Value '184584601' is below minimum 939524096 2024-09-21T06:43:42.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:42 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:42.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:42 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:43:42.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:42 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:43:42.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:42 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:42.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:42 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:42.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:42 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:42.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:42 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:42.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:42 smithi067 ceph-mon[31232]: Adjusting osd_memory_target on smithi067 to 176.0M 2024-09-21T06:43:42.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:42 smithi067 ceph-mon[31232]: Unable to set osd_memory_target on smithi067 to 184584601: error parsing value: Value '184584601' is below minimum 939524096 2024-09-21T06:43:42.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:42 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:42.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:42 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:43:42.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:42 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:43:42.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:42 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:42.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:42 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:42.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:42 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:42.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:42 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:42.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:42 smithi100 ceph-mon[33401]: Adjusting osd_memory_target on smithi067 to 176.0M 2024-09-21T06:43:42.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:42 smithi100 ceph-mon[33401]: Unable to set osd_memory_target on smithi067 to 184584601: error parsing value: Value '184584601' is below minimum 939524096 2024-09-21T06:43:42.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:42 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:42.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:42 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:43:42.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:42 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:43:43.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:43 smithi067 ceph-mon[31232]: pgmap v574: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 49 MiB/s wr, 786 op/s 2024-09-21T06:43:43.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:43 smithi100 ceph-mon[33401]: pgmap v574: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 49 MiB/s wr, 786 op/s 2024-09-21T06:43:43.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:43 smithi203 ceph-mon[33735]: pgmap v574: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 49 MiB/s wr, 786 op/s 2024-09-21T06:43:45.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:45 smithi067 ceph-mon[31232]: pgmap v575: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 40 MiB/s wr, 643 op/s 2024-09-21T06:43:45.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:45 smithi100 ceph-mon[33401]: pgmap v575: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 40 MiB/s wr, 643 op/s 2024-09-21T06:43:45.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:45 smithi203 ceph-mon[33735]: pgmap v575: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 40 MiB/s wr, 643 op/s 2024-09-21T06:43:47.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:47 smithi100 ceph-mon[33401]: pgmap v576: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 39 MiB/s wr, 626 op/s 2024-09-21T06:43:47.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:47 smithi203 ceph-mon[33735]: pgmap v576: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 39 MiB/s wr, 626 op/s 2024-09-21T06:43:48.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:47 smithi067 ceph-mon[31232]: pgmap v576: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 39 MiB/s wr, 626 op/s 2024-09-21T06:43:49.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:49 smithi100 ceph-mon[33401]: pgmap v577: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 47 MiB/s wr, 763 op/s 2024-09-21T06:43:49.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:49 smithi203 ceph-mon[33735]: pgmap v577: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 47 MiB/s wr, 763 op/s 2024-09-21T06:43:50.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:49 smithi067 ceph-mon[31232]: pgmap v577: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 47 MiB/s wr, 763 op/s 2024-09-21T06:43:51.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:51 smithi203 ceph-mon[33735]: pgmap v578: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 36 MiB/s wr, 595 op/s 2024-09-21T06:43:52.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:51 smithi067 ceph-mon[31232]: pgmap v578: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 36 MiB/s wr, 595 op/s 2024-09-21T06:43:52.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:51 smithi100 ceph-mon[33401]: pgmap v578: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 36 MiB/s wr, 595 op/s 2024-09-21T06:43:53.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:53 smithi203 ceph-mon[33735]: pgmap v579: 129 pgs: 129 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 48 MiB/s wr, 784 op/s 2024-09-21T06:43:54.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:53 smithi067 ceph-mon[31232]: pgmap v579: 129 pgs: 129 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 48 MiB/s wr, 784 op/s 2024-09-21T06:43:54.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:53 smithi100 ceph-mon[33401]: pgmap v579: 129 pgs: 129 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 48 MiB/s wr, 784 op/s 2024-09-21T06:43:55.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:55 smithi203 ceph-mon[33735]: pgmap v580: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 39 MiB/s wr, 646 op/s 2024-09-21T06:43:56.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:55 smithi067 ceph-mon[31232]: pgmap v580: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 39 MiB/s wr, 646 op/s 2024-09-21T06:43:56.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:55 smithi100 ceph-mon[33401]: pgmap v580: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 39 MiB/s wr, 646 op/s 2024-09-21T06:43:57.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:57 smithi203 ceph-mon[33735]: pgmap v581: 129 pgs: 129 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 38 MiB/s wr, 635 op/s 2024-09-21T06:43:58.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:57 smithi067 ceph-mon[31232]: pgmap v581: 129 pgs: 129 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 38 MiB/s wr, 635 op/s 2024-09-21T06:43:58.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:57 smithi100 ceph-mon[33401]: pgmap v581: 129 pgs: 129 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 38 MiB/s wr, 635 op/s 2024-09-21T06:43:59.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:43:59 smithi203 ceph-mon[33735]: pgmap v582: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 46 MiB/s wr, 776 op/s 2024-09-21T06:44:00.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:43:59 smithi067 ceph-mon[31232]: pgmap v582: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 46 MiB/s wr, 776 op/s 2024-09-21T06:44:00.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:43:59 smithi100 ceph-mon[33401]: pgmap v582: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 46 MiB/s wr, 776 op/s 2024-09-21T06:44:01.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:01 smithi203 ceph-mon[33735]: pgmap v583: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 36 MiB/s wr, 610 op/s 2024-09-21T06:44:02.044 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:01 smithi067 ceph-mon[31232]: pgmap v583: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 36 MiB/s wr, 610 op/s 2024-09-21T06:44:02.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:01 smithi100 ceph-mon[33401]: pgmap v583: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 36 MiB/s wr, 610 op/s 2024-09-21T06:44:03.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:03 smithi203 ceph-mon[33735]: pgmap v584: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 47 MiB/s wr, 792 op/s 2024-09-21T06:44:04.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:03 smithi067 ceph-mon[31232]: pgmap v584: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 47 MiB/s wr, 792 op/s 2024-09-21T06:44:04.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:03 smithi100 ceph-mon[33401]: pgmap v584: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 47 MiB/s wr, 792 op/s 2024-09-21T06:44:05.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:05 smithi203 ceph-mon[33735]: pgmap v585: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 38 MiB/s wr, 652 op/s 2024-09-21T06:44:06.043 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:05 smithi067 ceph-mon[31232]: pgmap v585: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 38 MiB/s wr, 652 op/s 2024-09-21T06:44:06.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:05 smithi100 ceph-mon[33401]: pgmap v585: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 38 MiB/s wr, 652 op/s 2024-09-21T06:44:07.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:07 smithi203 ceph-mon[33735]: pgmap v586: 129 pgs: 129 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 38 MiB/s wr, 642 op/s 2024-09-21T06:44:08.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:07 smithi067 ceph-mon[31232]: pgmap v586: 129 pgs: 129 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 38 MiB/s wr, 642 op/s 2024-09-21T06:44:08.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:07 smithi100 ceph-mon[33401]: pgmap v586: 129 pgs: 129 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 38 MiB/s wr, 642 op/s 2024-09-21T06:44:09.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:09 smithi203 ceph-mon[33735]: pgmap v587: 129 pgs: 129 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 45 MiB/s wr, 785 op/s 2024-09-21T06:44:10.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:09 smithi067 ceph-mon[31232]: pgmap v587: 129 pgs: 129 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 45 MiB/s wr, 785 op/s 2024-09-21T06:44:10.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:09 smithi100 ceph-mon[33401]: pgmap v587: 129 pgs: 129 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 45 MiB/s wr, 785 op/s 2024-09-21T06:44:11.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:11 smithi203 ceph-mon[33735]: pgmap v588: 129 pgs: 129 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 35 MiB/s wr, 617 op/s 2024-09-21T06:44:12.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:11 smithi067 ceph-mon[31232]: pgmap v588: 129 pgs: 129 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 35 MiB/s wr, 617 op/s 2024-09-21T06:44:12.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:11 smithi100 ceph-mon[33401]: pgmap v588: 129 pgs: 129 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 35 MiB/s wr, 617 op/s 2024-09-21T06:44:13.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:13 smithi203 ceph-mon[33735]: pgmap v589: 129 pgs: 129 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 46 MiB/s wr, 808 op/s 2024-09-21T06:44:14.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:13 smithi067 ceph-mon[31232]: pgmap v589: 129 pgs: 129 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 46 MiB/s wr, 808 op/s 2024-09-21T06:44:14.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:13 smithi100 ceph-mon[33401]: pgmap v589: 129 pgs: 129 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 46 MiB/s wr, 808 op/s 2024-09-21T06:44:14.880 INFO:tasks.quiescer.fs.[cephfs]:Total duration (42.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-21T06:44:14.881 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.05 2024-09-21T06:44:14.881 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:44:15.765 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:44:15.931 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:15 smithi067 ceph-mon[31232]: pgmap v590: 129 pgs: 129 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 38 MiB/s wr, 671 op/s 2024-09-21T06:44:15.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:15 smithi203 ceph-mon[33735]: pgmap v590: 129 pgs: 129 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 38 MiB/s wr, 671 op/s 2024-09-21T06:44:16.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:15 smithi100 ceph-mon[33401]: pgmap v590: 129 pgs: 129 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 38 MiB/s wr, 671 op/s 2024-09-21T06:44:17.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:17 smithi203 ceph-mon[33735]: pgmap v591: 129 pgs: 129 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 36 MiB/s wr, 652 op/s 2024-09-21T06:44:18.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:17 smithi067 ceph-mon[31232]: pgmap v591: 129 pgs: 129 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 36 MiB/s wr, 652 op/s 2024-09-21T06:44:18.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:17 smithi100 ceph-mon[33401]: pgmap v591: 129 pgs: 129 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 36 MiB/s wr, 652 op/s 2024-09-21T06:44:19.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:19 smithi203 ceph-mon[33735]: pgmap v592: 129 pgs: 129 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 44 MiB/s wr, 787 op/s 2024-09-21T06:44:20.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:19 smithi067 ceph-mon[31232]: pgmap v592: 129 pgs: 129 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 44 MiB/s wr, 787 op/s 2024-09-21T06:44:20.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:19 smithi100 ceph-mon[33401]: pgmap v592: 129 pgs: 129 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 44 MiB/s wr, 787 op/s 2024-09-21T06:44:21.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:21 smithi203 ceph-mon[33735]: pgmap v593: 129 pgs: 129 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 33 MiB/s wr, 610 op/s 2024-09-21T06:44:22.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:21 smithi067 ceph-mon[31232]: pgmap v593: 129 pgs: 129 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 33 MiB/s wr, 610 op/s 2024-09-21T06:44:22.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:21 smithi100 ceph-mon[33401]: pgmap v593: 129 pgs: 129 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 33 MiB/s wr, 610 op/s 2024-09-21T06:44:23.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:23 smithi203 ceph-mon[33735]: pgmap v594: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 42 MiB/s wr, 770 op/s 2024-09-21T06:44:24.043 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:23 smithi067 ceph-mon[31232]: pgmap v594: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 42 MiB/s wr, 770 op/s 2024-09-21T06:44:24.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:23 smithi100 ceph-mon[33401]: pgmap v594: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 42 MiB/s wr, 770 op/s 2024-09-21T06:44:24.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.093+0000 7f363c3ce640 1 -- 172.21.15.67:0/3387365174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3634114460 msgr2=0x7f3634116930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:24.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.093+0000 7f363c3ce640 1 --2- 172.21.15.67:0/3387365174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3634114460 0x7f3634116930 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3624009be0 tx=0x7f362402a710 comp rx=0 tx=0).stop 2024-09-21T06:44:24.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.094+0000 7f363c3ce640 1 -- 172.21.15.67:0/3387365174 shutdown_connections 2024-09-21T06:44:24.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.094+0000 7f363c3ce640 1 --2- 172.21.15.67:0/3387365174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3634114460 0x7f3634116930 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:24.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.094+0000 7f363c3ce640 1 --2- 172.21.15.67:0/3387365174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3634075dd0 0x7f36340761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:24.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.094+0000 7f363c3ce640 1 --2- 172.21.15.67:0/3387365174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3634077380 0x7f36340757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:24.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.094+0000 7f363c3ce640 1 -- 172.21.15.67:0/3387365174 >> 172.21.15.67:0/3387365174 conn(0x7f36340ff600 msgr2=0x7f3634101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:24.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.094+0000 7f363c3ce640 1 -- 172.21.15.67:0/3387365174 shutdown_connections 2024-09-21T06:44:24.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.094+0000 7f363c3ce640 1 -- 172.21.15.67:0/3387365174 wait complete. 2024-09-21T06:44:24.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.096+0000 7f363c3ce640 1 Processor -- start 2024-09-21T06:44:24.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.096+0000 7f363c3ce640 1 -- start start 2024-09-21T06:44:24.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.096+0000 7f363c3ce640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3634075dd0 0x7f3634074ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:24.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.096+0000 7f363c3ce640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3634077380 0x7f363406f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:24.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.096+0000 7f363c3ce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3634114460 0x7f363406f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:24.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.096+0000 7f363c3ce640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f363406d540 con 0x7f3634075dd0 2024-09-21T06:44:24.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.096+0000 7f363c3ce640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f363406d3c0 con 0x7f3634114460 2024-09-21T06:44:24.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.096+0000 7f363c3ce640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f363406d6c0 con 0x7f3634077380 2024-09-21T06:44:24.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.097+0000 7f3639969640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3634077380 0x7f363406f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:24.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.097+0000 7f3639969640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3634077380 0x7f363406f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.67:59422/0 (socket says 172.21.15.67:59422) 2024-09-21T06:44:24.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.097+0000 7f3639969640 1 -- 172.21.15.67:0/2424082217 learned_addr learned my addr 172.21.15.67:0/2424082217 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:44:24.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.097+0000 7f363a16a640 1 --2- 172.21.15.67:0/2424082217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3634075dd0 0x7f3634074ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:24.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.097+0000 7f363a96b640 1 --2- 172.21.15.67:0/2424082217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3634114460 0x7f363406f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:24.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.098+0000 7f3639969640 1 -- 172.21.15.67:0/2424082217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3634114460 msgr2=0x7f363406f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:24.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.098+0000 7f3639969640 1 --2- 172.21.15.67:0/2424082217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3634114460 0x7f363406f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:24.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.098+0000 7f3639969640 1 -- 172.21.15.67:0/2424082217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3634075dd0 msgr2=0x7f3634074ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:24.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.098+0000 7f3639969640 1 --2- 172.21.15.67:0/2424082217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3634075dd0 0x7f3634074ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:24.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.098+0000 7f3639969640 1 -- 172.21.15.67:0/2424082217 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f363406fe50 con 0x7f3634077380 2024-09-21T06:44:24.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.099+0000 7f3639969640 1 --2- 172.21.15.67:0/2424082217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3634077380 0x7f363406f190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f362800d770 tx=0x7f362800db70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:24.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.101+0000 7f36337fe640 1 -- 172.21.15.67:0/2424082217 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3628017070 con 0x7f3634077380 2024-09-21T06:44:24.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.101+0000 7f36337fe640 1 -- 172.21.15.67:0/2424082217 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3628010460 con 0x7f3634077380 2024-09-21T06:44:24.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.102+0000 7f36337fe640 1 -- 172.21.15.67:0/2424082217 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362800f620 con 0x7f3634077380 2024-09-21T06:44:24.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.102+0000 7f363c3ce640 1 -- 172.21.15.67:0/2424082217 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36341b5530 con 0x7f3634077380 2024-09-21T06:44:24.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.102+0000 7f363c3ce640 1 -- 172.21.15.67:0/2424082217 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36341b59c0 con 0x7f3634077380 2024-09-21T06:44:24.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.106+0000 7f363c3ce640 1 -- 172.21.15.67:0/2424082217 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3634076cc0 con 0x7f3634077380 2024-09-21T06:44:24.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.111+0000 7f36337fe640 1 -- 172.21.15.67:0/2424082217 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f362801a070 con 0x7f3634077380 2024-09-21T06:44:24.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.113+0000 7f36337fe640 1 --2- 172.21.15.67:0/2424082217 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f361c0b6720 0x7f361c0b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:24.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.113+0000 7f363a16a640 1 --2- 172.21.15.67:0/2424082217 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f361c0b6720 0x7f361c0b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:24.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.114+0000 7f363a16a640 1 --2- 172.21.15.67:0/2424082217 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f361c0b6720 0x7f361c0b8c30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3634074c20 tx=0x7f362c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:24.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.114+0000 7f36337fe640 1 -- 172.21.15.67:0/2424082217 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f36280e7020 con 0x7f3634077380 2024-09-21T06:44:24.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.119+0000 7f36337fe640 1 -- 172.21.15.67:0/2424082217 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f36280ad480 con 0x7f3634077380 2024-09-21T06:44:24.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.580+0000 7f363c3ce640 1 -- 172.21.15.67:0/2424082217 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36340761e0 con 0x7f3634077380 2024-09-21T06:44:24.586 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:44:24.586 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:44:24.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.580+0000 7f36337fe640 1 -- 172.21.15.67:0/2424082217 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7f36280b1860 con 0x7f3634077380 2024-09-21T06:44:24.587 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:44:24.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.584+0000 7f363c3ce640 1 -- 172.21.15.67:0/2424082217 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f361c0b6720 msgr2=0x7f361c0b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:24.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.584+0000 7f363c3ce640 1 --2- 172.21.15.67:0/2424082217 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f361c0b6720 0x7f361c0b8c30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3634074c20 tx=0x7f362c009260 comp rx=0 tx=0).stop 2024-09-21T06:44:24.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.584+0000 7f363c3ce640 1 -- 172.21.15.67:0/2424082217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3634077380 msgr2=0x7f363406f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:24.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.584+0000 7f363c3ce640 1 --2- 172.21.15.67:0/2424082217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3634077380 0x7f363406f190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f362800d770 tx=0x7f362800db70 comp rx=0 tx=0).stop 2024-09-21T06:44:24.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.586+0000 7f363c3ce640 1 -- 172.21.15.67:0/2424082217 shutdown_connections 2024-09-21T06:44:24.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.586+0000 7f363c3ce640 1 --2- 172.21.15.67:0/2424082217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3634114460 0x7f363406f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:24.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.586+0000 7f363c3ce640 1 --2- 172.21.15.67:0/2424082217 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f361c0b6720 0x7f361c0b8c30 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:24.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.586+0000 7f363c3ce640 1 --2- 172.21.15.67:0/2424082217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3634077380 0x7f363406f190 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:24.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.586+0000 7f363c3ce640 1 --2- 172.21.15.67:0/2424082217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3634075dd0 0x7f3634074ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:24.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.586+0000 7f363c3ce640 1 -- 172.21.15.67:0/2424082217 >> 172.21.15.67:0/2424082217 conn(0x7f36340ff600 msgr2=0x7f36340ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:24.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.588+0000 7f363c3ce640 1 -- 172.21.15.67:0/2424082217 shutdown_connections 2024-09-21T06:44:24.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:24.592+0000 7f363c3ce640 1 -- 172.21.15.67:0/2424082217 wait complete. 2024-09-21T06:44:24.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:24 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2424082217' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:44:25.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:24 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2424082217' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:44:25.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:24 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2424082217' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:44:25.682 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db / --timeout 90 --expiration 141.05 --await-for 45.0' 2024-09-21T06:44:25.683 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24671 quiesce db / --timeout 90 --expiration 141.05 --await-for 45.0 2024-09-21T06:44:25.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:25 smithi067 ceph-mon[31232]: pgmap v595: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 33 MiB/s wr, 613 op/s 2024-09-21T06:44:25.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:25 smithi203 ceph-mon[33735]: pgmap v595: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 33 MiB/s wr, 613 op/s 2024-09-21T06:44:26.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:25 smithi100 ceph-mon[33401]: pgmap v595: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 33 MiB/s wr, 613 op/s 2024-09-21T06:44:26.321 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:44:27.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.120+0000 7fc6f8c1d640 1 -- 172.21.15.67:0/3700634950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6f4108d10 msgr2=0x7fc6f410f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:27.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.120+0000 7fc6f8c1d640 1 --2- 172.21.15.67:0/3700634950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6f4108d10 0x7fc6f410f540 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8009900 tx=0x7fc6e802a410 comp rx=0 tx=0).stop 2024-09-21T06:44:27.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.124+0000 7fc6f8c1d640 1 -- 172.21.15.67:0/3700634950 shutdown_connections 2024-09-21T06:44:27.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.124+0000 7fc6f8c1d640 1 --2- 172.21.15.67:0/3700634950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6f4108d10 0x7fc6f410f540 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.124+0000 7fc6f8c1d640 1 --2- 172.21.15.67:0/3700634950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6f41083c0 0x7fc6f41087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.124+0000 7fc6f8c1d640 1 --2- 172.21.15.67:0/3700634950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6f4102a80 0x7fc6f4102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.124+0000 7fc6f8c1d640 1 -- 172.21.15.67:0/3700634950 >> 172.21.15.67:0/3700634950 conn(0x7fc6f4079760 msgr2=0x7fc6f4079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:27.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.127+0000 7fc6f8c1d640 1 -- 172.21.15.67:0/3700634950 shutdown_connections 2024-09-21T06:44:27.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.128+0000 7fc6f8c1d640 1 -- 172.21.15.67:0/3700634950 wait complete. 2024-09-21T06:44:27.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.129+0000 7fc6f8c1d640 1 Processor -- start 2024-09-21T06:44:27.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.129+0000 7fc6f8c1d640 1 -- start start 2024-09-21T06:44:27.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.130+0000 7fc6f8c1d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6f4102a80 0x7fc6f41a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:27.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.130+0000 7fc6f8c1d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6f41083c0 0x7fc6f41a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:27.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.130+0000 7fc6f8c1d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6f4108d10 0x7fc6f41a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:27.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.130+0000 7fc6f8c1d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f4112890 con 0x7fc6f41083c0 2024-09-21T06:44:27.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.130+0000 7fc6f2d9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6f4108d10 0x7fc6f41a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:27.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.130+0000 7fc6f2d9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6f4108d10 0x7fc6f41a6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.67:59448/0 (socket says 172.21.15.67:59448) 2024-09-21T06:44:27.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.130+0000 7fc6f2d9d640 1 -- 172.21.15.67:0/2342222525 learned_addr learned my addr 172.21.15.67:0/2342222525 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:44:27.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.130+0000 7fc6f8c1d640 1 -- 172.21.15.67:0/2342222525 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f4112710 con 0x7fc6f4102a80 2024-09-21T06:44:27.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.130+0000 7fc6f8c1d640 1 -- 172.21.15.67:0/2342222525 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f4112a10 con 0x7fc6f4108d10 2024-09-21T06:44:27.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.131+0000 7fc6f2d9d640 1 -- 172.21.15.67:0/2342222525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6f4102a80 msgr2=0x7fc6f41a0940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:44:27.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.131+0000 7fc6f2d9d640 1 --2- 172.21.15.67:0/2342222525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6f4102a80 0x7fc6f41a0940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.131+0000 7fc6f2d9d640 1 -- 172.21.15.67:0/2342222525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6f41083c0 msgr2=0x7fc6f41a0ee0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:44:27.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.131+0000 7fc6f2d9d640 1 --2- 172.21.15.67:0/2342222525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6f41083c0 0x7fc6f41a0ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.131+0000 7fc6f2d9d640 1 -- 172.21.15.67:0/2342222525 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f41a68b0 con 0x7fc6f4108d10 2024-09-21T06:44:27.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.132+0000 7fc6f2d9d640 1 --2- 172.21.15.67:0/2342222525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6f4108d10 0x7fc6f41a6180 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc6e802a980 tx=0x7fc6e8003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:27.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.133+0000 7fc6e37fe640 1 -- 172.21.15.67:0/2342222525 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e803f070 con 0x7fc6f4108d10 2024-09-21T06:44:27.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.133+0000 7fc6f8c1d640 1 -- 172.21.15.67:0/2342222525 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6f41a6b40 con 0x7fc6f4108d10 2024-09-21T06:44:27.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.133+0000 7fc6f8c1d640 1 -- 172.21.15.67:0/2342222525 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6f41ad300 con 0x7fc6f4108d10 2024-09-21T06:44:27.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.133+0000 7fc6e37fe640 1 -- 172.21.15.67:0/2342222525 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6e803baf0 con 0x7fc6f4108d10 2024-09-21T06:44:27.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.133+0000 7fc6e37fe640 1 -- 172.21.15.67:0/2342222525 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8035030 con 0x7fc6f4108d10 2024-09-21T06:44:27.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.137+0000 7fc6e37fe640 1 -- 172.21.15.67:0/2342222525 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fc6e8042070 con 0x7fc6f4108d10 2024-09-21T06:44:27.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.139+0000 7fc6e37fe640 1 --2- 172.21.15.67:0/2342222525 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc6dc0b6720 0x7fc6dc0b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:27.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.139+0000 7fc6e37fe640 1 -- 172.21.15.67:0/2342222525 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fc6e8107d10 con 0x7fc6f4108d10 2024-09-21T06:44:27.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.141+0000 7fc6f259c640 1 --2- 172.21.15.67:0/2342222525 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc6dc0b6720 0x7fc6dc0b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:27.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.142+0000 7fc6f259c640 1 --2- 172.21.15.67:0/2342222525 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc6dc0b6720 0x7fc6dc0b8c30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec00ab90 tx=0x7fc6ec009250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:27.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.168+0000 7fc6ff191740 1 Processor -- start 2024-09-21T06:44:27.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.168+0000 7fc6ff191740 1 -- start start 2024-09-21T06:44:27.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.169+0000 7fc6ff191740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1681c2850 0x55c168568e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:27.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.169+0000 7fc6ff191740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c168199990 0x55c1685693b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:27.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.169+0000 7fc6f259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1681c2850 0x55c168568e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:27.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.169+0000 7fc6f259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1681c2850 0x55c168568e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.67:57768/0 (socket says 172.21.15.67:57768) 2024-09-21T06:44:27.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.169+0000 7fc6ff191740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1681a1b70 0x55c1685698f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:27.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.169+0000 7fc6ff191740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c1684dfbb0 con 0x55c168199990 2024-09-21T06:44:27.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.169+0000 7fc6ff191740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c1684cefe0 con 0x55c1681c2850 2024-09-21T06:44:27.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.169+0000 7fc6ff191740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c1684d2df0 con 0x55c1681a1b70 2024-09-21T06:44:27.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.169+0000 7fc6f259c640 1 -- 172.21.15.67:0/122282139 learned_addr learned my addr 172.21.15.67:0/122282139 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:44:27.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.170+0000 7fc6f1d9b640 1 --2- 172.21.15.67:0/122282139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c168199990 0x55c1685693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:27.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.171+0000 7fc6f259c640 1 -- 172.21.15.67:0/122282139 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1681a1b70 msgr2=0x55c1685698f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:44:27.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.171+0000 7fc6f259c640 1 --2- 172.21.15.67:0/122282139 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1681a1b70 0x55c1685698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.171+0000 7fc6f259c640 1 -- 172.21.15.67:0/122282139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c168199990 msgr2=0x55c1685693b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:44:27.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.171+0000 7fc6f259c640 1 --2- 172.21.15.67:0/122282139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c168199990 0x55c1685693b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.171+0000 7fc6f259c640 1 -- 172.21.15.67:0/122282139 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c16823cba0 con 0x55c1681c2850 2024-09-21T06:44:27.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.171+0000 7fc6f259c640 1 --2- 172.21.15.67:0/122282139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1681c2850 0x55c168568e70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec009710 tx=0x7fc6ec00f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bc3c311736a1ec1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:27.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.174+0000 7fc6e0ff9640 1 -- 172.21.15.67:0/122282139 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec019420 con 0x55c1681c2850 2024-09-21T06:44:27.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.174+0000 7fc6e0ff9640 1 -- 172.21.15.67:0/122282139 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6ec019a50 con 0x55c1681c2850 2024-09-21T06:44:27.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.174+0000 7fc6e0ff9640 1 -- 172.21.15.67:0/122282139 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec021ad0 con 0x55c1681c2850 2024-09-21T06:44:27.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.175+0000 7fc6ff191740 1 -- 172.21.15.67:0/122282139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1681c2850 msgr2=0x55c168568e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:27.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.175+0000 7fc6ff191740 1 --2- 172.21.15.67:0/122282139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1681c2850 0x55c168568e70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec009710 tx=0x7fc6ec00f000 comp rx=0 tx=0).stop 2024-09-21T06:44:27.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.176+0000 7fc6ff191740 1 -- 172.21.15.67:0/122282139 shutdown_connections 2024-09-21T06:44:27.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.176+0000 7fc6ff191740 1 --2- 172.21.15.67:0/122282139 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1681a1b70 0x55c1685698f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.176+0000 7fc6ff191740 1 --2- 172.21.15.67:0/122282139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c168199990 0x55c1685693b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.176+0000 7fc6ff191740 1 --2- 172.21.15.67:0/122282139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1681c2850 0x55c168568e70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.176+0000 7fc6ff191740 1 -- 172.21.15.67:0/122282139 >> 172.21.15.67:0/122282139 conn(0x55c1681b6c40 msgr2=0x55c168566d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:27.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.177+0000 7fc6ff191740 1 -- 172.21.15.67:0/122282139 shutdown_connections 2024-09-21T06:44:27.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.177+0000 7fc6ff191740 1 -- 172.21.15.67:0/122282139 wait complete. 2024-09-21T06:44:27.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.178+0000 7fc6ff191740 1 -- start start 2024-09-21T06:44:27.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.178+0000 7fc6ff191740 1 Processor -- start 2024-09-21T06:44:27.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.179+0000 7fc6ff191740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c168199990 0x55c168574b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:27.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.179+0000 7fc6ff191740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1681a1b70 0x55c1685750d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:27.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.179+0000 7fc6f259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c168199990 0x55c168574b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:27.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.179+0000 7fc6f259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c168199990 0x55c168574b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.67:57778/0 (socket says 172.21.15.67:57778) 2024-09-21T06:44:27.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.179+0000 7fc6f259c640 1 -- 172.21.15.67:0/3278310304 learned_addr learned my addr 172.21.15.67:0/3278310304 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:44:27.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.179+0000 7fc6f1d9b640 1 --2- 172.21.15.67:0/3278310304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1681a1b70 0x55c1685750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:27.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.180+0000 7fc6ff191740 1 --2- 172.21.15.67:0/3278310304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1681c2850 0x55c1685809d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:27.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.180+0000 7fc6ff191740 1 -- 172.21.15.67:0/3278310304 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c1683307b0 con 0x55c1681c2850 2024-09-21T06:44:27.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.180+0000 7fc6ff191740 1 -- 172.21.15.67:0/3278310304 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c168469e90 con 0x55c168199990 2024-09-21T06:44:27.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.180+0000 7fc6ff191740 1 -- 172.21.15.67:0/3278310304 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c168577650 con 0x55c1681a1b70 2024-09-21T06:44:27.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.180+0000 7fc6ff191740 10 client.0 _ll_register_callbacks cb 0x55c1684cc710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:44:27.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.181+0000 7fc6f2d9d640 1 --2- 172.21.15.67:0/3278310304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1681c2850 0x55c1685809d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:27.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.182+0000 7fc6f2d9d640 1 -- 172.21.15.67:0/3278310304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1681a1b70 msgr2=0x55c1685750d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:44:27.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.182+0000 7fc6f2d9d640 1 --2- 172.21.15.67:0/3278310304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1681a1b70 0x55c1685750d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.182+0000 7fc6f2d9d640 1 -- 172.21.15.67:0/3278310304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c168199990 msgr2=0x55c168574b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:44:27.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.182+0000 7fc6f2d9d640 1 --2- 172.21.15.67:0/3278310304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c168199990 0x55c168574b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.182+0000 7fc6f2d9d640 1 -- 172.21.15.67:0/3278310304 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c168581040 con 0x55c1681c2850 2024-09-21T06:44:27.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.182+0000 7fc6f2d9d640 1 --2- 172.21.15.67:0/3278310304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1681c2850 0x55c1685809d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc6e81053e0 tx=0x7fc6e80cd480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d9911eb22a4a2b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:27.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.182+0000 7fc6d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:44:27.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.183+0000 7fc6d9ffb640 1 -- 172.21.15.67:0/3278310304 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e80c9050 con 0x55c1681c2850 2024-09-21T06:44:27.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.183+0000 7fc6ff191740 1 -- 172.21.15.67:0/3278310304 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6ec019a50 con 0x55c1681c2850 2024-09-21T06:44:27.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.183+0000 7fc6d9ffb640 1 -- 172.21.15.67:0/3278310304 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fc6e80c9ce0 con 0x55c1681c2850 2024-09-21T06:44:27.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.184+0000 7fc6d9ffb640 1 -- 172.21.15.67:0/3278310304 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc6e8049040 con 0x55c1681c2850 2024-09-21T06:44:27.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.184+0000 7fc6ff191740 10 client.15765 fetch_fsmap learned FSMap version 21 2024-09-21T06:44:27.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.184+0000 7fc6ff191740 1 -- 172.21.15.67:0/3278310304 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c168583910 con 0x55c1681c2850 2024-09-21T06:44:27.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.186+0000 7fc6d9ffb640 1 -- 172.21.15.67:0/3278310304 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6ec019a50 con 0x55c1681c2850 2024-09-21T06:44:27.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.186+0000 7fc6d9ffb640 1 -- 172.21.15.67:0/3278310304 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc6e80b8060 con 0x55c1681c2850 2024-09-21T06:44:27.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.186+0000 7fc6ff191740 10 client.15765 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:44:27.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.186+0000 7fc6ff191740 10 client.15765 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:44:27.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.186+0000 7fc6ff191740 20 client.15765 populate_metadata read hostname 'smithi067' 2024-09-21T06:44:27.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.186+0000 7fc6ff191740 1 --2- 172.21.15.67:0/3278310304 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55c1681811d0 0x55c168586f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:27.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.186+0000 7fc6ff191740 4 client.15765 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:44:27.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.186+0000 7fc6ff191740 1 -- 172.21.15.67:0/3278310304 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1684e1b00 con 0x55c1681811d0 2024-09-21T06:44:27.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.188+0000 7fc6f259c640 1 --2- 172.21.15.67:0/3278310304 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55c1681811d0 0x55c168586f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:27.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.190+0000 7fc6f259c640 1 --2- 172.21.15.67:0/3278310304 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55c1681811d0 0x55c168586f80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2a0e2fe16acef8d server_cookie=fda190ddfdd95078 in_seq=0 out_seq=0 2024-09-21T06:44:27.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.190+0000 7fc6d9ffb640 10 client.15765 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:44:27.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.191+0000 7fc6d9ffb640 1 -- 172.21.15.67:0/3278310304 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c1684e1b00 con 0x55c1681811d0 2024-09-21T06:44:27.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.191+0000 7fc6d9ffb640 10 client.15765 handle_command_reply: tid=0 2024-09-21T06:44:27.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.191+0000 7fc6ff191740 1 client.15765 shutdown 2024-09-21T06:44:27.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.193+0000 7fc6ff191740 1 -- 172.21.15.67:0/3278310304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1681c2850 msgr2=0x55c1685809d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:27.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.193+0000 7fc6ff191740 1 --2- 172.21.15.67:0/3278310304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1681c2850 0x55c1685809d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc6e81053e0 tx=0x7fc6e80cd480 comp rx=0 tx=0).stop 2024-09-21T06:44:27.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.194+0000 7fc6ff191740 1 -- 172.21.15.67:0/3278310304 shutdown_connections 2024-09-21T06:44:27.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.194+0000 7fc6ff191740 1 --2- 172.21.15.67:0/3278310304 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55c1681811d0 0x55c168586f80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.194+0000 7fc6ff191740 1 --2- 172.21.15.67:0/3278310304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1681c2850 0x55c1685809d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.194+0000 7fc6ff191740 1 --2- 172.21.15.67:0/3278310304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1681a1b70 0x55c1685750d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.194+0000 7fc6ff191740 1 --2- 172.21.15.67:0/3278310304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c168199990 0x55c168574b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.194+0000 7fc6ff191740 1 -- 172.21.15.67:0/3278310304 >> 172.21.15.67:0/3278310304 conn(0x55c1681b6c40 msgr2=0x55c16856c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:27.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.194+0000 7fc6d9ffb640 0 client.15765 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:44:27.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.196+0000 7fc6ff191740 1 -- 172.21.15.67:0/3278310304 shutdown_connections 2024-09-21T06:44:27.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.198+0000 7fc6ff191740 1 -- 172.21.15.67:0/3278310304 wait complete. 2024-09-21T06:44:27.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.199+0000 7fc6ff191740 20 client.15765 trim_cache size 0 max 16384 2024-09-21T06:44:27.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.235+0000 7fc6ff191740 1 Processor -- start 2024-09-21T06:44:27.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.235+0000 7fc6ff191740 1 -- start start 2024-09-21T06:44:27.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.236+0000 7fc6ff191740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c168199990 0x55c168576540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:27.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.236+0000 7fc6ff191740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1681a1b70 0x55c16856df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:27.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.236+0000 7fc6ff191740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1681c2850 0x55c16856e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:27.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.236+0000 7fc6ff191740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d400dd80 con 0x55c1681a1b70 2024-09-21T06:44:27.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.236+0000 7fc6ff191740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d4010600 con 0x55c168199990 2024-09-21T06:44:27.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.236+0000 7fc6ff191740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d400e3f0 con 0x55c1681c2850 2024-09-21T06:44:27.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.237+0000 7fc6f2d9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1681c2850 0x55c16856e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:27.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.237+0000 7fc6f2d9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1681c2850 0x55c16856e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.67:59470/0 (socket says 172.21.15.67:59470) 2024-09-21T06:44:27.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.237+0000 7fc6f2d9d640 1 -- 172.21.15.67:0/472403614 learned_addr learned my addr 172.21.15.67:0/472403614 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:44:27.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.237+0000 7fc6f1d9b640 1 --2- 172.21.15.67:0/472403614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1681a1b70 0x55c16856df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:27.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.238+0000 7fc6f2d9d640 1 -- 172.21.15.67:0/472403614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c168199990 msgr2=0x55c168576540 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:44:27.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.238+0000 7fc6f2d9d640 1 --2- 172.21.15.67:0/472403614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c168199990 0x55c168576540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.238+0000 7fc6f2d9d640 1 -- 172.21.15.67:0/472403614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1681a1b70 msgr2=0x55c16856df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:44:27.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.238+0000 7fc6f2d9d640 1 --2- 172.21.15.67:0/472403614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1681a1b70 0x55c16856df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.238+0000 7fc6f2d9d640 1 -- 172.21.15.67:0/472403614 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c168570130 con 0x55c1681c2850 2024-09-21T06:44:27.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.242+0000 7fc6f2d9d640 1 --2- 172.21.15.67:0/472403614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1681c2850 0x55c16856e440 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8006010 tx=0x7fc6e80cd4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffda283c97aab136 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:27.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.245+0000 7fc6dbfff640 1 -- 172.21.15.67:0/472403614 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e80e8070 con 0x55c1681c2850 2024-09-21T06:44:27.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.245+0000 7fc6dbfff640 1 -- 172.21.15.67:0/472403614 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6e80c9040 con 0x55c1681c2850 2024-09-21T06:44:27.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.246+0000 7fc6ff191740 1 -- 172.21.15.67:0/472403614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1681c2850 msgr2=0x55c16856e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:27.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.246+0000 7fc6ff191740 1 --2- 172.21.15.67:0/472403614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1681c2850 0x55c16856e440 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8006010 tx=0x7fc6e80cd4f0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.247+0000 7fc6ff191740 1 -- 172.21.15.67:0/472403614 shutdown_connections 2024-09-21T06:44:27.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.247+0000 7fc6ff191740 1 --2- 172.21.15.67:0/472403614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1681c2850 0x55c16856e440 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.247+0000 7fc6ff191740 1 --2- 172.21.15.67:0/472403614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1681a1b70 0x55c16856df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.247+0000 7fc6ff191740 1 --2- 172.21.15.67:0/472403614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c168199990 0x55c168576540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.247+0000 7fc6ff191740 1 -- 172.21.15.67:0/472403614 >> 172.21.15.67:0/472403614 conn(0x55c1681b6c40 msgr2=0x55c168575e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:27.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.250+0000 7fc6ff191740 1 -- 172.21.15.67:0/472403614 shutdown_connections 2024-09-21T06:44:27.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.252+0000 7fc6ff191740 1 -- 172.21.15.67:0/472403614 wait complete. 2024-09-21T06:44:27.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.253+0000 7fc6ff191740 1 -- start start 2024-09-21T06:44:27.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.253+0000 7fc6ff191740 1 Processor -- start 2024-09-21T06:44:27.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.254+0000 7fc6ff191740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c168199990 0x55c168578d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:27.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.254+0000 7fc6ff191740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1681a1b70 0x55c168579360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:27.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.254+0000 7fc6ff191740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1681c2850 0x55c168601910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:27.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.254+0000 7fc6ff191740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d400eac0 con 0x55c1681a1b70 2024-09-21T06:44:27.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.254+0000 7fc6ff191740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d400ff30 con 0x55c1681c2850 2024-09-21T06:44:27.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.254+0000 7fc6ff191740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d400f190 con 0x55c168199990 2024-09-21T06:44:27.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.254+0000 7fc6ff191740 10 client.0 _ll_register_callbacks cb 0x7fc6e8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:44:27.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.254+0000 7fc6f2d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1681c2850 0x55c168601910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:27.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.254+0000 7fc6f2d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1681c2850 0x55c168601910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.67:57794/0 (socket says 172.21.15.67:57794) 2024-09-21T06:44:27.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.254+0000 7fc6f2d9d640 1 -- 172.21.15.67:0/1244251195 learned_addr learned my addr 172.21.15.67:0/1244251195 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:44:27.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.255+0000 7fc6f2d9d640 1 -- 172.21.15.67:0/1244251195 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c168199990 msgr2=0x55c168578d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:44:27.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.255+0000 7fc6f2d9d640 1 --2- 172.21.15.67:0/1244251195 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c168199990 0x55c168578d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.255+0000 7fc6f2d9d640 1 -- 172.21.15.67:0/1244251195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1681a1b70 msgr2=0x55c168579360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:44:27.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.255+0000 7fc6f2d9d640 1 --2- 172.21.15.67:0/1244251195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1681a1b70 0x55c168579360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.255+0000 7fc6f2d9d640 1 -- 172.21.15.67:0/1244251195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c168601e50 con 0x55c1681c2850 2024-09-21T06:44:27.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.255+0000 7fc6f2d9d640 1 --2- 172.21.15.67:0/1244251195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1681c2850 0x55c168601910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8105430 tx=0x7fc6e80cd480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0b2bcdc6a521f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:27.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.255+0000 7fc6d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-21T06:44:27.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.256+0000 7fc6d9ffb640 1 -- 172.21.15.67:0/1244251195 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e80e8050 con 0x55c1681c2850 2024-09-21T06:44:27.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.256+0000 7fc6ff191740 1 -- 172.21.15.67:0/1244251195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6e80c9040 con 0x55c1681c2850 2024-09-21T06:44:27.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.256+0000 7fc6d9ffb640 1 -- 172.21.15.67:0/1244251195 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fc6e80c93c0 con 0x55c1681c2850 2024-09-21T06:44:27.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.256+0000 7fc6d9ffb640 1 -- 172.21.15.67:0/1244251195 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc6e80ce170 con 0x55c1681c2850 2024-09-21T06:44:27.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.256+0000 7fc6ff191740 10 client.25490 fetch_fsmap learned FSMap version 21 2024-09-21T06:44:27.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.256+0000 7fc6ff191740 1 -- 172.21.15.67:0/1244251195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c168601e50 con 0x55c1681c2850 2024-09-21T06:44:27.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.259+0000 7fc6d9ffb640 1 -- 172.21.15.67:0/1244251195 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6e80c9040 con 0x55c1681c2850 2024-09-21T06:44:27.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.259+0000 7fc6d9ffb640 1 -- 172.21.15.67:0/1244251195 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc6e80b8070 con 0x55c1681c2850 2024-09-21T06:44:27.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.259+0000 7fc6ff191740 10 client.25490 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:44:27.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.259+0000 7fc6ff191740 10 client.25490 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:44:27.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.259+0000 7fc6ff191740 20 client.25490 populate_metadata read hostname 'smithi067' 2024-09-21T06:44:27.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.259+0000 7fc6ff191740 1 --2- 172.21.15.67:0/1244251195 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55c168156580 0x55c168605350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:27.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.259+0000 7fc6ff191740 4 client.25490 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.05, "await_for": 45.0}] 2024-09-21T06:44:27.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.259+0000 7fc6ff191740 1 -- 172.21.15.67:0/1244251195 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.05, "await_for": 45.0}) -- 0x55c1684dd5b0 con 0x55c168156580 2024-09-21T06:44:27.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.259+0000 7fc6f259c640 1 --2- 172.21.15.67:0/1244251195 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55c168156580 0x55c168605350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:27.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.260+0000 7fc6d9ffb640 10 client.25490 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:44:27.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.261+0000 7fc6f259c640 1 --2- 172.21.15.67:0/1244251195 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55c168156580 0x55c168605350 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f3e7dbd42691573 server_cookie=6d924afc88aa9b20 in_seq=0 out_seq=0 2024-09-21T06:44:27.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.295+0000 7fc6d9ffb640 1 -- 172.21.15.67:0/1244251195 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c1684dd5b0 con 0x55c168156580 2024-09-21T06:44:27.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.295+0000 7fc6d9ffb640 10 client.25490 handle_command_reply: tid=0 2024-09-21T06:44:27.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.295+0000 7fc6ff191740 1 client.25490 shutdown 2024-09-21T06:44:27.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.296+0000 7fc6ff191740 1 -- 172.21.15.67:0/1244251195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1681c2850 msgr2=0x55c168601910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:27.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.296+0000 7fc6ff191740 1 --2- 172.21.15.67:0/1244251195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1681c2850 0x55c168601910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8105430 tx=0x7fc6e80cd480 comp rx=0 tx=0).stop 2024-09-21T06:44:27.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.296+0000 7fc6ff191740 1 -- 172.21.15.67:0/1244251195 shutdown_connections 2024-09-21T06:44:27.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.296+0000 7fc6ff191740 1 --2- 172.21.15.67:0/1244251195 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55c168156580 0x55c168605350 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.296+0000 7fc6ff191740 1 --2- 172.21.15.67:0/1244251195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1681c2850 0x55c168601910 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.296+0000 7fc6ff191740 1 --2- 172.21.15.67:0/1244251195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1681a1b70 0x55c168579360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.296+0000 7fc6ff191740 1 --2- 172.21.15.67:0/1244251195 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c168199990 0x55c168578d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.296+0000 7fc6d9ffb640 0 client.25490 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:44:27.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.296+0000 7fc6ff191740 1 -- 172.21.15.67:0/1244251195 >> 172.21.15.67:0/1244251195 conn(0x55c1681b6c40 msgr2=0x55c168575dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:27.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.296+0000 7fc6ff191740 1 -- 172.21.15.67:0/1244251195 shutdown_connections 2024-09-21T06:44:27.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.296+0000 7fc6ff191740 1 -- 172.21.15.67:0/1244251195 wait complete. 2024-09-21T06:44:27.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.296+0000 7fc6ff191740 20 client.25490 trim_cache size 0 max 16384 2024-09-21T06:44:27.300 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-21T06:44:27.300 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 21, 2024-09-21T06:44:27.300 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-09-21T06:44:27.300 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 30, 2024-09-21T06:44:27.300 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-21T06:44:27.300 INFO:teuthology.orchestra.run.smithi067.stdout: "45924b04": { 2024-09-21T06:44:27.301 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 30, 2024-09-21T06:44:27.301 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-21T06:44:27.301 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:44:27.301 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-21T06:44:27.301 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:44:27.301 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:44:27.301 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-21T06:44:27.301 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 141.1, 2024-09-21T06:44:27.301 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-21T06:44:27.301 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-21T06:44:27.301 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-21T06:44:27.301 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:44:27.301 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-21T06:44:27.301 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:44:27.301 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:44:27.302 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:44:27.302 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:44:27.302 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:44:27.302 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:44:27.302 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-21T06:44:27.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.297+0000 7fc6f8c1d640 1 -- 172.21.15.67:0/2342222525 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc6dc0b6720 msgr2=0x7fc6dc0b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:27.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.297+0000 7fc6f8c1d640 1 --2- 172.21.15.67:0/2342222525 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc6dc0b6720 0x7fc6dc0b8c30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec00ab90 tx=0x7fc6ec009250 comp rx=0 tx=0).stop 2024-09-21T06:44:27.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.297+0000 7fc6f8c1d640 1 -- 172.21.15.67:0/2342222525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6f4108d10 msgr2=0x7fc6f41a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:27.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.297+0000 7fc6f8c1d640 1 --2- 172.21.15.67:0/2342222525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6f4108d10 0x7fc6f41a6180 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc6e802a980 tx=0x7fc6e8003b90 comp rx=0 tx=0).stop 2024-09-21T06:44:27.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.297+0000 7fc6f8c1d640 1 -- 172.21.15.67:0/2342222525 shutdown_connections 2024-09-21T06:44:27.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.297+0000 7fc6f8c1d640 1 --2- 172.21.15.67:0/2342222525 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc6dc0b6720 0x7fc6dc0b8c30 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.297+0000 7fc6f8c1d640 1 --2- 172.21.15.67:0/2342222525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6f4108d10 0x7fc6f41a6180 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.297+0000 7fc6f8c1d640 1 --2- 172.21.15.67:0/2342222525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6f41083c0 0x7fc6f41a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.297+0000 7fc6f8c1d640 1 --2- 172.21.15.67:0/2342222525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6f4102a80 0x7fc6f41a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:27.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.297+0000 7fc6f8c1d640 1 -- 172.21.15.67:0/2342222525 >> 172.21.15.67:0/2342222525 conn(0x7fc6f4079760 msgr2=0x7fc6f406b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:27.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.297+0000 7fc6f8c1d640 1 -- 172.21.15.67:0/2342222525 shutdown_connections 2024-09-21T06:44:27.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:27.297+0000 7fc6f8c1d640 1 -- 172.21.15.67:0/2342222525 wait complete. 2024-09-21T06:44:27.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:27 smithi203 ceph-mon[33735]: pgmap v596: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 31 MiB/s wr, 588 op/s 2024-09-21T06:44:28.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:27 smithi067 ceph-mon[31232]: pgmap v596: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 31 MiB/s wr, 588 op/s 2024-09-21T06:44:28.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:27 smithi100 ceph-mon[33401]: pgmap v596: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 31 MiB/s wr, 588 op/s 2024-09-21T06:44:28.908 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '45924b04', quiesce took 14.0 seconds. Will release after: 7.050000000000001 2024-09-21T06:44:29.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:29 smithi203 ceph-mon[33735]: pgmap v597: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 37 MiB/s wr, 702 op/s 2024-09-21T06:44:30.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:29 smithi067 ceph-mon[31232]: pgmap v597: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 37 MiB/s wr, 702 op/s 2024-09-21T06:44:30.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:29 smithi100 ceph-mon[33401]: pgmap v597: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 37 MiB/s wr, 702 op/s 2024-09-21T06:44:31.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:31 smithi203 ceph-mon[33735]: pgmap v598: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 28 MiB/s wr, 527 op/s 2024-09-21T06:44:32.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:31 smithi067 ceph-mon[31232]: pgmap v598: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 28 MiB/s wr, 527 op/s 2024-09-21T06:44:32.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:31 smithi100 ceph-mon[33401]: pgmap v598: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 28 MiB/s wr, 527 op/s 2024-09-21T06:44:33.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:33 smithi203 ceph-mon[33735]: pgmap v599: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 29 MiB/s wr, 545 op/s 2024-09-21T06:44:34.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:33 smithi067 ceph-mon[31232]: pgmap v599: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 29 MiB/s wr, 545 op/s 2024-09-21T06:44:34.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:33 smithi100 ceph-mon[33401]: pgmap v599: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 29 MiB/s wr, 545 op/s 2024-09-21T06:44:35.959 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:44:35.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:35 smithi203 ceph-mon[33735]: pgmap v600: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 19 MiB/s wr, 357 op/s 2024-09-21T06:44:35.991 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:35 smithi067 ceph-mon[31232]: pgmap v600: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 19 MiB/s wr, 357 op/s 2024-09-21T06:44:36.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:35 smithi100 ceph-mon[33401]: pgmap v600: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 19 MiB/s wr, 357 op/s 2024-09-21T06:44:36.231 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:44:36.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.843+0000 7f11d9c74640 1 -- 172.21.15.67:0/3710319675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11d406b2f0 msgr2=0x7f11d4102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:36.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.843+0000 7f11d9c74640 1 --2- 172.21.15.67:0/3710319675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11d406b2f0 0x7f11d4102230 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f11c4009900 tx=0x7f11c402a410 comp rx=0 tx=0).stop 2024-09-21T06:44:36.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.843+0000 7f11d9c74640 1 -- 172.21.15.67:0/3710319675 shutdown_connections 2024-09-21T06:44:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.843+0000 7f11d9c74640 1 --2- 172.21.15.67:0/3710319675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d4102770 0x7f11d4107b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.843+0000 7f11d9c74640 1 --2- 172.21.15.67:0/3710319675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11d4102b80 0x7f11d41080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.843+0000 7f11d9c74640 1 --2- 172.21.15.67:0/3710319675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11d406b2f0 0x7f11d4102230 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.843+0000 7f11d9c74640 1 -- 172.21.15.67:0/3710319675 >> 172.21.15.67:0/3710319675 conn(0x7f11d4079e40 msgr2=0x7f11d407a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.844+0000 7f11d9c74640 1 -- 172.21.15.67:0/3710319675 shutdown_connections 2024-09-21T06:44:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.844+0000 7f11d9c74640 1 -- 172.21.15.67:0/3710319675 wait complete. 2024-09-21T06:44:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.844+0000 7f11d9c74640 1 Processor -- start 2024-09-21T06:44:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.844+0000 7f11d9c74640 1 -- start start 2024-09-21T06:44:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.845+0000 7f11d9c74640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d406b2f0 0x7f11d41992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.845+0000 7f11d9c74640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11d4102770 0x7f11d4199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.845+0000 7f11d9c74640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11d4102b80 0x7f11d419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.845+0000 7f11d9c74640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f11d41128a0 con 0x7f11d4102770 2024-09-21T06:44:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.845+0000 7f11d9c74640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f11d4112720 con 0x7f11d406b2f0 2024-09-21T06:44:36.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.845+0000 7f11d9c74640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f11d4112a20 con 0x7f11d4102b80 2024-09-21T06:44:36.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.845+0000 7f11d2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11d4102770 0x7f11d4199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:36.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.845+0000 7f11d2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11d4102770 0x7f11d4199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40572/0 (socket says 172.21.15.67:40572) 2024-09-21T06:44:36.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.845+0000 7f11d2ffd640 1 -- 172.21.15.67:0/2449991376 learned_addr learned my addr 172.21.15.67:0/2449991376 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:44:36.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.845+0000 7f11d3fff640 1 --2- 172.21.15.67:0/2449991376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11d4102b80 0x7f11d419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:36.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.845+0000 7f11d37fe640 1 --2- 172.21.15.67:0/2449991376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d406b2f0 0x7f11d41992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:36.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.845+0000 7f11d2ffd640 1 -- 172.21.15.67:0/2449991376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11d4102b80 msgr2=0x7f11d419eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:36.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.845+0000 7f11d2ffd640 1 --2- 172.21.15.67:0/2449991376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11d4102b80 0x7f11d419eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:36.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.845+0000 7f11d2ffd640 1 -- 172.21.15.67:0/2449991376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d406b2f0 msgr2=0x7f11d41992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:36.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.845+0000 7f11d2ffd640 1 --2- 172.21.15.67:0/2449991376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d406b2f0 0x7f11d41992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:36.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.845+0000 7f11d2ffd640 1 -- 172.21.15.67:0/2449991376 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d419f160 con 0x7f11d4102770 2024-09-21T06:44:36.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.845+0000 7f11d3fff640 1 --2- 172.21.15.67:0/2449991376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11d4102b80 0x7f11d419eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:44:36.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.845+0000 7f11d2ffd640 1 --2- 172.21.15.67:0/2449991376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11d4102770 0x7f11d4199850 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f11c8007940 tx=0x7f11c8007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:36.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.845+0000 7f11d0ff9640 1 -- 172.21.15.67:0/2449991376 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c800e6f0 con 0x7f11d4102770 2024-09-21T06:44:36.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.846+0000 7f11d9c74640 1 -- 172.21.15.67:0/2449991376 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d419f450 con 0x7f11d4102770 2024-09-21T06:44:36.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.846+0000 7f11d0ff9640 1 -- 172.21.15.67:0/2449991376 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11c800ed20 con 0x7f11d4102770 2024-09-21T06:44:36.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.846+0000 7f11d0ff9640 1 -- 172.21.15.67:0/2449991376 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c800ac70 con 0x7f11d4102770 2024-09-21T06:44:36.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.846+0000 7f11d9c74640 1 -- 172.21.15.67:0/2449991376 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d4108cc0 con 0x7f11d4102770 2024-09-21T06:44:36.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.847+0000 7f11d9c74640 1 -- 172.21.15.67:0/2449991376 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11d41034b0 con 0x7f11d4102770 2024-09-21T06:44:36.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.848+0000 7f11d0ff9640 1 -- 172.21.15.67:0/2449991376 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f11c800e890 con 0x7f11d4102770 2024-09-21T06:44:36.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.849+0000 7f11d0ff9640 1 --2- 172.21.15.67:0/2449991376 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f11bc0b6540 0x7f11bc0b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:36.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.849+0000 7f11d37fe640 1 --2- 172.21.15.67:0/2449991376 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f11bc0b6540 0x7f11bc0b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:36.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.849+0000 7f11d0ff9640 1 -- 172.21.15.67:0/2449991376 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f11c80e8ee0 con 0x7f11d4102770 2024-09-21T06:44:36.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.850+0000 7f11d37fe640 1 --2- 172.21.15.67:0/2449991376 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f11bc0b6540 0x7f11bc0b8a50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f11c4000c00 tx=0x7f11c4009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:36.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:36.853+0000 7f11d0ff9640 1 -- 172.21.15.67:0/2449991376 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f11c8023060 con 0x7f11d4102770 2024-09-21T06:44:37.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:37.083+0000 7f11d9c74640 1 -- 172.21.15.67:0/2449991376 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11d410a0a0 con 0x7f11d4102770 2024-09-21T06:44:37.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:37.084+0000 7f11d0ff9640 1 -- 172.21.15.67:0/2449991376 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7f11c80af430 con 0x7f11d4102770 2024-09-21T06:44:37.087 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:44:37.087 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:44:37.087 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:44:37.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:37.087+0000 7f11d9c74640 1 -- 172.21.15.67:0/2449991376 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f11bc0b6540 msgr2=0x7f11bc0b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:37.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:37.087+0000 7f11d9c74640 1 --2- 172.21.15.67:0/2449991376 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f11bc0b6540 0x7f11bc0b8a50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f11c4000c00 tx=0x7f11c4009680 comp rx=0 tx=0).stop 2024-09-21T06:44:37.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:37.087+0000 7f11d9c74640 1 -- 172.21.15.67:0/2449991376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11d4102770 msgr2=0x7f11d4199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:37.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:37.087+0000 7f11d9c74640 1 --2- 172.21.15.67:0/2449991376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11d4102770 0x7f11d4199850 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f11c8007940 tx=0x7f11c8007e00 comp rx=0 tx=0).stop 2024-09-21T06:44:37.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:37.087+0000 7f11d9c74640 1 -- 172.21.15.67:0/2449991376 shutdown_connections 2024-09-21T06:44:37.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:37.087+0000 7f11d9c74640 1 --2- 172.21.15.67:0/2449991376 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f11bc0b6540 0x7f11bc0b8a50 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:37.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:37.087+0000 7f11d9c74640 1 --2- 172.21.15.67:0/2449991376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11d4102b80 0x7f11d419eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:37.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:37.087+0000 7f11d9c74640 1 --2- 172.21.15.67:0/2449991376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11d4102770 0x7f11d4199850 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:37.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:37.087+0000 7f11d9c74640 1 --2- 172.21.15.67:0/2449991376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d406b2f0 0x7f11d41992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:37.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:37.087+0000 7f11d9c74640 1 -- 172.21.15.67:0/2449991376 >> 172.21.15.67:0/2449991376 conn(0x7f11d4079e40 msgr2=0x7f11d4100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:37.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:37.087+0000 7f11d9c74640 1 -- 172.21.15.67:0/2449991376 shutdown_connections 2024-09-21T06:44:37.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:37.087+0000 7f11d9c74640 1 -- 172.21.15.67:0/2449991376 wait complete. 2024-09-21T06:44:37.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:37 smithi067 ceph-mon[31232]: pgmap v601: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 15 MiB/s wr, 295 op/s 2024-09-21T06:44:37.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:37 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2449991376' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:44:37.964 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db --set-id 45924b04 --release --await' 2024-09-21T06:44:37.964 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24671 quiesce db --set-id 45924b04 --release --await 2024-09-21T06:44:37.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:37 smithi203 ceph-mon[33735]: pgmap v601: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 15 MiB/s wr, 295 op/s 2024-09-21T06:44:37.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:37 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2449991376' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:44:38.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:37 smithi100 ceph-mon[33401]: pgmap v601: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 15 MiB/s wr, 295 op/s 2024-09-21T06:44:38.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:37 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2449991376' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:44:38.365 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:44:38.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.996+0000 7fdda6721640 1 -- 172.21.15.67:0/2523685289 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdda010de00 msgr2=0x7fdda0110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:38.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.996+0000 7fdda6721640 1 --2- 172.21.15.67:0/2523685289 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdda010de00 0x7fdda0110310 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fdd98009ca0 tx=0x7fdd9802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:44:38.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.996+0000 7fdda6721640 1 -- 172.21.15.67:0/2523685289 shutdown_connections 2024-09-21T06:44:38.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.996+0000 7fdda6721640 1 --2- 172.21.15.67:0/2523685289 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdda010de00 0x7fdda0110310 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:38.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.996+0000 7fdda6721640 1 --2- 172.21.15.67:0/2523685289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdda0105280 0x7fdda010d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:38.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.996+0000 7fdda6721640 1 --2- 172.21.15.67:0/2523685289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdda0104850 0x7fdda0104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:38.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.996+0000 7fdda6721640 1 -- 172.21.15.67:0/2523685289 >> 172.21.15.67:0/2523685289 conn(0x7fdda0079710 msgr2=0x7fdda0079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:38.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.996+0000 7fdda6721640 1 -- 172.21.15.67:0/2523685289 shutdown_connections 2024-09-21T06:44:38.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.996+0000 7fdda6721640 1 -- 172.21.15.67:0/2523685289 wait complete. 2024-09-21T06:44:39.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.996+0000 7fdda6721640 1 Processor -- start 2024-09-21T06:44:39.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.997+0000 7fdda6721640 1 -- start start 2024-09-21T06:44:39.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.997+0000 7fdda6721640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdda0104850 0x7fdda019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:39.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.997+0000 7fdda6721640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdda0105280 0x7fdda019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:39.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.997+0000 7fdda6721640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdda010de00 0x7fdda019fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:39.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.997+0000 7fdda6721640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdda0113980 con 0x7fdda0104850 2024-09-21T06:44:39.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.997+0000 7fdda6721640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdda0113800 con 0x7fdda0105280 2024-09-21T06:44:39.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.997+0000 7fdda6721640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdda0113b00 con 0x7fdda010de00 2024-09-21T06:44:39.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.997+0000 7fdd9ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdda0104850 0x7fdda019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:39.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.997+0000 7fdd9ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdda0104850 0x7fdda019a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43628/0 (socket says 172.21.15.67:43628) 2024-09-21T06:44:39.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.997+0000 7fdd9ffff640 1 -- 172.21.15.67:0/1865879761 learned_addr learned my addr 172.21.15.67:0/1865879761 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:44:39.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.997+0000 7fdd9f7fe640 1 --2- 172.21.15.67:0/1865879761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdda0105280 0x7fdda019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:39.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.997+0000 7fdda4cbe640 1 --2- 172.21.15.67:0/1865879761 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdda010de00 0x7fdda019fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:39.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.998+0000 7fdd9ffff640 1 -- 172.21.15.67:0/1865879761 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdda010de00 msgr2=0x7fdda019fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:39.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.998+0000 7fdd9ffff640 1 --2- 172.21.15.67:0/1865879761 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdda010de00 0x7fdda019fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.998+0000 7fdd9ffff640 1 -- 172.21.15.67:0/1865879761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdda0105280 msgr2=0x7fdda019a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:39.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.998+0000 7fdd9ffff640 1 --2- 172.21.15.67:0/1865879761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdda0105280 0x7fdda019a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.998+0000 7fdd9ffff640 1 -- 172.21.15.67:0/1865879761 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda01a0200 con 0x7fdda0104850 2024-09-21T06:44:39.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.998+0000 7fdd9ffff640 1 --2- 172.21.15.67:0/1865879761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdda0104850 0x7fdda019a350 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fdd9000c920 tx=0x7fdd9000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:39.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.998+0000 7fdd9d7fa640 1 -- 172.21.15.67:0/1865879761 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd90007c70 con 0x7fdda0104850 2024-09-21T06:44:39.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.998+0000 7fdd9d7fa640 1 -- 172.21.15.67:0/1865879761 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd90007e10 con 0x7fdda0104850 2024-09-21T06:44:39.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.998+0000 7fdd9d7fa640 1 -- 172.21.15.67:0/1865879761 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd90018710 con 0x7fdda0104850 2024-09-21T06:44:39.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.998+0000 7fdda6721640 1 -- 172.21.15.67:0/1865879761 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdda01a04f0 con 0x7fdda0104850 2024-09-21T06:44:39.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:38.998+0000 7fdda6721640 1 -- 172.21.15.67:0/1865879761 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda01a6d10 con 0x7fdda0104850 2024-09-21T06:44:39.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.001+0000 7fddacc95740 1 Processor -- start 2024-09-21T06:44:39.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.001+0000 7fdd9d7fa640 1 -- 172.21.15.67:0/1865879761 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fdd9000f450 con 0x7fdda0104850 2024-09-21T06:44:39.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.001+0000 7fddacc95740 1 -- start start 2024-09-21T06:44:39.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.001+0000 7fddacc95740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdb7bbc520 0x55cdb7f62510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:39.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.001+0000 7fddacc95740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cdb7b93660 0x55cdb7f62a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:39.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.001+0000 7fddacc95740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cdb7b9b840 0x55cdb7f62f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:39.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.001+0000 7fddacc95740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55cdb7e4ddc0 con 0x55cdb7b93660 2024-09-21T06:44:39.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.001+0000 7fddacc95740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55cdb7eda1f0 con 0x55cdb7bbc520 2024-09-21T06:44:39.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.001+0000 7fddacc95740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55cdb7ec9620 con 0x55cdb7b9b840 2024-09-21T06:44:39.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.001+0000 7fdd9ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cdb7b93660 0x55cdb7f62a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:39.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.001+0000 7fdd9ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cdb7b93660 0x55cdb7f62a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43634/0 (socket says 172.21.15.67:43634) 2024-09-21T06:44:39.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.001+0000 7fdd9ffff640 1 -- 172.21.15.67:0/2528782736 learned_addr learned my addr 172.21.15.67:0/2528782736 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:44:39.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.001+0000 7fdda4cbe640 1 --2- 172.21.15.67:0/2528782736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdb7bbc520 0x55cdb7f62510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:39.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.002+0000 7fdd9f7fe640 1 --2- 172.21.15.67:0/2528782736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cdb7b9b840 0x55cdb7f62f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:39.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.002+0000 7fdd9ffff640 1 -- 172.21.15.67:0/2528782736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cdb7b9b840 msgr2=0x55cdb7f62f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:44:39.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.002+0000 7fdd9ffff640 1 --2- 172.21.15.67:0/2528782736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cdb7b9b840 0x55cdb7f62f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.002+0000 7fdd9ffff640 1 -- 172.21.15.67:0/2528782736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdb7bbc520 msgr2=0x55cdb7f62510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:44:39.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.002+0000 7fdd9ffff640 1 --2- 172.21.15.67:0/2528782736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdb7bbc520 0x55cdb7f62510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.002+0000 7fdd9ffff640 1 -- 172.21.15.67:0/2528782736 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdb7e91fd0 con 0x55cdb7b93660 2024-09-21T06:44:39.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.002+0000 7fdda4cbe640 1 --2- 172.21.15.67:0/2528782736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdb7bbc520 0x55cdb7f62510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:44:39.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.002+0000 7fdd9d7fa640 1 --2- 172.21.15.67:0/1865879761 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fdd880b65c0 0x7fdd880b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:39.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.002+0000 7fdd9ffff640 1 --2- 172.21.15.67:0/2528782736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cdb7b93660 0x55cdb7f62a50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fdd900edc10 tx=0x7fdd900edd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857edc5681715cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:39.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.002+0000 7fdda4cbe640 1 --2- 172.21.15.67:0/1865879761 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fdd880b65c0 0x7fdd880b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:39.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.002+0000 7fdd867fc640 1 -- 172.21.15.67:0/2528782736 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd900a9b20 con 0x55cdb7b93660 2024-09-21T06:44:39.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.002+0000 7fdd9d7fa640 1 -- 172.21.15.67:0/1865879761 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fdd900e7710 con 0x7fdda0104850 2024-09-21T06:44:39.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.002+0000 7fdd867fc640 1 -- 172.21.15.67:0/2528782736 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd900acfe0 con 0x55cdb7b93660 2024-09-21T06:44:39.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.003+0000 7fdd867fc640 1 -- 172.21.15.67:0/2528782736 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd90077360 con 0x55cdb7b93660 2024-09-21T06:44:39.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.003+0000 7fddacc95740 1 -- 172.21.15.67:0/2528782736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cdb7b93660 msgr2=0x55cdb7f62a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:39.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.003+0000 7fdda4cbe640 1 --2- 172.21.15.67:0/1865879761 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fdd880b65c0 0x7fdd880b8ad0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdd98006010 tx=0x7fdd98009900 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:39.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.003+0000 7fddacc95740 1 --2- 172.21.15.67:0/2528782736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cdb7b93660 0x55cdb7f62a50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fdd900edc10 tx=0x7fdd900edd00 comp rx=0 tx=0).stop 2024-09-21T06:44:39.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.003+0000 7fddacc95740 1 -- 172.21.15.67:0/2528782736 shutdown_connections 2024-09-21T06:44:39.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.003+0000 7fddacc95740 1 --2- 172.21.15.67:0/2528782736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cdb7b9b840 0x55cdb7f62f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.003+0000 7fddacc95740 1 --2- 172.21.15.67:0/2528782736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cdb7b93660 0x55cdb7f62a50 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.003+0000 7fddacc95740 1 --2- 172.21.15.67:0/2528782736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdb7bbc520 0x55cdb7f62510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.003+0000 7fddacc95740 1 -- 172.21.15.67:0/2528782736 >> 172.21.15.67:0/2528782736 conn(0x55cdb7bb0910 msgr2=0x55cdb7f60400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:39.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.004+0000 7fddacc95740 1 -- 172.21.15.67:0/2528782736 shutdown_connections 2024-09-21T06:44:39.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.004+0000 7fddacc95740 1 -- 172.21.15.67:0/2528782736 wait complete. 2024-09-21T06:44:39.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.004+0000 7fddacc95740 1 -- start start 2024-09-21T06:44:39.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.004+0000 7fddacc95740 1 Processor -- start 2024-09-21T06:44:39.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.005+0000 7fddacc95740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdb7b93660 0x55cdb7f6dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:39.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.005+0000 7fddacc95740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cdb7b9b840 0x55cdb7f6e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:39.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.005+0000 7fddacc95740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cdb7bbc520 0x55cdb7f79e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:39.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.005+0000 7fddacc95740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55cdb7e64300 con 0x55cdb7bbc520 2024-09-21T06:44:39.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.005+0000 7fddacc95740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55cdb7ecd430 con 0x55cdb7b93660 2024-09-21T06:44:39.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.005+0000 7fddacc95740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55cdb7d2abb0 con 0x55cdb7b9b840 2024-09-21T06:44:39.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.005+0000 7fddacc95740 10 client.0 _ll_register_callbacks cb 0x55cdb7e7e120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:44:39.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.005+0000 7fdd9ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdb7b93660 0x55cdb7f6dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:39.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.005+0000 7fdd9ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdb7b93660 0x55cdb7f6dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.67:56952/0 (socket says 172.21.15.67:56952) 2024-09-21T06:44:39.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.005+0000 7fdd9ffff640 1 -- 172.21.15.67:0/3666356743 learned_addr learned my addr 172.21.15.67:0/3666356743 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:44:39.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.005+0000 7fdd9ffff640 1 -- 172.21.15.67:0/3666356743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cdb7b9b840 msgr2=0x55cdb7f6e150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:44:39.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.005+0000 7fdd9ffff640 1 --2- 172.21.15.67:0/3666356743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cdb7b9b840 0x55cdb7f6e150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.005+0000 7fdd9ffff640 1 -- 172.21.15.67:0/3666356743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cdb7bbc520 msgr2=0x55cdb7f79e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:44:39.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.005+0000 7fdd9ffff640 1 --2- 172.21.15.67:0/3666356743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cdb7bbc520 0x55cdb7f79e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.005+0000 7fdd9ffff640 1 -- 172.21.15.67:0/3666356743 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdb7e53780 con 0x55cdb7b93660 2024-09-21T06:44:39.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.005+0000 7fdd9ffff640 1 --2- 172.21.15.67:0/3666356743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdb7b93660 0x55cdb7f6dc10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdd900edc10 tx=0x7fdd900adcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a61e22837077c84 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:39.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.006+0000 7fdd7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-21T06:44:39.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.006+0000 7fdd7f7fe640 1 -- 172.21.15.67:0/3666356743 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd90077550 con 0x55cdb7b93660 2024-09-21T06:44:39.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.006+0000 7fddacc95740 1 -- 172.21.15.67:0/3666356743 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd900acfe0 con 0x55cdb7b93660 2024-09-21T06:44:39.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.006+0000 7fdd7f7fe640 1 -- 172.21.15.67:0/3666356743 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fdd900e7420 con 0x55cdb7b93660 2024-09-21T06:44:39.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.006+0000 7fdd7f7fe640 1 -- 172.21.15.67:0/3666356743 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdd90077730 con 0x55cdb7b93660 2024-09-21T06:44:39.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.006+0000 7fddacc95740 10 client.25502 fetch_fsmap learned FSMap version 21 2024-09-21T06:44:39.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.006+0000 7fddacc95740 1 -- 172.21.15.67:0/3666356743 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55cdb7c27b00 con 0x55cdb7b93660 2024-09-21T06:44:39.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.006+0000 7fdd7f7fe640 1 -- 172.21.15.67:0/3666356743 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd900acfe0 con 0x55cdb7b93660 2024-09-21T06:44:39.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.006+0000 7fdd7f7fe640 1 -- 172.21.15.67:0/3666356743 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fdd90094040 con 0x55cdb7b93660 2024-09-21T06:44:39.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.006+0000 7fddacc95740 10 client.25502 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:44:39.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.006+0000 7fddacc95740 10 client.25502 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:44:39.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.007+0000 7fddacc95740 20 client.25502 populate_metadata read hostname 'smithi067' 2024-09-21T06:44:39.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.007+0000 7fddacc95740 1 --2- 172.21.15.67:0/3666356743 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55cdb7b7aea0 0x55cdb7f7fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:39.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.007+0000 7fddacc95740 4 client.25502 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:44:39.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.007+0000 7fddacc95740 1 -- 172.21.15.67:0/3666356743 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdb7edc140 con 0x55cdb7b7aea0 2024-09-21T06:44:39.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.007+0000 7fdd9f7fe640 1 --2- 172.21.15.67:0/3666356743 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55cdb7b7aea0 0x55cdb7f7fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:39.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.008+0000 7fdd7f7fe640 10 client.25502 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:44:39.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.008+0000 7fdd9f7fe640 1 --2- 172.21.15.67:0/3666356743 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55cdb7b7aea0 0x55cdb7f7fdb0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaebe4101de8e66 server_cookie=4915a56ab3e8a1d9 in_seq=0 out_seq=0 2024-09-21T06:44:39.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.008+0000 7fdd7f7fe640 1 -- 172.21.15.67:0/3666356743 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cdb7edc140 con 0x55cdb7b7aea0 2024-09-21T06:44:39.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.008+0000 7fdd7f7fe640 10 client.25502 handle_command_reply: tid=0 2024-09-21T06:44:39.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.008+0000 7fddacc95740 1 client.25502 shutdown 2024-09-21T06:44:39.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.009+0000 7fddacc95740 1 -- 172.21.15.67:0/3666356743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdb7b93660 msgr2=0x55cdb7f6dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:39.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.009+0000 7fddacc95740 1 --2- 172.21.15.67:0/3666356743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdb7b93660 0x55cdb7f6dc10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdd900edc10 tx=0x7fdd900adcb0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.009+0000 7fddacc95740 1 -- 172.21.15.67:0/3666356743 shutdown_connections 2024-09-21T06:44:39.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.009+0000 7fddacc95740 1 --2- 172.21.15.67:0/3666356743 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55cdb7b7aea0 0x55cdb7f7fdb0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.009+0000 7fddacc95740 1 --2- 172.21.15.67:0/3666356743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cdb7bbc520 0x55cdb7f79e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.009+0000 7fddacc95740 1 --2- 172.21.15.67:0/3666356743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cdb7b9b840 0x55cdb7f6e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.009+0000 7fdd7f7fe640 0 client.25502 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:44:39.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.009+0000 7fddacc95740 1 --2- 172.21.15.67:0/3666356743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdb7b93660 0x55cdb7f6dc10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.009+0000 7fddacc95740 1 -- 172.21.15.67:0/3666356743 >> 172.21.15.67:0/3666356743 conn(0x55cdb7bb0910 msgr2=0x55cdb7f656f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:39.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.009+0000 7fddacc95740 1 -- 172.21.15.67:0/3666356743 shutdown_connections 2024-09-21T06:44:39.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.009+0000 7fddacc95740 1 -- 172.21.15.67:0/3666356743 wait complete. 2024-09-21T06:44:39.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.009+0000 7fddacc95740 20 client.25502 trim_cache size 0 max 16384 2024-09-21T06:44:39.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.026+0000 7fddacc95740 1 Processor -- start 2024-09-21T06:44:39.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.026+0000 7fddacc95740 1 -- start start 2024-09-21T06:44:39.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.026+0000 7fddacc95740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cdb7bb0910 0x55cdb7f69350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:39.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.026+0000 7fddacc95740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdb7b93660 0x55cdb7f6cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:39.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.026+0000 7fddacc95740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cdb7b9b840 0x55cdb7f6ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:39.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.026+0000 7fddacc95740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdda01aeaa0 con 0x55cdb7bb0910 2024-09-21T06:44:39.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.026+0000 7fddacc95740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdda01b1320 con 0x55cdb7b93660 2024-09-21T06:44:39.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.026+0000 7fddacc95740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdda01af110 con 0x55cdb7b9b840 2024-09-21T06:44:39.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.026+0000 7fdd9ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cdb7bb0910 0x55cdb7f69350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:39.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.026+0000 7fdd9ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cdb7bb0910 0x55cdb7f69350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43656/0 (socket says 172.21.15.67:43656) 2024-09-21T06:44:39.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.026+0000 7fdd9ffff640 1 -- 172.21.15.67:0/3741706370 learned_addr learned my addr 172.21.15.67:0/3741706370 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:44:39.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.027+0000 7fdd9f7fe640 1 --2- 172.21.15.67:0/3741706370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdb7b93660 0x55cdb7f6cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:39.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.027+0000 7fdd9ffff640 1 -- 172.21.15.67:0/3741706370 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cdb7b9b840 msgr2=0x55cdb7f6ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:44:39.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.027+0000 7fdd9ffff640 1 --2- 172.21.15.67:0/3741706370 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cdb7b9b840 0x55cdb7f6ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.027+0000 7fdd9ffff640 1 -- 172.21.15.67:0/3741706370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdb7b93660 msgr2=0x55cdb7f6cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:44:39.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.027+0000 7fdd9ffff640 1 --2- 172.21.15.67:0/3741706370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdb7b93660 0x55cdb7f6cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.027+0000 7fdd9ffff640 1 -- 172.21.15.67:0/3741706370 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdb7c9e960 con 0x55cdb7bb0910 2024-09-21T06:44:39.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.027+0000 7fdd9f7fe640 1 --2- 172.21.15.67:0/3741706370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdb7b93660 0x55cdb7f6cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:44:39.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.027+0000 7fdd9ffff640 1 --2- 172.21.15.67:0/3741706370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cdb7bb0910 0x55cdb7f69350 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdd900aa740 tx=0x7fdd900ed1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdda22ba80d64957 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:39.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.027+0000 7fdd85ffb640 1 -- 172.21.15.67:0/3741706370 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd900acc40 con 0x55cdb7bb0910 2024-09-21T06:44:39.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.027+0000 7fdd85ffb640 1 -- 172.21.15.67:0/3741706370 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd900acfe0 con 0x55cdb7bb0910 2024-09-21T06:44:39.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.027+0000 7fdd85ffb640 1 -- 172.21.15.67:0/3741706370 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd900ace20 con 0x55cdb7bb0910 2024-09-21T06:44:39.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.028+0000 7fddacc95740 1 -- 172.21.15.67:0/3741706370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cdb7bb0910 msgr2=0x55cdb7f69350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:39.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.028+0000 7fddacc95740 1 --2- 172.21.15.67:0/3741706370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cdb7bb0910 0x55cdb7f69350 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdd900aa740 tx=0x7fdd900ed1f0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.028+0000 7fddacc95740 1 -- 172.21.15.67:0/3741706370 shutdown_connections 2024-09-21T06:44:39.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.028+0000 7fddacc95740 1 --2- 172.21.15.67:0/3741706370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdb7b93660 0x55cdb7f6cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.028+0000 7fddacc95740 1 --2- 172.21.15.67:0/3741706370 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cdb7b9b840 0x55cdb7f6ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.028+0000 7fddacc95740 1 --2- 172.21.15.67:0/3741706370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cdb7bb0910 0x55cdb7f69350 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.028+0000 7fddacc95740 1 -- 172.21.15.67:0/3741706370 >> 172.21.15.67:0/3741706370 conn(0x55cdb7f6e2d0 msgr2=0x55cdb7f6c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:39.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.028+0000 7fddacc95740 1 -- 172.21.15.67:0/3741706370 shutdown_connections 2024-09-21T06:44:39.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.028+0000 7fddacc95740 1 -- 172.21.15.67:0/3741706370 wait complete. 2024-09-21T06:44:39.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.028+0000 7fddacc95740 1 -- start start 2024-09-21T06:44:39.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.029+0000 7fddacc95740 1 Processor -- start 2024-09-21T06:44:39.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.029+0000 7fddacc95740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cdb7b93660 0x55cdb7f815e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:39.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.029+0000 7fddacc95740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cdb7b9b840 0x55cdb7f81b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:39.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.029+0000 7fddacc95740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdb7bb0910 0x55cdb7fe9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:39.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.029+0000 7fddacc95740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdda01af7e0 con 0x55cdb7b9b840 2024-09-21T06:44:39.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.029+0000 7fddacc95740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdda01b0c50 con 0x55cdb7bb0910 2024-09-21T06:44:39.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.029+0000 7fdd9f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cdb7b9b840 0x55cdb7f81b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:39.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.029+0000 7fddacc95740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdda01afeb0 con 0x55cdb7b93660 2024-09-21T06:44:39.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.029+0000 7fdd9f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cdb7b9b840 0x55cdb7f81b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43668/0 (socket says 172.21.15.67:43668) 2024-09-21T06:44:39.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.029+0000 7fdd9f7fe640 1 -- 172.21.15.67:0/2043115231 learned_addr learned my addr 172.21.15.67:0/2043115231 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:44:39.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.029+0000 7fdd9ffff640 1 --2- 172.21.15.67:0/2043115231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cdb7b93660 0x55cdb7f815e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:39.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.029+0000 7fdda4cbe640 1 --2- 172.21.15.67:0/2043115231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdb7bb0910 0x55cdb7fe9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:39.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.029+0000 7fddacc95740 10 client.0 _ll_register_callbacks cb 0x7fdd980093e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:44:39.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.030+0000 7fdd9f7fe640 1 -- 172.21.15.67:0/2043115231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cdb7b93660 msgr2=0x55cdb7f815e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:44:39.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.030+0000 7fdd9f7fe640 1 --2- 172.21.15.67:0/2043115231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cdb7b93660 0x55cdb7f815e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.030+0000 7fdd9f7fe640 1 -- 172.21.15.67:0/2043115231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdb7bb0910 msgr2=0x55cdb7fe9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:44:39.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.030+0000 7fdd9f7fe640 1 --2- 172.21.15.67:0/2043115231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdb7bb0910 0x55cdb7fe9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.030+0000 7fdd9f7fe640 1 -- 172.21.15.67:0/2043115231 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdb7fe9af0 con 0x55cdb7b9b840 2024-09-21T06:44:39.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.030+0000 7fdd9ffff640 1 --2- 172.21.15.67:0/2043115231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cdb7b93660 0x55cdb7f815e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:44:39.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.030+0000 7fdd9f7fe640 1 --2- 172.21.15.67:0/2043115231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cdb7b9b840 0x55cdb7f81b20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fdd9400ab70 tx=0x7fdd940108c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c868363741e12ef4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:39.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.030+0000 7fdd7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:44:39.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.030+0000 7fdd7f7fe640 1 -- 172.21.15.67:0/2043115231 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9401f040 con 0x55cdb7b9b840 2024-09-21T06:44:39.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.030+0000 7fdd7f7fe640 1 -- 172.21.15.67:0/2043115231 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fdd94005460 con 0x55cdb7b9b840 2024-09-21T06:44:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.030+0000 7fddacc95740 1 -- 172.21.15.67:0/2043115231 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd900acfe0 con 0x55cdb7b9b840 2024-09-21T06:44:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.030+0000 7fdd7f7fe640 1 -- 172.21.15.67:0/2043115231 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdd9402c050 con 0x55cdb7b9b840 2024-09-21T06:44:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.030+0000 7fddacc95740 10 client.15789 fetch_fsmap learned FSMap version 21 2024-09-21T06:44:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.030+0000 7fddacc95740 1 -- 172.21.15.67:0/2043115231 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55cdb7fef1a0 con 0x55cdb7b9b840 2024-09-21T06:44:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.030+0000 7fdd7f7fe640 1 -- 172.21.15.67:0/2043115231 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd900acfe0 con 0x55cdb7b9b840 2024-09-21T06:44:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.031+0000 7fdd7f7fe640 1 -- 172.21.15.67:0/2043115231 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fdd9401d7a0 con 0x55cdb7b9b840 2024-09-21T06:44:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.031+0000 7fddacc95740 10 client.15789 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:44:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.031+0000 7fddacc95740 10 client.15789 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:44:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.031+0000 7fddacc95740 20 client.15789 populate_metadata read hostname 'smithi067' 2024-09-21T06:44:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.031+0000 7fddacc95740 1 --2- 172.21.15.67:0/2043115231 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55cdb7bbc520 0x55cdb7ff1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.031+0000 7fddacc95740 4 client.15789 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "45924b04", "release": true, "await": true}] 2024-09-21T06:44:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.031+0000 7fddacc95740 1 -- 172.21.15.67:0/2043115231 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "quiesce db", "set_id": "45924b04", "release": true, "await": true}) -- 0x55cdb7ed7bf0 con 0x55cdb7bbc520 2024-09-21T06:44:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.031+0000 7fdd9ffff640 1 --2- 172.21.15.67:0/2043115231 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55cdb7bbc520 0x55cdb7ff1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.031+0000 7fdd7f7fe640 10 client.15789 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:44:39.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.031+0000 7fdd9ffff640 1 --2- 172.21.15.67:0/2043115231 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55cdb7bbc520 0x55cdb7ff1990 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9df1f1eb76bdde21 server_cookie=6646baaac1f8e5f0 in_seq=0 out_seq=0 2024-09-21T06:44:39.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.034+0000 7fdd7f7fe640 1 -- 172.21.15.67:0/2043115231 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55cdb7ed7bf0 con 0x55cdb7bbc520 2024-09-21T06:44:39.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.034+0000 7fdd7f7fe640 10 client.15789 handle_command_reply: tid=0 2024-09-21T06:44:39.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.034+0000 7fddacc95740 1 client.15789 shutdown 2024-09-21T06:44:39.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.035+0000 7fddacc95740 1 -- 172.21.15.67:0/2043115231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cdb7b9b840 msgr2=0x55cdb7f81b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:39.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.035+0000 7fddacc95740 1 --2- 172.21.15.67:0/2043115231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cdb7b9b840 0x55cdb7f81b20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fdd9400ab70 tx=0x7fdd940108c0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.035+0000 7fddacc95740 1 -- 172.21.15.67:0/2043115231 shutdown_connections 2024-09-21T06:44:39.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.035+0000 7fddacc95740 1 --2- 172.21.15.67:0/2043115231 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55cdb7bbc520 0x55cdb7ff1990 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.035+0000 7fddacc95740 1 --2- 172.21.15.67:0/2043115231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdb7bb0910 0x55cdb7fe9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.035+0000 7fddacc95740 1 --2- 172.21.15.67:0/2043115231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cdb7b9b840 0x55cdb7f81b20 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.035+0000 7fddacc95740 1 --2- 172.21.15.67:0/2043115231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cdb7b93660 0x55cdb7f815e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.035+0000 7fddacc95740 1 -- 172.21.15.67:0/2043115231 >> 172.21.15.67:0/2043115231 conn(0x55cdb7f6e2d0 msgr2=0x55cdb7f6ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:39.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.035+0000 7fdd7f7fe640 0 client.15789 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:44:39.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.035+0000 7fddacc95740 1 -- 172.21.15.67:0/2043115231 shutdown_connections 2024-09-21T06:44:39.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.035+0000 7fddacc95740 1 -- 172.21.15.67:0/2043115231 wait complete. 2024-09-21T06:44:39.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.035+0000 7fddacc95740 20 client.15789 trim_cache size 0 max 16384 2024-09-21T06:44:39.040 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-21T06:44:39.040 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 21, 2024-09-21T06:44:39.040 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-09-21T06:44:39.040 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 32, 2024-09-21T06:44:39.040 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-21T06:44:39.040 INFO:teuthology.orchestra.run.smithi067.stdout: "45924b04": { 2024-09-21T06:44:39.040 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 32, 2024-09-21T06:44:39.040 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-21T06:44:39.040 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:44:39.040 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-21T06:44:39.040 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:44:39.041 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:44:39.041 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-21T06:44:39.041 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 141.1, 2024-09-21T06:44:39.041 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-21T06:44:39.041 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-21T06:44:39.041 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-21T06:44:39.041 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:44:39.041 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-21T06:44:39.041 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:44:39.041 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:44:39.041 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:44:39.041 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:44:39.041 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:44:39.042 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:44:39.042 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-21T06:44:39.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.036+0000 7fdda6721640 1 -- 172.21.15.67:0/1865879761 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fdd880b65c0 msgr2=0x7fdd880b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:39.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.036+0000 7fdda6721640 1 --2- 172.21.15.67:0/1865879761 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fdd880b65c0 0x7fdd880b8ad0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdd98006010 tx=0x7fdd98009900 comp rx=0 tx=0).stop 2024-09-21T06:44:39.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.036+0000 7fdda6721640 1 -- 172.21.15.67:0/1865879761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdda0104850 msgr2=0x7fdda019a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:39.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.036+0000 7fdda6721640 1 --2- 172.21.15.67:0/1865879761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdda0104850 0x7fdda019a350 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fdd9000c920 tx=0x7fdd9000cde0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.036+0000 7fdda6721640 1 -- 172.21.15.67:0/1865879761 shutdown_connections 2024-09-21T06:44:39.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.036+0000 7fdda6721640 1 --2- 172.21.15.67:0/1865879761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdda0105280 0x7fdda019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.036+0000 7fdda6721640 1 --2- 172.21.15.67:0/1865879761 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fdd880b65c0 0x7fdd880b8ad0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.036+0000 7fdda6721640 1 --2- 172.21.15.67:0/1865879761 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdda010de00 0x7fdda019fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.036+0000 7fdda6721640 1 --2- 172.21.15.67:0/1865879761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdda0104850 0x7fdda019a350 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:39.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.036+0000 7fdda6721640 1 -- 172.21.15.67:0/1865879761 >> 172.21.15.67:0/1865879761 conn(0x7fdda0079710 msgr2=0x7fdda0107980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:39.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.036+0000 7fdda6721640 1 -- 172.21.15.67:0/1865879761 shutdown_connections 2024-09-21T06:44:39.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:39.036+0000 7fdda6721640 1 -- 172.21.15.67:0/1865879761 wait complete. 2024-09-21T06:44:39.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:39 smithi067 ceph-mon[31232]: pgmap v602: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 13 MiB/s wr, 251 op/s 2024-09-21T06:44:39.813 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '45924b04', total seconds elapsed: 24.9 2024-09-21T06:44:39.813 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.05 2024-09-21T06:44:39.814 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:44:39.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:39 smithi203 ceph-mon[33735]: pgmap v602: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 13 MiB/s wr, 251 op/s 2024-09-21T06:44:40.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:39 smithi100 ceph-mon[33401]: pgmap v602: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 13 MiB/s wr, 251 op/s 2024-09-21T06:44:40.478 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:44:41.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.144+0000 7fd6d5bc9640 1 -- 172.21.15.67:0/1227256162 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6d0110440 msgr2=0x7fd6d0116c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:41.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.144+0000 7fd6d5bc9640 1 --2- 172.21.15.67:0/1227256162 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6d0110440 0x7fd6d0116c70 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0009900 tx=0x7fd6c002a440 comp rx=0 tx=0).stop 2024-09-21T06:44:41.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.144+0000 7fd6d5bc9640 1 -- 172.21.15.67:0/1227256162 shutdown_connections 2024-09-21T06:44:41.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.144+0000 7fd6d5bc9640 1 --2- 172.21.15.67:0/1227256162 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6d0110440 0x7fd6d0116c70 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:41.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.144+0000 7fd6d5bc9640 1 --2- 172.21.15.67:0/1227256162 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6d010faf0 0x7fd6d010ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:41.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.144+0000 7fd6d5bc9640 1 --2- 172.21.15.67:0/1227256162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6d010a1b0 0x7fd6d010a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:41.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.144+0000 7fd6d5bc9640 1 -- 172.21.15.67:0/1227256162 >> 172.21.15.67:0/1227256162 conn(0x7fd6d00780e0 msgr2=0x7fd6d01081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:41.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.150+0000 7fd6d5bc9640 1 -- 172.21.15.67:0/1227256162 shutdown_connections 2024-09-21T06:44:41.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.153+0000 7fd6d5bc9640 1 -- 172.21.15.67:0/1227256162 wait complete. 2024-09-21T06:44:41.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.154+0000 7fd6d5bc9640 1 Processor -- start 2024-09-21T06:44:41.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.154+0000 7fd6d5bc9640 1 -- start start 2024-09-21T06:44:41.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.154+0000 7fd6d5bc9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6d010a1b0 0x7fd6d0074ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:41.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.154+0000 7fd6d5bc9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6d010faf0 0x7fd6d006e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:41.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.154+0000 7fd6d5bc9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6d0110440 0x7fd6d006e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:41.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.154+0000 7fd6d5bc9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d011a090 con 0x7fd6d0110440 2024-09-21T06:44:41.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.154+0000 7fd6d5bc9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d0119f10 con 0x7fd6d010a1b0 2024-09-21T06:44:41.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.154+0000 7fd6d5bc9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d011a210 con 0x7fd6d010faf0 2024-09-21T06:44:41.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.155+0000 7fd6ceffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6d010faf0 0x7fd6d006e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:41.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.155+0000 7fd6ceffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6d010faf0 0x7fd6d006e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.67:55500/0 (socket says 172.21.15.67:55500) 2024-09-21T06:44:41.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.155+0000 7fd6ceffd640 1 -- 172.21.15.67:0/1940041023 learned_addr learned my addr 172.21.15.67:0/1940041023 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:44:41.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.156+0000 7fd6ceffd640 1 -- 172.21.15.67:0/1940041023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6d010a1b0 msgr2=0x7fd6d0074ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:41.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.156+0000 7fd6ceffd640 1 --2- 172.21.15.67:0/1940041023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6d010a1b0 0x7fd6d0074ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:41.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.156+0000 7fd6ceffd640 1 -- 172.21.15.67:0/1940041023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6d0110440 msgr2=0x7fd6d006e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:41.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.156+0000 7fd6ceffd640 1 --2- 172.21.15.67:0/1940041023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6d0110440 0x7fd6d006e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:41.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.156+0000 7fd6ceffd640 1 -- 172.21.15.67:0/1940041023 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d006efd0 con 0x7fd6d010faf0 2024-09-21T06:44:41.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.156+0000 7fd6ceffd640 1 --2- 172.21.15.67:0/1940041023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6d010faf0 0x7fd6d006e1b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800b9b0 tx=0x7fd6c800be70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:41.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.157+0000 7fd6ccff9640 1 -- 172.21.15.67:0/1940041023 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8019070 con 0x7fd6d010faf0 2024-09-21T06:44:41.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.157+0000 7fd6d5bc9640 1 -- 172.21.15.67:0/1940041023 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d0073c70 con 0x7fd6d010faf0 2024-09-21T06:44:41.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.157+0000 7fd6d5bc9640 1 -- 172.21.15.67:0/1940041023 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d0074230 con 0x7fd6d010faf0 2024-09-21T06:44:41.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.157+0000 7fd6ccff9640 1 -- 172.21.15.67:0/1940041023 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6c800cba0 con 0x7fd6d010faf0 2024-09-21T06:44:41.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.157+0000 7fd6ccff9640 1 -- 172.21.15.67:0/1940041023 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c80125a0 con 0x7fd6d010faf0 2024-09-21T06:44:41.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.161+0000 7fd6ccff9640 1 -- 172.21.15.67:0/1940041023 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fd6c801c070 con 0x7fd6d010faf0 2024-09-21T06:44:41.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.162+0000 7fd6d5bc9640 1 -- 172.21.15.67:0/1940041023 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6bc0052c0 con 0x7fd6d010faf0 2024-09-21T06:44:41.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.163+0000 7fd6ccff9640 1 --2- 172.21.15.67:0/1940041023 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd6b80b6720 0x7fd6b80b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:41.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.163+0000 7fd6cf7fe640 1 --2- 172.21.15.67:0/1940041023 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd6b80b6720 0x7fd6b80b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:41.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.164+0000 7fd6ccff9640 1 -- 172.21.15.67:0/1940041023 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fd6c80e69a0 con 0x7fd6d010faf0 2024-09-21T06:44:41.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.165+0000 7fd6cf7fe640 1 --2- 172.21.15.67:0/1940041023 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd6b80b6720 0x7fd6b80b8c30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd6c40096f0 tx=0x7fd6c4006c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:41.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.171+0000 7fd6ccff9640 1 -- 172.21.15.67:0/1940041023 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd6c80ace00 con 0x7fd6d010faf0 2024-09-21T06:44:41.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.632+0000 7fd6d5bc9640 1 -- 172.21.15.67:0/1940041023 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6bc0055b0 con 0x7fd6d010faf0 2024-09-21T06:44:41.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.636+0000 7fd6ccff9640 1 -- 172.21.15.67:0/1940041023 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7fd6c80b11e0 con 0x7fd6d010faf0 2024-09-21T06:44:41.638 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:44:41.638 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:44:41.638 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:44:41.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.641+0000 7fd6d5bc9640 1 -- 172.21.15.67:0/1940041023 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd6b80b6720 msgr2=0x7fd6b80b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:41.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.641+0000 7fd6d5bc9640 1 --2- 172.21.15.67:0/1940041023 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd6b80b6720 0x7fd6b80b8c30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd6c40096f0 tx=0x7fd6c4006c40 comp rx=0 tx=0).stop 2024-09-21T06:44:41.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.641+0000 7fd6d5bc9640 1 -- 172.21.15.67:0/1940041023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6d010faf0 msgr2=0x7fd6d006e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:41.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.641+0000 7fd6d5bc9640 1 --2- 172.21.15.67:0/1940041023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6d010faf0 0x7fd6d006e1b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800b9b0 tx=0x7fd6c800be70 comp rx=0 tx=0).stop 2024-09-21T06:44:41.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.643+0000 7fd6d5bc9640 1 -- 172.21.15.67:0/1940041023 shutdown_connections 2024-09-21T06:44:41.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.643+0000 7fd6d5bc9640 1 --2- 172.21.15.67:0/1940041023 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fd6b80b6720 0x7fd6b80b8c30 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:41.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.643+0000 7fd6d5bc9640 1 --2- 172.21.15.67:0/1940041023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6d0110440 0x7fd6d006e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:41.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.643+0000 7fd6d5bc9640 1 --2- 172.21.15.67:0/1940041023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6d010faf0 0x7fd6d006e1b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:41.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.643+0000 7fd6d5bc9640 1 --2- 172.21.15.67:0/1940041023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6d010a1b0 0x7fd6d0074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:41.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.643+0000 7fd6d5bc9640 1 -- 172.21.15.67:0/1940041023 >> 172.21.15.67:0/1940041023 conn(0x7fd6d00780e0 msgr2=0x7fd6d010e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:41.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.644+0000 7fd6d5bc9640 1 -- 172.21.15.67:0/1940041023 shutdown_connections 2024-09-21T06:44:41.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:41.644+0000 7fd6d5bc9640 1 -- 172.21.15.67:0/1940041023 wait complete. 2024-09-21T06:44:41.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:41 smithi203 ceph-mon[33735]: pgmap v603: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 4.3 MiB/s wr, 81 op/s 2024-09-21T06:44:41.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:41 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1940041023' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:44:41.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:41 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:44:42.031 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:41 smithi067 ceph-mon[31232]: pgmap v603: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 4.3 MiB/s wr, 81 op/s 2024-09-21T06:44:42.031 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:41 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1940041023' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:44:42.031 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:41 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:44:42.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:41 smithi100 ceph-mon[33401]: pgmap v603: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 4.3 MiB/s wr, 81 op/s 2024-09-21T06:44:42.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:41 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1940041023' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:44:42.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:41 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:44:42.423 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db / --timeout 90 --expiration 141.05 --await-for 45.0' 2024-09-21T06:44:42.424 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24671 quiesce db / --timeout 90 --expiration 141.05 --await-for 45.0 2024-09-21T06:44:42.735 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:42 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:44:42.735 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:42 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:44:42.735 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:42 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:44:43.043 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:42 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T06:44:43.044 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:42 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:44:43.044 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:42 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:44:43.044 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:42 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:44:43.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:42 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:44:43.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:42 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:44:43.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:42 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:44:43.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:42 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T06:44:43.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:42 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:44:43.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:42 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:44:43.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:42 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:44:43.087 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:44:43.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:42 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:44:43.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:42 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:44:43.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:42 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:44:43.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:42 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T06:44:43.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:42 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:44:43.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:42 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:44:43.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:42 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:44:43.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.773+0000 7f5562685640 1 -- 172.21.15.67:0/42070632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f555c101ff0 msgr2=0x7f555c102400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:43.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.773+0000 7f5562685640 1 --2- 172.21.15.67:0/42070632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f555c101ff0 0x7f555c102400 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f554c005fd0 tx=0x7f554c02a6d0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.775+0000 7f5562685640 1 -- 172.21.15.67:0/42070632 shutdown_connections 2024-09-21T06:44:43.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.775+0000 7f5562685640 1 --2- 172.21.15.67:0/42070632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f555c108280 0x7f555c10eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.775+0000 7f5562685640 1 --2- 172.21.15.67:0/42070632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f555c107930 0x7f555c107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.775+0000 7f5562685640 1 --2- 172.21.15.67:0/42070632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f555c101ff0 0x7f555c102400 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.775+0000 7f5562685640 1 -- 172.21.15.67:0/42070632 >> 172.21.15.67:0/42070632 conn(0x7f555c077a50 msgr2=0x7f555c077e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:43.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.775+0000 7f5562685640 1 -- 172.21.15.67:0/42070632 shutdown_connections 2024-09-21T06:44:43.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.775+0000 7f5562685640 1 -- 172.21.15.67:0/42070632 wait complete. 2024-09-21T06:44:43.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.776+0000 7f5562685640 1 Processor -- start 2024-09-21T06:44:43.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.776+0000 7f5562685640 1 -- start start 2024-09-21T06:44:43.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.777+0000 7f5562685640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f555c101ff0 0x7f555c0749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:43.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.777+0000 7f5562685640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f555c107930 0x7f555c074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:43.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.777+0000 7f5562685640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f555c108280 0x7f555c06ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:43.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.777+0000 7f5562685640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f555c06d170 con 0x7f555c107930 2024-09-21T06:44:43.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.777+0000 7f5562685640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f555c06cff0 con 0x7f555c101ff0 2024-09-21T06:44:43.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.777+0000 7f5562685640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f555c06d2f0 con 0x7f555c108280 2024-09-21T06:44:43.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.777+0000 7f5561683640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f555c101ff0 0x7f555c0749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:43.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.777+0000 7f5561683640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f555c101ff0 0x7f555c0749a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.67:57006/0 (socket says 172.21.15.67:57006) 2024-09-21T06:44:43.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.777+0000 7f5561683640 1 -- 172.21.15.67:0/1962105876 learned_addr learned my addr 172.21.15.67:0/1962105876 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:44:43.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.777+0000 7f5560e82640 1 --2- 172.21.15.67:0/1962105876 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f555c107930 0x7f555c074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:43.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.777+0000 7f5561e84640 1 --2- 172.21.15.67:0/1962105876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f555c108280 0x7f555c06ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:43.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.777+0000 7f5561683640 1 -- 172.21.15.67:0/1962105876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f555c108280 msgr2=0x7f555c06ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:43.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.777+0000 7f5561683640 1 --2- 172.21.15.67:0/1962105876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f555c108280 0x7f555c06ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.777+0000 7f5561683640 1 -- 172.21.15.67:0/1962105876 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f555c107930 msgr2=0x7f555c074f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:43.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.777+0000 7f5561683640 1 --2- 172.21.15.67:0/1962105876 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f555c107930 0x7f555c074f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.777+0000 7f5561683640 1 -- 172.21.15.67:0/1962105876 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f555c06f4a0 con 0x7f555c101ff0 2024-09-21T06:44:43.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.778+0000 7f5561683640 1 --2- 172.21.15.67:0/1962105876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f555c101ff0 0x7f555c0749a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f554c00ac10 tx=0x7f554c003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:43.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.778+0000 7f555a7fc640 1 -- 172.21.15.67:0/1962105876 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c041020 con 0x7f555c101ff0 2024-09-21T06:44:43.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.779+0000 7f5562685640 1 -- 172.21.15.67:0/1962105876 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f555c06f730 con 0x7f555c101ff0 2024-09-21T06:44:43.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.779+0000 7f5562685640 1 -- 172.21.15.67:0/1962105876 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f555c1172a0 con 0x7f555c101ff0 2024-09-21T06:44:43.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.779+0000 7f555a7fc640 1 -- 172.21.15.67:0/1962105876 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f554c037040 con 0x7f555c101ff0 2024-09-21T06:44:43.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.779+0000 7f555a7fc640 1 -- 172.21.15.67:0/1962105876 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c036070 con 0x7f555c101ff0 2024-09-21T06:44:43.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.787+0000 7f5569e5b740 1 Processor -- start 2024-09-21T06:44:43.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.790+0000 7f5569e5b740 1 -- start start 2024-09-21T06:44:43.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.790+0000 7f5569e5b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a543a8d850 0x55a543e34080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:43.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.790+0000 7f5569e5b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a543a64990 0x55a543e345c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:43.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.790+0000 7f5569e5b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a543a6cb70 0x55a543e34b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:43.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.790+0000 7f5569e5b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a543daabb0 con 0x55a543a8d850 2024-09-21T06:44:43.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.790+0000 7f5569e5b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55a543d99fe0 con 0x55a543a6cb70 2024-09-21T06:44:43.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.790+0000 7f5569e5b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a543d9ddf0 con 0x55a543a64990 2024-09-21T06:44:43.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.792+0000 7f5561683640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a543a64990 0x55a543e345c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:43.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.792+0000 7f5561683640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a543a64990 0x55a543e345c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.67:55528/0 (socket says 172.21.15.67:55528) 2024-09-21T06:44:43.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.792+0000 7f5561683640 1 -- 172.21.15.67:0/4244682986 learned_addr learned my addr 172.21.15.67:0/4244682986 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:44:43.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.793+0000 7f5561683640 1 -- 172.21.15.67:0/4244682986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a543a6cb70 msgr2=0x55a543e34b00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:44:43.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.793+0000 7f5561683640 1 --2- 172.21.15.67:0/4244682986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a543a6cb70 0x55a543e34b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.793+0000 7f5561683640 1 -- 172.21.15.67:0/4244682986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a543a8d850 msgr2=0x55a543e34080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:44:43.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.793+0000 7f5561683640 1 --2- 172.21.15.67:0/4244682986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a543a8d850 0x55a543e34080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.793+0000 7f5561683640 1 -- 172.21.15.67:0/4244682986 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a543b07ba0 con 0x55a543a64990 2024-09-21T06:44:43.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.794+0000 7f5561683640 1 --2- 172.21.15.67:0/4244682986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a543a64990 0x55a543e345c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f554c110480 tx=0x7f554c110570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c53db06f95339ea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:43.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.794+0000 7f55437fe640 1 -- 172.21.15.67:0/4244682986 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c10a2d0 con 0x55a543a64990 2024-09-21T06:44:43.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.794+0000 7f55437fe640 1 -- 172.21.15.67:0/4244682986 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f554c115030 con 0x55a543a64990 2024-09-21T06:44:43.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.794+0000 7f55437fe640 1 -- 172.21.15.67:0/4244682986 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c119740 con 0x55a543a64990 2024-09-21T06:44:43.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.795+0000 7f555a7fc640 1 -- 172.21.15.67:0/1962105876 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f554c044020 con 0x7f555c101ff0 2024-09-21T06:44:43.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.795+0000 7f5569e5b740 1 -- 172.21.15.67:0/4244682986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a543a64990 msgr2=0x55a543e345c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:43.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.795+0000 7f5569e5b740 1 --2- 172.21.15.67:0/4244682986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a543a64990 0x55a543e345c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f554c110480 tx=0x7f554c110570 comp rx=0 tx=0).stop 2024-09-21T06:44:43.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.796+0000 7f555a7fc640 1 --2- 172.21.15.67:0/1962105876 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f55440b6640 0x7f55440b8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:43.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.796+0000 7f555a7fc640 1 -- 172.21.15.67:0/1962105876 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f554c1082c0 con 0x7f555c101ff0 2024-09-21T06:44:43.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.800+0000 7f5561e84640 1 --2- 172.21.15.67:0/1962105876 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f55440b6640 0x7f55440b8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:43.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.801+0000 7f5561e84640 1 --2- 172.21.15.67:0/1962105876 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f55440b6640 0x7f55440b8b50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f555400b3d0 tx=0x7f5554005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:43.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.802+0000 7f5569e5b740 1 -- 172.21.15.67:0/4244682986 shutdown_connections 2024-09-21T06:44:43.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.802+0000 7f5569e5b740 1 --2- 172.21.15.67:0/4244682986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a543a6cb70 0x55a543e34b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.802+0000 7f5569e5b740 1 --2- 172.21.15.67:0/4244682986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a543a64990 0x55a543e345c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.802+0000 7f5569e5b740 1 --2- 172.21.15.67:0/4244682986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a543a8d850 0x55a543e34080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.802+0000 7f5569e5b740 1 -- 172.21.15.67:0/4244682986 >> 172.21.15.67:0/4244682986 conn(0x55a543a81c40 msgr2=0x55a543e31f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:43.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.803+0000 7f5569e5b740 1 -- 172.21.15.67:0/4244682986 shutdown_connections 2024-09-21T06:44:43.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.804+0000 7f5569e5b740 1 -- 172.21.15.67:0/4244682986 wait complete. 2024-09-21T06:44:43.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.804+0000 7f5569e5b740 1 -- start start 2024-09-21T06:44:43.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.805+0000 7f5569e5b740 1 Processor -- start 2024-09-21T06:44:43.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.807+0000 7f5569e5b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a543a64990 0x55a543e3fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:43.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.807+0000 7f5569e5b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a543a6cb70 0x55a543e40120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:43.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.807+0000 7f5569e5b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a543a8d850 0x55a543e4b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:43.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.807+0000 7f5569e5b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a543bfb7b0 con 0x55a543a64990 2024-09-21T06:44:43.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.807+0000 7f5569e5b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55a543d34e90 con 0x55a543a8d850 2024-09-21T06:44:43.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.807+0000 7f5569e5b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a543e42620 con 0x55a543a6cb70 2024-09-21T06:44:43.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.807+0000 7f5569e5b740 10 client.0 _ll_register_callbacks cb 0x55a543d97710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:44:43.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.808+0000 7f5561683640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a543a64990 0x55a543e3fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:43.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.808+0000 7f5561683640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a543a64990 0x55a543e3fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43722/0 (socket says 172.21.15.67:43722) 2024-09-21T06:44:43.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.808+0000 7f5561683640 1 -- 172.21.15.67:0/1971875077 learned_addr learned my addr 172.21.15.67:0/1971875077 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:44:43.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.808+0000 7f5561683640 1 -- 172.21.15.67:0/1971875077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a543a6cb70 msgr2=0x55a543e40120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:44:43.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.808+0000 7f5561683640 1 --2- 172.21.15.67:0/1971875077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a543a6cb70 0x55a543e40120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.808+0000 7f5561683640 1 -- 172.21.15.67:0/1971875077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a543a8d850 msgr2=0x55a543e4b930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:44:43.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.808+0000 7f5561683640 1 --2- 172.21.15.67:0/1971875077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a543a8d850 0x55a543e4b930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.808+0000 7f5561683640 1 -- 172.21.15.67:0/1971875077 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a543e4be70 con 0x55a543a64990 2024-09-21T06:44:43.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.809+0000 7f5561683640 1 --2- 172.21.15.67:0/1971875077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a543a64990 0x55a543e3fbe0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f554c0cdfb0 tx=0x7f554c10b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f5374ef2333b57 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:43.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.809+0000 7f5540ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:44:43.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.809+0000 7f5540ff9640 1 -- 172.21.15.67:0/1971875077 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c0ca670 con 0x55a543a64990 2024-09-21T06:44:43.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.809+0000 7f5569e5b740 1 -- 172.21.15.67:0/1971875077 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f554c115030 con 0x55a543a64990 2024-09-21T06:44:43.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.809+0000 7f5540ff9640 1 -- 172.21.15.67:0/1971875077 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f554c0d27f0 con 0x55a543a64990 2024-09-21T06:44:43.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.810+0000 7f5540ff9640 1 -- 172.21.15.67:0/1971875077 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f554c02f860 con 0x55a543a64990 2024-09-21T06:44:43.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.812+0000 7f5569e5b740 10 client.15798 fetch_fsmap learned FSMap version 21 2024-09-21T06:44:43.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.812+0000 7f5569e5b740 1 -- 172.21.15.67:0/1971875077 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a543e4e770 con 0x55a543a64990 2024-09-21T06:44:43.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.812+0000 7f5540ff9640 1 -- 172.21.15.67:0/1971875077 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f554c115030 con 0x55a543a64990 2024-09-21T06:44:43.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.812+0000 7f5540ff9640 1 -- 172.21.15.67:0/1971875077 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f554c0ca290 con 0x55a543a64990 2024-09-21T06:44:43.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.812+0000 7f5569e5b740 10 client.15798 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:44:43.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.812+0000 7f5569e5b740 10 client.15798 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:44:43.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.812+0000 7f5569e5b740 20 client.15798 populate_metadata read hostname 'smithi067' 2024-09-21T06:44:43.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.812+0000 7f5569e5b740 1 --2- 172.21.15.67:0/1971875077 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55a543a4c1d0 0x55a543e51d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:43.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.813+0000 7f5569e5b740 4 client.15798 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:44:43.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.813+0000 7f5569e5b740 1 -- 172.21.15.67:0/1971875077 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a543dacb00 con 0x55a543a4c1d0 2024-09-21T06:44:43.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.813+0000 7f5560e82640 1 --2- 172.21.15.67:0/1971875077 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55a543a4c1d0 0x55a543e51d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:43.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.815+0000 7f5540ff9640 10 client.15798 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:44:43.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.815+0000 7f5560e82640 1 --2- 172.21.15.67:0/1971875077 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55a543a4c1d0 0x55a543e51d50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d939396b92f00ae1 server_cookie=3f6a399cb030c2bb in_seq=0 out_seq=0 2024-09-21T06:44:43.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.816+0000 7f5540ff9640 1 -- 172.21.15.67:0/1971875077 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a543dacb00 con 0x55a543a4c1d0 2024-09-21T06:44:43.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.816+0000 7f5540ff9640 10 client.15798 handle_command_reply: tid=0 2024-09-21T06:44:43.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.816+0000 7f5569e5b740 1 client.15798 shutdown 2024-09-21T06:44:43.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.820+0000 7f5569e5b740 1 -- 172.21.15.67:0/1971875077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a543a64990 msgr2=0x55a543e3fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:43.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.820+0000 7f5569e5b740 1 --2- 172.21.15.67:0/1971875077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a543a64990 0x55a543e3fbe0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f554c0cdfb0 tx=0x7f554c10b720 comp rx=0 tx=0).stop 2024-09-21T06:44:43.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.823+0000 7f5569e5b740 1 -- 172.21.15.67:0/1971875077 shutdown_connections 2024-09-21T06:44:43.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.823+0000 7f5569e5b740 1 --2- 172.21.15.67:0/1971875077 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55a543a4c1d0 0x55a543e51d50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.823+0000 7f5569e5b740 1 --2- 172.21.15.67:0/1971875077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a543a8d850 0x55a543e4b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.823+0000 7f5569e5b740 1 --2- 172.21.15.67:0/1971875077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a543a6cb70 0x55a543e40120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.823+0000 7f5569e5b740 1 --2- 172.21.15.67:0/1971875077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a543a64990 0x55a543e3fbe0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.823+0000 7f5569e5b740 1 -- 172.21.15.67:0/1971875077 >> 172.21.15.67:0/1971875077 conn(0x55a543a81c40 msgr2=0x55a543e37210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:43.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.824+0000 7f5540ff9640 0 client.15798 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:44:43.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.829+0000 7f5569e5b740 1 -- 172.21.15.67:0/1971875077 shutdown_connections 2024-09-21T06:44:43.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.833+0000 7f5569e5b740 1 -- 172.21.15.67:0/1971875077 wait complete. 2024-09-21T06:44:43.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.833+0000 7f5569e5b740 20 client.15798 trim_cache size 0 max 16384 2024-09-21T06:44:43.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:43 smithi067 ceph-mon[31232]: pgmap v604: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.9 MiB/s wr, 153 op/s 2024-09-21T06:44:43.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:43 smithi067 ceph-mon[31232]: Adjusting osd_memory_target on smithi203 to 176.0M 2024-09-21T06:44:43.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:43 smithi067 ceph-mon[31232]: Unable to set osd_memory_target on smithi203 to 184550195: error parsing value: Value '184550195' is below minimum 939524096 2024-09-21T06:44:43.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.868+0000 7f5569e5b740 1 Processor -- start 2024-09-21T06:44:43.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.869+0000 7f5569e5b740 1 -- start start 2024-09-21T06:44:43.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.869+0000 7f5569e5b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a543a64990 0x55a543e37b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:43.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.870+0000 7f5569e5b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a543a6cb70 0x55a543e3f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:43.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.870+0000 7f5569e5b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a543a8d850 0x55a543e3cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:43.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.870+0000 7f5569e5b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f554800dd70 con 0x55a543a64990 2024-09-21T06:44:43.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.870+0000 7f5569e5b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f55480105f0 con 0x55a543a6cb70 2024-09-21T06:44:43.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.870+0000 7f5569e5b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f554800e3e0 con 0x55a543a8d850 2024-09-21T06:44:43.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.870+0000 7f5561683640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a543a64990 0x55a543e37b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:43.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.870+0000 7f5561683640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a543a64990 0x55a543e37b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43728/0 (socket says 172.21.15.67:43728) 2024-09-21T06:44:43.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.870+0000 7f5561683640 1 -- 172.21.15.67:0/1182965251 learned_addr learned my addr 172.21.15.67:0/1182965251 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:44:43.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.870+0000 7f5560e82640 1 --2- 172.21.15.67:0/1182965251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a543a6cb70 0x55a543e3f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:43.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.870+0000 7f5561683640 1 -- 172.21.15.67:0/1182965251 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a543a8d850 msgr2=0x55a543e3cbd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:44:43.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.870+0000 7f5561683640 1 --2- 172.21.15.67:0/1182965251 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a543a8d850 0x55a543e3cbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.870+0000 7f5561683640 1 -- 172.21.15.67:0/1182965251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a543a6cb70 msgr2=0x55a543e3f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:44:43.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.870+0000 7f5561683640 1 --2- 172.21.15.67:0/1182965251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a543a6cb70 0x55a543e3f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.870+0000 7f5561683640 1 -- 172.21.15.67:0/1182965251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a543e42390 con 0x55a543a64990 2024-09-21T06:44:43.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.871+0000 7f5561683640 1 --2- 172.21.15.67:0/1182965251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a543a64990 0x55a543e37b40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f554c0f3040 tx=0x7f554c110570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112f2823992c180a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:43.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.871+0000 7f5542ffd640 1 -- 172.21.15.67:0/1182965251 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c0cd790 con 0x55a543a64990 2024-09-21T06:44:43.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.871+0000 7f5542ffd640 1 -- 172.21.15.67:0/1182965251 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f554c115030 con 0x55a543a64990 2024-09-21T06:44:43.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.872+0000 7f5569e5b740 1 -- 172.21.15.67:0/1182965251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a543a64990 msgr2=0x55a543e37b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:43.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.872+0000 7f5569e5b740 1 --2- 172.21.15.67:0/1182965251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a543a64990 0x55a543e37b40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f554c0f3040 tx=0x7f554c110570 comp rx=0 tx=0).stop 2024-09-21T06:44:43.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.876+0000 7f5569e5b740 1 -- 172.21.15.67:0/1182965251 shutdown_connections 2024-09-21T06:44:43.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.876+0000 7f5569e5b740 1 --2- 172.21.15.67:0/1182965251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a543a6cb70 0x55a543e3f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.876+0000 7f5569e5b740 1 --2- 172.21.15.67:0/1182965251 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a543a8d850 0x55a543e3cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.876+0000 7f5569e5b740 1 --2- 172.21.15.67:0/1182965251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a543a64990 0x55a543e37b40 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.876+0000 7f5569e5b740 1 -- 172.21.15.67:0/1182965251 >> 172.21.15.67:0/1182965251 conn(0x55a543a81c40 msgr2=0x55a543e31df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:43.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.881+0000 7f5569e5b740 1 -- 172.21.15.67:0/1182965251 shutdown_connections 2024-09-21T06:44:43.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.881+0000 7f5569e5b740 1 -- 172.21.15.67:0/1182965251 wait complete. 2024-09-21T06:44:43.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.881+0000 7f5569e5b740 1 -- start start 2024-09-21T06:44:43.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.881+0000 7f5569e5b740 1 Processor -- start 2024-09-21T06:44:43.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.882+0000 7f5569e5b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a543a64990 0x55a543e481c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:43.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.882+0000 7f5569e5b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a543a6cb70 0x55a543e43670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:43.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.882+0000 7f5569e5b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a543a8d850 0x55a543e43cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:43.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.882+0000 7f5569e5b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f554800eab0 con 0x55a543a6cb70 2024-09-21T06:44:43.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.882+0000 7f5569e5b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f554800ff20 con 0x55a543a64990 2024-09-21T06:44:43.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.882+0000 7f5569e5b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f554800f180 con 0x55a543a8d850 2024-09-21T06:44:43.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.882+0000 7f5569e5b740 10 client.0 _ll_register_callbacks cb 0x7f554c110300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:44:43.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.884+0000 7f5561683640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a543a64990 0x55a543e481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:43.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.884+0000 7f5560e82640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a543a6cb70 0x55a543e43670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:43.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.884+0000 7f5560e82640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a543a6cb70 0x55a543e43670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43740/0 (socket says 172.21.15.67:43740) 2024-09-21T06:44:43.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.884+0000 7f5560e82640 1 -- 172.21.15.67:0/2396962648 learned_addr learned my addr 172.21.15.67:0/2396962648 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:44:43.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.886+0000 7f5561e84640 1 --2- 172.21.15.67:0/2396962648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a543a8d850 0x55a543e43cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:43.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.886+0000 7f5560e82640 1 -- 172.21.15.67:0/2396962648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a543a8d850 msgr2=0x55a543e43cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:44:43.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.886+0000 7f5560e82640 1 --2- 172.21.15.67:0/2396962648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a543a8d850 0x55a543e43cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.886+0000 7f5560e82640 1 -- 172.21.15.67:0/2396962648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a543a64990 msgr2=0x55a543e481c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:44:43.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.886+0000 7f5560e82640 1 --2- 172.21.15.67:0/2396962648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a543a64990 0x55a543e481c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.886+0000 7f5560e82640 1 -- 172.21.15.67:0/2396962648 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a543e44450 con 0x55a543a6cb70 2024-09-21T06:44:43.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.886+0000 7f5560e82640 1 --2- 172.21.15.67:0/2396962648 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a543a6cb70 0x55a543e43670 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5550009730 tx=0x7f5550015790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1d36c0dd7a8d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:43.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.887+0000 7f5540ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:44:43.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.887+0000 7f5540ff9640 1 -- 172.21.15.67:0/2396962648 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555001f070 con 0x55a543a6cb70 2024-09-21T06:44:43.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.887+0000 7f5540ff9640 1 -- 172.21.15.67:0/2396962648 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f5550016430 con 0x55a543a6cb70 2024-09-21T06:44:43.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.887+0000 7f5569e5b740 1 -- 172.21.15.67:0/2396962648 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f554c115030 con 0x55a543a6cb70 2024-09-21T06:44:43.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.888+0000 7f5540ff9640 1 -- 172.21.15.67:0/2396962648 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5550029050 con 0x55a543a6cb70 2024-09-21T06:44:43.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.889+0000 7f5569e5b740 10 client.15804 fetch_fsmap learned FSMap version 21 2024-09-21T06:44:43.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.889+0000 7f5569e5b740 1 -- 172.21.15.67:0/2396962648 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a543ebc810 con 0x55a543a6cb70 2024-09-21T06:44:43.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.891+0000 7f5540ff9640 1 -- 172.21.15.67:0/2396962648 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f554c115030 con 0x55a543a6cb70 2024-09-21T06:44:43.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.891+0000 7f5540ff9640 1 -- 172.21.15.67:0/2396962648 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f55500245e0 con 0x55a543a6cb70 2024-09-21T06:44:43.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.892+0000 7f5569e5b740 10 client.15804 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:44:43.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.892+0000 7f5569e5b740 10 client.15804 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:44:43.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.892+0000 7f5569e5b740 20 client.15804 populate_metadata read hostname 'smithi067' 2024-09-21T06:44:43.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.892+0000 7f5569e5b740 1 --2- 172.21.15.67:0/2396962648 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55a543a21580 0x55a543ebf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:43.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.892+0000 7f5569e5b740 4 client.15804 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.05, "await_for": 45.0}] 2024-09-21T06:44:43.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.892+0000 7f5569e5b740 1 -- 172.21.15.67:0/2396962648 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.05, "await_for": 45.0}) -- 0x55a543da85b0 con 0x55a543a21580 2024-09-21T06:44:43.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.893+0000 7f5561683640 1 --2- 172.21.15.67:0/2396962648 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55a543a21580 0x55a543ebf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:43.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.899+0000 7f5561683640 1 --2- 172.21.15.67:0/2396962648 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55a543a21580 0x55a543ebf040 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=111866546d6caf1 server_cookie=7f9da9a19162b9fa in_seq=0 out_seq=0 2024-09-21T06:44:43.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.899+0000 7f5540ff9640 10 client.15804 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:44:43.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.944+0000 7f5540ff9640 1 -- 172.21.15.67:0/2396962648 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a543da85b0 con 0x55a543a21580 2024-09-21T06:44:43.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.944+0000 7f5540ff9640 10 client.15804 handle_command_reply: tid=0 2024-09-21T06:44:43.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.944+0000 7f5569e5b740 1 client.15804 shutdown 2024-09-21T06:44:43.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.944+0000 7f5569e5b740 1 -- 172.21.15.67:0/2396962648 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a543a6cb70 msgr2=0x55a543e43670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:43.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.944+0000 7f5569e5b740 1 --2- 172.21.15.67:0/2396962648 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a543a6cb70 0x55a543e43670 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5550009730 tx=0x7f5550015790 comp rx=0 tx=0).stop 2024-09-21T06:44:43.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.944+0000 7f5569e5b740 1 -- 172.21.15.67:0/2396962648 shutdown_connections 2024-09-21T06:44:43.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.944+0000 7f5569e5b740 1 --2- 172.21.15.67:0/2396962648 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55a543a21580 0x55a543ebf040 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.944+0000 7f5569e5b740 1 --2- 172.21.15.67:0/2396962648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a543a8d850 0x55a543e43cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.944+0000 7f5569e5b740 1 --2- 172.21.15.67:0/2396962648 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a543a6cb70 0x55a543e43670 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.944+0000 7f5569e5b740 1 --2- 172.21.15.67:0/2396962648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a543a64990 0x55a543e481c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.944+0000 7f5569e5b740 1 -- 172.21.15.67:0/2396962648 >> 172.21.15.67:0/2396962648 conn(0x55a543a81c40 msgr2=0x55a543e30530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:43.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.944+0000 7f5540ff9640 0 client.15804 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:44:43.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.944+0000 7f5569e5b740 1 -- 172.21.15.67:0/2396962648 shutdown_connections 2024-09-21T06:44:43.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.944+0000 7f5569e5b740 1 -- 172.21.15.67:0/2396962648 wait complete. 2024-09-21T06:44:43.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.944+0000 7f5569e5b740 20 client.15804 trim_cache size 0 max 16384 2024-09-21T06:44:43.947 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-21T06:44:43.947 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 21, 2024-09-21T06:44:43.947 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-09-21T06:44:43.947 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 34, 2024-09-21T06:44:43.947 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-21T06:44:43.948 INFO:teuthology.orchestra.run.smithi067.stdout: "70ca9b66": { 2024-09-21T06:44:43.948 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 34, 2024-09-21T06:44:43.948 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-21T06:44:43.948 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:44:43.948 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-21T06:44:43.948 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:44:43.948 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:44:43.948 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-21T06:44:43.948 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 141.1, 2024-09-21T06:44:43.948 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-21T06:44:43.948 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-21T06:44:43.948 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-21T06:44:43.948 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:44:43.948 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-21T06:44:43.948 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:44:43.948 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:44:43.949 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:44:43.949 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:44:43.949 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:44:43.949 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:44:43.949 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-21T06:44:43.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.945+0000 7f5562685640 1 -- 172.21.15.67:0/1962105876 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f55440b6640 msgr2=0x7f55440b8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:43.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.945+0000 7f5562685640 1 --2- 172.21.15.67:0/1962105876 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f55440b6640 0x7f55440b8b50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f555400b3d0 tx=0x7f5554005ed0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.945+0000 7f5562685640 1 -- 172.21.15.67:0/1962105876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f555c101ff0 msgr2=0x7f555c0749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:43.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.945+0000 7f5562685640 1 --2- 172.21.15.67:0/1962105876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f555c101ff0 0x7f555c0749a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f554c00ac10 tx=0x7f554c003f10 comp rx=0 tx=0).stop 2024-09-21T06:44:43.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.945+0000 7f5562685640 1 -- 172.21.15.67:0/1962105876 shutdown_connections 2024-09-21T06:44:43.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.945+0000 7f5562685640 1 --2- 172.21.15.67:0/1962105876 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f55440b6640 0x7f55440b8b50 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.945+0000 7f5562685640 1 --2- 172.21.15.67:0/1962105876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f555c108280 0x7f555c06ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.945+0000 7f5562685640 1 --2- 172.21.15.67:0/1962105876 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f555c107930 0x7f555c074f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.945+0000 7f5562685640 1 --2- 172.21.15.67:0/1962105876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f555c101ff0 0x7f555c0749a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:43.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.945+0000 7f5562685640 1 -- 172.21.15.67:0/1962105876 >> 172.21.15.67:0/1962105876 conn(0x7f555c077a50 msgr2=0x7f555c1059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:43.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.946+0000 7f5562685640 1 -- 172.21.15.67:0/1962105876 shutdown_connections 2024-09-21T06:44:43.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:44:43.946+0000 7f5562685640 1 -- 172.21.15.67:0/1962105876 wait complete. 2024-09-21T06:44:44.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:43 smithi100 ceph-mon[33401]: pgmap v604: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.9 MiB/s wr, 153 op/s 2024-09-21T06:44:44.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:43 smithi100 ceph-mon[33401]: Adjusting osd_memory_target on smithi203 to 176.0M 2024-09-21T06:44:44.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:43 smithi100 ceph-mon[33401]: Unable to set osd_memory_target on smithi203 to 184550195: error parsing value: Value '184550195' is below minimum 939524096 2024-09-21T06:44:44.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:43 smithi203 ceph-mon[33735]: pgmap v604: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.9 MiB/s wr, 153 op/s 2024-09-21T06:44:44.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:43 smithi203 ceph-mon[33735]: Adjusting osd_memory_target on smithi203 to 176.0M 2024-09-21T06:44:44.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:43 smithi203 ceph-mon[33735]: Unable to set osd_memory_target on smithi203 to 184550195: error parsing value: Value '184550195' is below minimum 939524096 2024-09-21T06:44:45.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:44 smithi067 ceph-mon[31232]: pgmap v605: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.4 MiB/s wr, 162 op/s 2024-09-21T06:44:45.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:44 smithi100 ceph-mon[33401]: pgmap v605: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.4 MiB/s wr, 162 op/s 2024-09-21T06:44:45.144 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '70ca9b66', quiesce took 5.3 seconds. Will release after: 15.75 2024-09-21T06:44:45.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:44 smithi203 ceph-mon[33735]: pgmap v605: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.4 MiB/s wr, 162 op/s 2024-09-21T06:44:47.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:47 smithi203 ceph-mon[33735]: pgmap v606: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 199 op/s 2024-09-21T06:44:47.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:47 smithi067 ceph-mon[31232]: pgmap v606: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 199 op/s 2024-09-21T06:44:47.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:47 smithi100 ceph-mon[33401]: pgmap v606: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 199 op/s 2024-09-21T06:44:49.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:49 smithi203 ceph-mon[33735]: pgmap v607: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 15 MiB/s wr, 283 op/s 2024-09-21T06:44:49.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:49 smithi067 ceph-mon[31232]: pgmap v607: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 15 MiB/s wr, 283 op/s 2024-09-21T06:44:49.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:49 smithi100 ceph-mon[33401]: pgmap v607: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 15 MiB/s wr, 283 op/s 2024-09-21T06:44:51.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:51 smithi203 ceph-mon[33735]: pgmap v608: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 14 MiB/s wr, 280 op/s 2024-09-21T06:44:51.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:51 smithi067 ceph-mon[31232]: pgmap v608: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 14 MiB/s wr, 280 op/s 2024-09-21T06:44:51.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:51 smithi100 ceph-mon[33401]: pgmap v608: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 14 MiB/s wr, 280 op/s 2024-09-21T06:44:53.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:53 smithi203 ceph-mon[33735]: pgmap v609: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 15 MiB/s wr, 289 op/s 2024-09-21T06:44:53.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:53 smithi067 ceph-mon[31232]: pgmap v609: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 15 MiB/s wr, 289 op/s 2024-09-21T06:44:53.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:53 smithi100 ceph-mon[33401]: pgmap v609: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 15 MiB/s wr, 289 op/s 2024-09-21T06:44:55.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:55 smithi203 ceph-mon[33735]: pgmap v610: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 10 MiB/s wr, 196 op/s 2024-09-21T06:44:55.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:55 smithi067 ceph-mon[31232]: pgmap v610: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 10 MiB/s wr, 196 op/s 2024-09-21T06:44:55.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:55 smithi100 ceph-mon[33401]: pgmap v610: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 10 MiB/s wr, 196 op/s 2024-09-21T06:44:57.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:57 smithi203 ceph-mon[33735]: pgmap v611: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.3 MiB/s wr, 141 op/s 2024-09-21T06:44:57.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:57 smithi067 ceph-mon[31232]: pgmap v611: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.3 MiB/s wr, 141 op/s 2024-09-21T06:44:57.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:57 smithi100 ceph-mon[33401]: pgmap v611: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.3 MiB/s wr, 141 op/s 2024-09-21T06:44:59.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:44:59 smithi203 ceph-mon[33735]: pgmap v612: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.3 MiB/s wr, 102 op/s 2024-09-21T06:44:59.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:44:59 smithi067 ceph-mon[31232]: pgmap v612: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.3 MiB/s wr, 102 op/s 2024-09-21T06:44:59.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:44:59 smithi100 ceph-mon[33401]: pgmap v612: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.3 MiB/s wr, 102 op/s 2024-09-21T06:45:00.895 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:45:01.168 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:45:01.289 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:01 smithi067 ceph-mon[31232]: pgmap v613: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 838 KiB/s wr, 15 op/s 2024-09-21T06:45:01.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:01 smithi203 ceph-mon[33735]: pgmap v613: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 838 KiB/s wr, 15 op/s 2024-09-21T06:45:01.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:01 smithi100 ceph-mon[33401]: pgmap v613: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 838 KiB/s wr, 15 op/s 2024-09-21T06:45:01.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.664+0000 7f8fbd592640 1 -- 172.21.15.67:0/561586541 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fb81094b0 msgr2=0x7f8fb81098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:45:01.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.664+0000 7f8fbd592640 1 --2- 172.21.15.67:0/561586541 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fb81094b0 0x7f8fb81098c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8fac009900 tx=0x7f8fac02a410 comp rx=0 tx=0).stop 2024-09-21T06:45:01.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.664+0000 7f8fbd592640 1 -- 172.21.15.67:0/561586541 shutdown_connections 2024-09-21T06:45:01.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.664+0000 7f8fbd592640 1 --2- 172.21.15.67:0/561586541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fb8109e00 0x7f8fb8110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:01.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.664+0000 7f8fbd592640 1 --2- 172.21.15.67:0/561586541 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fb81094b0 0x7f8fb81098c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:01.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.664+0000 7f8fbd592640 1 --2- 172.21.15.67:0/561586541 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fb8103b70 0x7f8fb8103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:01.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.664+0000 7f8fbd592640 1 -- 172.21.15.67:0/561586541 >> 172.21.15.67:0/561586541 conn(0x7f8fb80ff5e0 msgr2=0x7f8fb8101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:45:01.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.664+0000 7f8fbd592640 1 -- 172.21.15.67:0/561586541 shutdown_connections 2024-09-21T06:45:01.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.664+0000 7f8fbd592640 1 -- 172.21.15.67:0/561586541 wait complete. 2024-09-21T06:45:01.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.664+0000 7f8fbd592640 1 Processor -- start 2024-09-21T06:45:01.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.665+0000 7f8fbd592640 1 -- start start 2024-09-21T06:45:01.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.665+0000 7f8fbd592640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fb8103b70 0x7f8fb8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:45:01.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.665+0000 7f8fbd592640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fb81094b0 0x7f8fb8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:45:01.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.665+0000 7f8fbd592640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fb8109e00 0x7f8fb8076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:45:01.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.665+0000 7f8fbd592640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb8113970 con 0x7f8fb8109e00 2024-09-21T06:45:01.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.665+0000 7f8fbd592640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb81137f0 con 0x7f8fb81094b0 2024-09-21T06:45:01.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.665+0000 7f8fbd592640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb8113af0 con 0x7f8fb8103b70 2024-09-21T06:45:01.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.665+0000 7f8fb77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fb8109e00 0x7f8fb8076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:45:01.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.665+0000 7f8fb77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fb8109e00 0x7f8fb8076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39694/0 (socket says 172.21.15.67:39694) 2024-09-21T06:45:01.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.665+0000 7f8fb77fe640 1 -- 172.21.15.67:0/1592334630 learned_addr learned my addr 172.21.15.67:0/1592334630 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:45:01.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.665+0000 7f8fb6ffd640 1 --2- 172.21.15.67:0/1592334630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fb8103b70 0x7f8fb8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:45:01.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.665+0000 7f8fb67fc640 1 --2- 172.21.15.67:0/1592334630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fb81094b0 0x7f8fb8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:45:01.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.666+0000 7f8fb77fe640 1 -- 172.21.15.67:0/1592334630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fb8103b70 msgr2=0x7f8fb8075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:45:01.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.666+0000 7f8fb77fe640 1 --2- 172.21.15.67:0/1592334630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fb8103b70 0x7f8fb8075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:01.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.666+0000 7f8fb77fe640 1 -- 172.21.15.67:0/1592334630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fb81094b0 msgr2=0x7f8fb8075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:45:01.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.666+0000 7f8fb77fe640 1 --2- 172.21.15.67:0/1592334630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fb81094b0 0x7f8fb8075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:01.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.666+0000 7f8fb77fe640 1 -- 172.21.15.67:0/1592334630 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb81a86a0 con 0x7f8fb8109e00 2024-09-21T06:45:01.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.666+0000 7f8fb77fe640 1 --2- 172.21.15.67:0/1592334630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fb8109e00 0x7f8fb8076190 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000eaa0 tx=0x7f8fb000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:45:01.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.666+0000 7f8f9ffff640 1 -- 172.21.15.67:0/1592334630 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb000cdb0 con 0x7f8fb8109e00 2024-09-21T06:45:01.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.666+0000 7f8fbd592640 1 -- 172.21.15.67:0/1592334630 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb81a8990 con 0x7f8fb8109e00 2024-09-21T06:45:01.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.666+0000 7f8f9ffff640 1 -- 172.21.15.67:0/1592334630 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8fb0005020 con 0x7f8fb8109e00 2024-09-21T06:45:01.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.666+0000 7f8f9ffff640 1 -- 172.21.15.67:0/1592334630 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb00116d0 con 0x7f8fb8109e00 2024-09-21T06:45:01.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.666+0000 7f8fbd592640 1 -- 172.21.15.67:0/1592334630 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb81a8e50 con 0x7f8fb8109e00 2024-09-21T06:45:01.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.668+0000 7f8fbd592640 1 -- 172.21.15.67:0/1592334630 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb8103f80 con 0x7f8fb8109e00 2024-09-21T06:45:01.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.669+0000 7f8f9ffff640 1 -- 172.21.15.67:0/1592334630 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f8fb00051d0 con 0x7f8fb8109e00 2024-09-21T06:45:01.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.670+0000 7f8f9ffff640 1 --2- 172.21.15.67:0/1592334630 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f8fa00b6510 0x7f8fa00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:45:01.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.670+0000 7f8fb6ffd640 1 --2- 172.21.15.67:0/1592334630 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f8fa00b6510 0x7f8fa00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:45:01.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.670+0000 7f8f9ffff640 1 -- 172.21.15.67:0/1592334630 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f8fb00e77f0 con 0x7f8fb8109e00 2024-09-21T06:45:01.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.670+0000 7f8fb6ffd640 1 --2- 172.21.15.67:0/1592334630 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f8fa00b6510 0x7f8fa00b8a20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8077160 tx=0x7f8fa800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:45:01.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.674+0000 7f8f9ffff640 1 -- 172.21.15.67:0/1592334630 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8fb0016030 con 0x7f8fb8109e00 2024-09-21T06:45:01.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.904+0000 7f8fbd592640 1 -- 172.21.15.67:0/1592334630 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8fb802e640 con 0x7f8fb8109e00 2024-09-21T06:45:01.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.905+0000 7f8f9ffff640 1 -- 172.21.15.67:0/1592334630 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7f8fb00b2090 con 0x7f8fb8109e00 2024-09-21T06:45:01.906 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:45:01.906 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:45:01.907 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:45:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.907+0000 7f8fbd592640 1 -- 172.21.15.67:0/1592334630 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f8fa00b6510 msgr2=0x7f8fa00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:45:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.907+0000 7f8fbd592640 1 --2- 172.21.15.67:0/1592334630 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f8fa00b6510 0x7f8fa00b8a20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8077160 tx=0x7f8fa800b390 comp rx=0 tx=0).stop 2024-09-21T06:45:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.907+0000 7f8fbd592640 1 -- 172.21.15.67:0/1592334630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fb8109e00 msgr2=0x7f8fb8076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:45:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.907+0000 7f8fbd592640 1 --2- 172.21.15.67:0/1592334630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fb8109e00 0x7f8fb8076190 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000eaa0 tx=0x7f8fb000ef60 comp rx=0 tx=0).stop 2024-09-21T06:45:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.907+0000 7f8fbd592640 1 -- 172.21.15.67:0/1592334630 shutdown_connections 2024-09-21T06:45:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.907+0000 7f8fbd592640 1 --2- 172.21.15.67:0/1592334630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fb81094b0 0x7f8fb8075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.907+0000 7f8fbd592640 1 --2- 172.21.15.67:0/1592334630 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f8fa00b6510 0x7f8fa00b8a20 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.907+0000 7f8fbd592640 1 --2- 172.21.15.67:0/1592334630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fb8109e00 0x7f8fb8076190 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.907+0000 7f8fbd592640 1 --2- 172.21.15.67:0/1592334630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fb8103b70 0x7f8fb8075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.907+0000 7f8fbd592640 1 -- 172.21.15.67:0/1592334630 >> 172.21.15.67:0/1592334630 conn(0x7f8fb80ff5e0 msgr2=0x7f8fb8101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:45:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.908+0000 7f8fbd592640 1 -- 172.21.15.67:0/1592334630 shutdown_connections 2024-09-21T06:45:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:01.908+0000 7f8fbd592640 1 -- 172.21.15.67:0/1592334630 wait complete. 2024-09-21T06:45:02.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:02 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1592334630' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:45:02.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:02 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1592334630' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:45:02.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:02 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1592334630' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:45:02.689 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db --set-id 70ca9b66 --release --await' 2024-09-21T06:45:02.689 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24671 quiesce db --set-id 70ca9b66 --release --await 2024-09-21T06:45:02.956 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:45:03.398 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:03 smithi067 ceph-mon[31232]: pgmap v614: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 662 KiB/s wr, 12 op/s 2024-09-21T06:45:03.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:03 smithi203 ceph-mon[33735]: pgmap v614: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 662 KiB/s wr, 12 op/s 2024-09-21T06:45:03.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.497+0000 7f4b46d99640 1 -- 172.21.15.67:0/2324324563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b40108d10 msgr2=0x7f4b4010f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:45:03.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.497+0000 7f4b46d99640 1 --2- 172.21.15.67:0/2324324563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b40108d10 0x7f4b4010f540 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4b38009ca0 tx=0x7f4b3802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.497+0000 7f4b46d99640 1 -- 172.21.15.67:0/2324324563 shutdown_connections 2024-09-21T06:45:03.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.497+0000 7f4b46d99640 1 --2- 172.21.15.67:0/2324324563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b401083c0 0x7f4b401087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.497+0000 7f4b46d99640 1 --2- 172.21.15.67:0/2324324563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b40108d10 0x7f4b4010f540 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.497+0000 7f4b46d99640 1 --2- 172.21.15.67:0/2324324563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b40102a80 0x7f4b40102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.497+0000 7f4b46d99640 1 -- 172.21.15.67:0/2324324563 >> 172.21.15.67:0/2324324563 conn(0x7f4b40079740 msgr2=0x7f4b40079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:45:03.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.497+0000 7f4b46d99640 1 -- 172.21.15.67:0/2324324563 shutdown_connections 2024-09-21T06:45:03.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.497+0000 7f4b46d99640 1 -- 172.21.15.67:0/2324324563 wait complete. 2024-09-21T06:45:03.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.498+0000 7f4b46d99640 1 Processor -- start 2024-09-21T06:45:03.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.498+0000 7f4b46d99640 1 -- start start 2024-09-21T06:45:03.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.498+0000 7f4b46d99640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b40102a80 0x7f4b4019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:45:03.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.498+0000 7f4b46d99640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b401083c0 0x7f4b4019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:45:03.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.498+0000 7f4b46d99640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b40108d10 0x7f4b4019fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:45:03.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.498+0000 7f4b46d99640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4b40112880 con 0x7f4b40102a80 2024-09-21T06:45:03.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.498+0000 7f4b46d99640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4b40112700 con 0x7f4b401083c0 2024-09-21T06:45:03.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.498+0000 7f4b46d99640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4b40112a00 con 0x7f4b40108d10 2024-09-21T06:45:03.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.498+0000 7f4b44b35640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b40102a80 0x7f4b4019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:45:03.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.498+0000 7f4b44b35640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b40102a80 0x7f4b4019a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39712/0 (socket says 172.21.15.67:39712) 2024-09-21T06:45:03.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.498+0000 7f4b44b35640 1 -- 172.21.15.67:0/3865973431 learned_addr learned my addr 172.21.15.67:0/3865973431 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:45:03.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.498+0000 7f4b45336640 1 --2- 172.21.15.67:0/3865973431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b40108d10 0x7f4b4019fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:45:03.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.498+0000 7f4b3ffff640 1 --2- 172.21.15.67:0/3865973431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b401083c0 0x7f4b4019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:45:03.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.499+0000 7f4b44b35640 1 -- 172.21.15.67:0/3865973431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b40108d10 msgr2=0x7f4b4019fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:45:03.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.499+0000 7f4b44b35640 1 --2- 172.21.15.67:0/3865973431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b40108d10 0x7f4b4019fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.499+0000 7f4b44b35640 1 -- 172.21.15.67:0/3865973431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b401083c0 msgr2=0x7f4b4019a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:45:03.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.499+0000 7f4b44b35640 1 --2- 172.21.15.67:0/3865973431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b401083c0 0x7f4b4019a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.499+0000 7f4b44b35640 1 -- 172.21.15.67:0/3865973431 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b401a0250 con 0x7f4b40102a80 2024-09-21T06:45:03.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.499+0000 7f4b45336640 1 --2- 172.21.15.67:0/3865973431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b40108d10 0x7f4b4019fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:45:03.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.499+0000 7f4b44b35640 1 --2- 172.21.15.67:0/3865973431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b40102a80 0x7f4b4019a3a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4b3400c920 tx=0x7f4b3400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:45:03.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.499+0000 7f4b3dffb640 1 -- 172.21.15.67:0/3865973431 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34007c70 con 0x7f4b40102a80 2024-09-21T06:45:03.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.499+0000 7f4b3dffb640 1 -- 172.21.15.67:0/3865973431 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b34007e10 con 0x7f4b40102a80 2024-09-21T06:45:03.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.499+0000 7f4b46d99640 1 -- 172.21.15.67:0/3865973431 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b401a0540 con 0x7f4b40102a80 2024-09-21T06:45:03.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.499+0000 7f4b3dffb640 1 -- 172.21.15.67:0/3865973431 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34018710 con 0x7f4b40102a80 2024-09-21T06:45:03.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.499+0000 7f4b46d99640 1 -- 172.21.15.67:0/3865973431 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b401a6d60 con 0x7f4b40102a80 2024-09-21T06:45:03.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.502+0000 7f4b3dffb640 1 -- 172.21.15.67:0/3865973431 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f4b3400f450 con 0x7f4b40102a80 2024-09-21T06:45:03.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.502+0000 7f4b4d30d740 1 Processor -- start 2024-09-21T06:45:03.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.502+0000 7f4b4d30d740 1 -- start start 2024-09-21T06:45:03.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.502+0000 7f4b4d30d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fce9b2f520 0x55fce9ed5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:45:03.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.502+0000 7f4b4d30d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fce9b06660 0x55fce9ed5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:45:03.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.502+0000 7f4b4d30d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce9b0e840 0x55fce9ed5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:45:03.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.502+0000 7f4b4d30d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55fce9dc0dc0 con 0x55fce9b06660 2024-09-21T06:45:03.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.502+0000 7f4b4d30d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55fce9e4d1f0 con 0x55fce9b0e840 2024-09-21T06:45:03.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.502+0000 7f4b4d30d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55fce9e3c620 con 0x55fce9b2f520 2024-09-21T06:45:03.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.502+0000 7f4b44b35640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fce9b06660 0x55fce9ed5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:45:03.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.502+0000 7f4b44b35640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fce9b06660 0x55fce9ed5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39720/0 (socket says 172.21.15.67:39720) 2024-09-21T06:45:03.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.503+0000 7f4b44b35640 1 -- 172.21.15.67:0/2345589344 learned_addr learned my addr 172.21.15.67:0/2345589344 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:45:03.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.503+0000 7f4b45336640 1 --2- 172.21.15.67:0/2345589344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fce9b2f520 0x55fce9ed5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:45:03.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.503+0000 7f4b3ffff640 1 --2- 172.21.15.67:0/2345589344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce9b0e840 0x55fce9ed5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:45:03.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.503+0000 7f4b45336640 1 -- 172.21.15.67:0/2345589344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce9b0e840 msgr2=0x55fce9ed5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:45:03.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.503+0000 7f4b45336640 1 --2- 172.21.15.67:0/2345589344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce9b0e840 0x55fce9ed5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.503+0000 7f4b45336640 1 -- 172.21.15.67:0/2345589344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fce9b06660 msgr2=0x55fce9ed5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:45:03.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.503+0000 7f4b3dffb640 1 --2- 172.21.15.67:0/3865973431 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f4b300b65c0 0x7f4b300b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:45:03.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.503+0000 7f4b45336640 1 --2- 172.21.15.67:0/2345589344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fce9b06660 0x55fce9ed5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.503+0000 7f4b45336640 1 -- 172.21.15.67:0/2345589344 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fce9e04fd0 con 0x55fce9b2f520 2024-09-21T06:45:03.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.503+0000 7f4b44b35640 1 --2- 172.21.15.67:0/2345589344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fce9b06660 0x55fce9ed5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:45:03.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.503+0000 7f4b3ffff640 1 --2- 172.21.15.67:0/2345589344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce9b0e840 0x55fce9ed5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:45:03.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.503+0000 7f4b3dffb640 1 -- 172.21.15.67:0/3865973431 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f4b340e7890 con 0x7f4b40102a80 2024-09-21T06:45:03.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.503+0000 7f4b45336640 1 --2- 172.21.15.67:0/3865973431 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f4b300b65c0 0x7f4b300b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:45:03.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.503+0000 7f4b45336640 1 --2- 172.21.15.67:0/2345589344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fce9b2f520 0x55fce9ed5510 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4b3802ade0 tx=0x7f4b38031610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b9d5791ff4c6ac6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:45:03.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.504+0000 7f4b2affd640 1 -- 172.21.15.67:0/2345589344 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3804d070 con 0x55fce9b2f520 2024-09-21T06:45:03.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.504+0000 7f4b2affd640 1 -- 172.21.15.67:0/2345589344 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b38040070 con 0x55fce9b2f520 2024-09-21T06:45:03.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.504+0000 7f4b2affd640 1 -- 172.21.15.67:0/2345589344 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b38030820 con 0x55fce9b2f520 2024-09-21T06:45:03.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.504+0000 7f4b4d30d740 1 -- 172.21.15.67:0/2345589344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fce9b2f520 msgr2=0x55fce9ed5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:45:03.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.504+0000 7f4b4d30d740 1 --2- 172.21.15.67:0/2345589344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fce9b2f520 0x55fce9ed5510 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4b3802ade0 tx=0x7f4b38031610 comp rx=0 tx=0).stop 2024-09-21T06:45:03.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.504+0000 7f4b45336640 1 --2- 172.21.15.67:0/3865973431 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f4b300b65c0 0x7f4b300b8ad0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4b3804a000 tx=0x7f4b3804b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:45:03.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.505+0000 7f4b4d30d740 1 -- 172.21.15.67:0/2345589344 shutdown_connections 2024-09-21T06:45:03.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.505+0000 7f4b4d30d740 1 --2- 172.21.15.67:0/2345589344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce9b0e840 0x55fce9ed5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.505+0000 7f4b4d30d740 1 --2- 172.21.15.67:0/2345589344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fce9b06660 0x55fce9ed5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.505+0000 7f4b4d30d740 1 --2- 172.21.15.67:0/2345589344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fce9b2f520 0x55fce9ed5510 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.505+0000 7f4b4d30d740 1 -- 172.21.15.67:0/2345589344 >> 172.21.15.67:0/2345589344 conn(0x55fce9b23910 msgr2=0x55fce9ed3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:45:03.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.505+0000 7f4b4d30d740 1 -- 172.21.15.67:0/2345589344 shutdown_connections 2024-09-21T06:45:03.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.505+0000 7f4b4d30d740 1 -- 172.21.15.67:0/2345589344 wait complete. 2024-09-21T06:45:03.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.505+0000 7f4b4d30d740 1 -- start start 2024-09-21T06:45:03.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.505+0000 7f4b4d30d740 1 Processor -- start 2024-09-21T06:45:03.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.506+0000 7f4b4d30d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce9b06660 0x55fce9ee4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:45:03.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.506+0000 7f4b4d30d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fce9b0e840 0x55fce9ee5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:45:03.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.506+0000 7f4b4d30d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fce9b2f520 0x55fce9ee59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:45:03.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.506+0000 7f4b4d30d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55fce9dd7300 con 0x55fce9b2f520 2024-09-21T06:45:03.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.506+0000 7f4b4d30d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55fce9e40430 con 0x55fce9b06660 2024-09-21T06:45:03.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.506+0000 7f4b4d30d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55fce9c9dbb0 con 0x55fce9b0e840 2024-09-21T06:45:03.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.506+0000 7f4b4d30d740 10 client.0 _ll_register_callbacks cb 0x55fce9df1120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:45:03.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.506+0000 7f4b45336640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fce9b2f520 0x55fce9ee59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:45:03.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.506+0000 7f4b44b35640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce9b06660 0x55fce9ee4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:45:03.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.506+0000 7f4b45336640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fce9b2f520 0x55fce9ee59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39722/0 (socket says 172.21.15.67:39722) 2024-09-21T06:45:03.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.506+0000 7f4b45336640 1 -- 172.21.15.67:0/927459785 learned_addr learned my addr 172.21.15.67:0/927459785 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:45:03.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.506+0000 7f4b3ffff640 1 --2- 172.21.15.67:0/927459785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fce9b0e840 0x55fce9ee5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:45:03.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.506+0000 7f4b45336640 1 -- 172.21.15.67:0/927459785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fce9b0e840 msgr2=0x55fce9ee5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:45:03.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.506+0000 7f4b45336640 1 --2- 172.21.15.67:0/927459785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fce9b0e840 0x55fce9ee5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.506+0000 7f4b45336640 1 -- 172.21.15.67:0/927459785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce9b06660 msgr2=0x55fce9ee4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:45:03.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.506+0000 7f4b45336640 1 --2- 172.21.15.67:0/927459785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce9b06660 0x55fce9ee4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.506+0000 7f4b45336640 1 -- 172.21.15.67:0/927459785 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fce9dc6780 con 0x55fce9b2f520 2024-09-21T06:45:03.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.506+0000 7f4b45336640 1 --2- 172.21.15.67:0/927459785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fce9b2f520 0x55fce9ee59d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4b3802a900 tx=0x7f4b38004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2107c6be86f393bf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:45:03.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.506+0000 7f4b1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:45:03.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.507+0000 7f4b1ffff640 1 -- 172.21.15.67:0/927459785 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3804d040 con 0x55fce9b2f520 2024-09-21T06:45:03.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.507+0000 7f4b1ffff640 1 -- 172.21.15.67:0/927459785 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f4b380307c0 con 0x55fce9b2f520 2024-09-21T06:45:03.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.507+0000 7f4b4d30d740 1 -- 172.21.15.67:0/927459785 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b38040070 con 0x55fce9b2f520 2024-09-21T06:45:03.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.507+0000 7f4b1ffff640 1 -- 172.21.15.67:0/927459785 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4b38065050 con 0x55fce9b2f520 2024-09-21T06:45:03.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.507+0000 7f4b4d30d740 10 client.15822 fetch_fsmap learned FSMap version 21 2024-09-21T06:45:03.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.507+0000 7f4b4d30d740 1 -- 172.21.15.67:0/927459785 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55fce9b9ab00 con 0x55fce9b2f520 2024-09-21T06:45:03.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.507+0000 7f4b1ffff640 1 -- 172.21.15.67:0/927459785 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b38040070 con 0x55fce9b2f520 2024-09-21T06:45:03.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.507+0000 7f4b1ffff640 1 -- 172.21.15.67:0/927459785 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f4b38051660 con 0x55fce9b2f520 2024-09-21T06:45:03.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.507+0000 7f4b4d30d740 10 client.15822 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:45:03.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.507+0000 7f4b4d30d740 10 client.15822 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:45:03.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.507+0000 7f4b4d30d740 20 client.15822 populate_metadata read hostname 'smithi067' 2024-09-21T06:45:03.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.507+0000 7f4b4d30d740 1 --2- 172.21.15.67:0/927459785 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55fce9aedea0 0x55fce9ef70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:45:03.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.507+0000 7f4b4d30d740 4 client.15822 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:45:03.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.507+0000 7f4b4d30d740 1 -- 172.21.15.67:0/927459785 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fce9e4f140 con 0x55fce9aedea0 2024-09-21T06:45:03.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.508+0000 7f4b44b35640 1 --2- 172.21.15.67:0/927459785 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55fce9aedea0 0x55fce9ef70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:45:03.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.508+0000 7f4b44b35640 1 --2- 172.21.15.67:0/927459785 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55fce9aedea0 0x55fce9ef70b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac6ed94011394a90 server_cookie=2fa43167cabf343b in_seq=0 out_seq=0 2024-09-21T06:45:03.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.508+0000 7f4b1ffff640 10 client.15822 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:45:03.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.509+0000 7f4b1ffff640 1 -- 172.21.15.67:0/927459785 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fce9e4f140 con 0x55fce9aedea0 2024-09-21T06:45:03.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.509+0000 7f4b1ffff640 10 client.15822 handle_command_reply: tid=0 2024-09-21T06:45:03.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.509+0000 7f4b4d30d740 1 client.15822 shutdown 2024-09-21T06:45:03.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.509+0000 7f4b4d30d740 1 -- 172.21.15.67:0/927459785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fce9b2f520 msgr2=0x55fce9ee59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:45:03.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.509+0000 7f4b4d30d740 1 --2- 172.21.15.67:0/927459785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fce9b2f520 0x55fce9ee59d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4b3802a900 tx=0x7f4b38004590 comp rx=0 tx=0).stop 2024-09-21T06:45:03.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.509+0000 7f4b4d30d740 1 -- 172.21.15.67:0/927459785 shutdown_connections 2024-09-21T06:45:03.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.509+0000 7f4b4d30d740 1 --2- 172.21.15.67:0/927459785 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55fce9aedea0 0x55fce9ef70b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.509+0000 7f4b1ffff640 0 client.15822 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:45:03.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.509+0000 7f4b4d30d740 1 --2- 172.21.15.67:0/927459785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fce9b2f520 0x55fce9ee59d0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.509+0000 7f4b4d30d740 1 --2- 172.21.15.67:0/927459785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fce9b0e840 0x55fce9ee5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.509+0000 7f4b4d30d740 1 --2- 172.21.15.67:0/927459785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce9b06660 0x55fce9ee4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.509+0000 7f4b4d30d740 1 -- 172.21.15.67:0/927459785 >> 172.21.15.67:0/927459785 conn(0x55fce9b23910 msgr2=0x55fce9ed86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:45:03.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.509+0000 7f4b4d30d740 1 -- 172.21.15.67:0/927459785 shutdown_connections 2024-09-21T06:45:03.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.509+0000 7f4b4d30d740 1 -- 172.21.15.67:0/927459785 wait complete. 2024-09-21T06:45:03.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.509+0000 7f4b4d30d740 20 client.15822 trim_cache size 0 max 16384 2024-09-21T06:45:03.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.527+0000 7f4b4d30d740 1 Processor -- start 2024-09-21T06:45:03.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.527+0000 7f4b4d30d740 1 -- start start 2024-09-21T06:45:03.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.527+0000 7f4b4d30d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fce9aedea0 0x55fce9ee02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:45:03.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.527+0000 7f4b4d30d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fce9b06660 0x55fce9ee5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:45:03.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.527+0000 7f4b4d30d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce9b0e840 0x55fce9ed6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:45:03.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.527+0000 7f4b4d30d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4b401ac900 con 0x55fce9b06660 2024-09-21T06:45:03.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.527+0000 7f4b4d30d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4b401af180 con 0x55fce9b0e840 2024-09-21T06:45:03.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.527+0000 7f4b4d30d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4b401acf70 con 0x55fce9aedea0 2024-09-21T06:45:03.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.527+0000 7f4b3ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fce9b06660 0x55fce9ee5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:45:03.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.527+0000 7f4b3ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fce9b06660 0x55fce9ee5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39728/0 (socket says 172.21.15.67:39728) 2024-09-21T06:45:03.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.527+0000 7f4b3ffff640 1 -- 172.21.15.67:0/3006047510 learned_addr learned my addr 172.21.15.67:0/3006047510 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:45:03.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.527+0000 7f4b45336640 1 --2- 172.21.15.67:0/3006047510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce9b0e840 0x55fce9ed6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:45:03.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.527+0000 7f4b44b35640 1 --2- 172.21.15.67:0/3006047510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fce9aedea0 0x55fce9ee02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:45:03.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.528+0000 7f4b3ffff640 1 -- 172.21.15.67:0/3006047510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fce9aedea0 msgr2=0x55fce9ee02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:45:03.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.528+0000 7f4b3ffff640 1 --2- 172.21.15.67:0/3006047510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fce9aedea0 0x55fce9ee02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.528+0000 7f4b3ffff640 1 -- 172.21.15.67:0/3006047510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce9b0e840 msgr2=0x55fce9ed6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:45:03.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.528+0000 7f4b3ffff640 1 --2- 172.21.15.67:0/3006047510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce9b0e840 0x55fce9ed6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.528+0000 7f4b3ffff640 1 -- 172.21.15.67:0/3006047510 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fce9c11960 con 0x55fce9b06660 2024-09-21T06:45:03.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.528+0000 7f4b45336640 1 --2- 172.21.15.67:0/3006047510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce9b0e840 0x55fce9ed6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:45:03.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.528+0000 7f4b3ffff640 1 --2- 172.21.15.67:0/3006047510 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fce9b06660 0x55fce9ee5860 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c010bc0 tx=0x7f4b2c012450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8349e5c1c3f0c0a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:45:03.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.528+0000 7f4b2a7fc640 1 -- 172.21.15.67:0/3006047510 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c004220 con 0x55fce9b06660 2024-09-21T06:45:03.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.528+0000 7f4b2a7fc640 1 -- 172.21.15.67:0/3006047510 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b2c0043c0 con 0x55fce9b06660 2024-09-21T06:45:03.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.528+0000 7f4b2a7fc640 1 -- 172.21.15.67:0/3006047510 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c0035f0 con 0x55fce9b06660 2024-09-21T06:45:03.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.529+0000 7f4b4d30d740 1 -- 172.21.15.67:0/3006047510 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fce9b06660 msgr2=0x55fce9ee5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:45:03.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.529+0000 7f4b4d30d740 1 --2- 172.21.15.67:0/3006047510 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fce9b06660 0x55fce9ee5860 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c010bc0 tx=0x7f4b2c012450 comp rx=0 tx=0).stop 2024-09-21T06:45:03.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.529+0000 7f4b4d30d740 1 -- 172.21.15.67:0/3006047510 shutdown_connections 2024-09-21T06:45:03.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.529+0000 7f4b4d30d740 1 --2- 172.21.15.67:0/3006047510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce9b0e840 0x55fce9ed6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.529+0000 7f4b4d30d740 1 --2- 172.21.15.67:0/3006047510 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fce9b06660 0x55fce9ee5860 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.529+0000 7f4b4d30d740 1 --2- 172.21.15.67:0/3006047510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fce9aedea0 0x55fce9ee02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.529+0000 7f4b4d30d740 1 -- 172.21.15.67:0/3006047510 >> 172.21.15.67:0/3006047510 conn(0x55fce9b23910 msgr2=0x55fce9ed6540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:45:03.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.529+0000 7f4b4d30d740 1 -- 172.21.15.67:0/3006047510 shutdown_connections 2024-09-21T06:45:03.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.529+0000 7f4b4d30d740 1 -- 172.21.15.67:0/3006047510 wait complete. 2024-09-21T06:45:03.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.529+0000 7f4b4d30d740 1 -- start start 2024-09-21T06:45:03.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.530+0000 7f4b4d30d740 1 Processor -- start 2024-09-21T06:45:03.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.530+0000 7f4b4d30d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce9aedea0 0x55fce9ef84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:45:03.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.530+0000 7f4b4d30d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fce9b06660 0x55fce9ef89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:45:03.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.530+0000 7f4b4d30d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fce9b0e840 0x55fce9f60830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:45:03.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.530+0000 7f4b4d30d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4b401ad640 con 0x55fce9b0e840 2024-09-21T06:45:03.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.530+0000 7f4b4d30d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4b401aeab0 con 0x55fce9aedea0 2024-09-21T06:45:03.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.530+0000 7f4b4d30d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4b401add10 con 0x55fce9b06660 2024-09-21T06:45:03.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.530+0000 7f4b4d30d740 10 client.0 _ll_register_callbacks cb 0x7f4b3804b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:45:03.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.530+0000 7f4b45336640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fce9b0e840 0x55fce9f60830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:45:03.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.530+0000 7f4b45336640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fce9b0e840 0x55fce9f60830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39738/0 (socket says 172.21.15.67:39738) 2024-09-21T06:45:03.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.530+0000 7f4b45336640 1 -- 172.21.15.67:0/946871866 learned_addr learned my addr 172.21.15.67:0/946871866 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:45:03.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.530+0000 7f4b44b35640 1 --2- 172.21.15.67:0/946871866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce9aedea0 0x55fce9ef84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:45:03.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.530+0000 7f4b3ffff640 1 --2- 172.21.15.67:0/946871866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fce9b06660 0x55fce9ef89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:45:03.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.531+0000 7f4b45336640 1 -- 172.21.15.67:0/946871866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fce9b06660 msgr2=0x55fce9ef89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:45:03.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.531+0000 7f4b45336640 1 --2- 172.21.15.67:0/946871866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fce9b06660 0x55fce9ef89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.531+0000 7f4b45336640 1 -- 172.21.15.67:0/946871866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce9aedea0 msgr2=0x55fce9ef84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:45:03.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.531+0000 7f4b45336640 1 --2- 172.21.15.67:0/946871866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce9aedea0 0x55fce9ef84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.531+0000 7f4b45336640 1 -- 172.21.15.67:0/946871866 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fce9f60d70 con 0x55fce9b0e840 2024-09-21T06:45:03.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.531+0000 7f4b44b35640 1 --2- 172.21.15.67:0/946871866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce9aedea0 0x55fce9ef84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:45:03.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.531+0000 7f4b45336640 1 --2- 172.21.15.67:0/946871866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fce9b0e840 0x55fce9f60830 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4b38051c20 tx=0x7f4b380048b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f847ff129d056 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:45:03.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.531+0000 7f4b1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:45:03.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.531+0000 7f4b1ffff640 1 -- 172.21.15.67:0/946871866 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3806e060 con 0x55fce9b0e840 2024-09-21T06:45:03.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.531+0000 7f4b1ffff640 1 -- 172.21.15.67:0/946871866 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f4b380305c0 con 0x55fce9b0e840 2024-09-21T06:45:03.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.531+0000 7f4b4d30d740 1 -- 172.21.15.67:0/946871866 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b2c0043c0 con 0x55fce9b0e840 2024-09-21T06:45:03.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.531+0000 7f4b1ffff640 1 -- 172.21.15.67:0/946871866 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4b38075050 con 0x55fce9b0e840 2024-09-21T06:45:03.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.531+0000 7f4b4d30d740 10 client.15828 fetch_fsmap learned FSMap version 21 2024-09-21T06:45:03.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.531+0000 7f4b4d30d740 1 -- 172.21.15.67:0/946871866 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55fce9f61ed0 con 0x55fce9b0e840 2024-09-21T06:45:03.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.531+0000 7f4b1ffff640 1 -- 172.21.15.67:0/946871866 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b2c0043c0 con 0x55fce9b0e840 2024-09-21T06:45:03.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.532+0000 7f4b1ffff640 1 -- 172.21.15.67:0/946871866 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f4b38069430 con 0x55fce9b0e840 2024-09-21T06:45:03.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.532+0000 7f4b4d30d740 10 client.15828 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:45:03.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.532+0000 7f4b4d30d740 10 client.15828 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:45:03.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.532+0000 7f4b4d30d740 20 client.15828 populate_metadata read hostname 'smithi067' 2024-09-21T06:45:03.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.532+0000 7f4b4d30d740 1 --2- 172.21.15.67:0/946871866 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55fce9b2f520 0x55fce9f64540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:45:03.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.532+0000 7f4b4d30d740 4 client.15828 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "70ca9b66", "release": true, "await": true}] 2024-09-21T06:45:03.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.532+0000 7f4b4d30d740 1 -- 172.21.15.67:0/946871866 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "quiesce db", "set_id": "70ca9b66", "release": true, "await": true}) -- 0x55fce9e4abf0 con 0x55fce9b2f520 2024-09-21T06:45:03.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.532+0000 7f4b44b35640 1 --2- 172.21.15.67:0/946871866 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55fce9b2f520 0x55fce9f64540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:45:03.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.532+0000 7f4b44b35640 1 --2- 172.21.15.67:0/946871866 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55fce9b2f520 0x55fce9f64540 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83946cdd7c9f230a server_cookie=c96ef01d7ef3e023 in_seq=0 out_seq=0 2024-09-21T06:45:03.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.532+0000 7f4b1ffff640 10 client.15828 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:45:03.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.538+0000 7f4b1ffff640 1 -- 172.21.15.67:0/946871866 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55fce9e4abf0 con 0x55fce9b2f520 2024-09-21T06:45:03.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.538+0000 7f4b1ffff640 10 client.15828 handle_command_reply: tid=0 2024-09-21T06:45:03.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.538+0000 7f4b4d30d740 1 client.15828 shutdown 2024-09-21T06:45:03.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.538+0000 7f4b4d30d740 1 -- 172.21.15.67:0/946871866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fce9b0e840 msgr2=0x55fce9f60830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:45:03.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.538+0000 7f4b4d30d740 1 --2- 172.21.15.67:0/946871866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fce9b0e840 0x55fce9f60830 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4b38051c20 tx=0x7f4b380048b0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.538+0000 7f4b4d30d740 1 -- 172.21.15.67:0/946871866 shutdown_connections 2024-09-21T06:45:03.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.538+0000 7f4b4d30d740 1 --2- 172.21.15.67:0/946871866 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55fce9b2f520 0x55fce9f64540 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.538+0000 7f4b4d30d740 1 --2- 172.21.15.67:0/946871866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fce9b0e840 0x55fce9f60830 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.538+0000 7f4b4d30d740 1 --2- 172.21.15.67:0/946871866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fce9b06660 0x55fce9ef89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.538+0000 7f4b4d30d740 1 --2- 172.21.15.67:0/946871866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce9aedea0 0x55fce9ef84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.538+0000 7f4b4d30d740 1 -- 172.21.15.67:0/946871866 >> 172.21.15.67:0/946871866 conn(0x55fce9b23910 msgr2=0x55fce9ed9060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:45:03.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.538+0000 7f4b1ffff640 0 client.15828 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:45:03.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.538+0000 7f4b4d30d740 1 -- 172.21.15.67:0/946871866 shutdown_connections 2024-09-21T06:45:03.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.538+0000 7f4b4d30d740 1 -- 172.21.15.67:0/946871866 wait complete. 2024-09-21T06:45:03.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.539+0000 7f4b4d30d740 20 client.15828 trim_cache size 0 max 16384 2024-09-21T06:45:03.541 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-21T06:45:03.541 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 21, 2024-09-21T06:45:03.541 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-09-21T06:45:03.541 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 36, 2024-09-21T06:45:03.542 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-21T06:45:03.542 INFO:teuthology.orchestra.run.smithi067.stdout: "70ca9b66": { 2024-09-21T06:45:03.542 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 36, 2024-09-21T06:45:03.542 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-21T06:45:03.542 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:45:03.542 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-21T06:45:03.542 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:45:03.542 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:45:03.542 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-21T06:45:03.542 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 141.1, 2024-09-21T06:45:03.542 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-21T06:45:03.542 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-21T06:45:03.542 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-21T06:45:03.543 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:45:03.543 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-21T06:45:03.543 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:45:03.543 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:45:03.543 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:45:03.543 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:45:03.543 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:45:03.543 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:45:03.543 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-21T06:45:03.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.539+0000 7f4b46d99640 1 -- 172.21.15.67:0/3865973431 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f4b300b65c0 msgr2=0x7f4b300b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:45:03.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.539+0000 7f4b46d99640 1 --2- 172.21.15.67:0/3865973431 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f4b300b65c0 0x7f4b300b8ad0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4b3804a000 tx=0x7f4b3804b000 comp rx=0 tx=0).stop 2024-09-21T06:45:03.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.539+0000 7f4b46d99640 1 -- 172.21.15.67:0/3865973431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b40102a80 msgr2=0x7f4b4019a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:45:03.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.539+0000 7f4b46d99640 1 --2- 172.21.15.67:0/3865973431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b40102a80 0x7f4b4019a3a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4b3400c920 tx=0x7f4b3400cde0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.540+0000 7f4b46d99640 1 -- 172.21.15.67:0/3865973431 shutdown_connections 2024-09-21T06:45:03.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.540+0000 7f4b46d99640 1 --2- 172.21.15.67:0/3865973431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b401083c0 0x7f4b4019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.540+0000 7f4b46d99640 1 --2- 172.21.15.67:0/3865973431 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f4b300b65c0 0x7f4b300b8ad0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.540+0000 7f4b46d99640 1 --2- 172.21.15.67:0/3865973431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b40108d10 0x7f4b4019fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.540+0000 7f4b46d99640 1 --2- 172.21.15.67:0/3865973431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b40102a80 0x7f4b4019a3a0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:03.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.540+0000 7f4b46d99640 1 -- 172.21.15.67:0/3865973431 >> 172.21.15.67:0/3865973431 conn(0x7f4b40079740 msgr2=0x7f4b40101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:45:03.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.540+0000 7f4b46d99640 1 -- 172.21.15.67:0/3865973431 shutdown_connections 2024-09-21T06:45:03.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:45:03.540+0000 7f4b46d99640 1 -- 172.21.15.67:0/3865973431 wait complete. 2024-09-21T06:45:03.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:03 smithi100 ceph-mon[33401]: pgmap v614: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 662 KiB/s wr, 12 op/s 2024-09-21T06:45:04.388 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '70ca9b66', total seconds elapsed: 24.6 2024-09-21T06:45:05.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:05 smithi203 ceph-mon[33735]: pgmap v615: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 161 KiB/s wr, 4 op/s 2024-09-21T06:45:05.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:05 smithi067 ceph-mon[31232]: pgmap v615: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 161 KiB/s wr, 4 op/s 2024-09-21T06:45:05.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:05 smithi100 ceph-mon[33401]: pgmap v615: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 161 KiB/s wr, 4 op/s 2024-09-21T06:45:07.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:07 smithi203 ceph-mon[33735]: pgmap v616: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.1 MiB/s wr, 24 op/s 2024-09-21T06:45:07.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:07 smithi067 ceph-mon[31232]: pgmap v616: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.1 MiB/s wr, 24 op/s 2024-09-21T06:45:07.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:07 smithi100 ceph-mon[33401]: pgmap v616: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.1 MiB/s wr, 24 op/s 2024-09-21T06:45:09.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:09 smithi203 ceph-mon[33735]: pgmap v617: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.1 MiB/s wr, 162 op/s 2024-09-21T06:45:09.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:09 smithi067 ceph-mon[31232]: pgmap v617: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.1 MiB/s wr, 162 op/s 2024-09-21T06:45:09.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:09 smithi100 ceph-mon[33401]: pgmap v617: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.1 MiB/s wr, 162 op/s 2024-09-21T06:45:11.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:11 smithi203 ceph-mon[33735]: pgmap v618: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 10 MiB/s wr, 203 op/s 2024-09-21T06:45:11.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:11 smithi067 ceph-mon[31232]: pgmap v618: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 10 MiB/s wr, 203 op/s 2024-09-21T06:45:11.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:11 smithi100 ceph-mon[33401]: pgmap v618: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 10 MiB/s wr, 203 op/s 2024-09-21T06:45:13.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:13 smithi203 ceph-mon[33735]: pgmap v619: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 21 MiB/s wr, 428 op/s 2024-09-21T06:45:13.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:13 smithi067 ceph-mon[31232]: pgmap v619: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 21 MiB/s wr, 428 op/s 2024-09-21T06:45:13.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:13 smithi100 ceph-mon[33401]: pgmap v619: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 21 MiB/s wr, 428 op/s 2024-09-21T06:45:15.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:15 smithi203 ceph-mon[33735]: pgmap v620: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 25 MiB/s wr, 504 op/s 2024-09-21T06:45:15.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:15 smithi067 ceph-mon[31232]: pgmap v620: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 25 MiB/s wr, 504 op/s 2024-09-21T06:45:15.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:15 smithi100 ceph-mon[33401]: pgmap v620: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 25 MiB/s wr, 504 op/s 2024-09-21T06:45:17.481 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:17 smithi067 ceph-mon[31232]: pgmap v621: 129 pgs: 129 active+clean; 10 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 28 MiB/s wr, 564 op/s 2024-09-21T06:45:17.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:17 smithi203 ceph-mon[33735]: pgmap v621: 129 pgs: 129 active+clean; 10 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 28 MiB/s wr, 564 op/s 2024-09-21T06:45:17.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:17 smithi100 ceph-mon[33401]: pgmap v621: 129 pgs: 129 active+clean; 10 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 28 MiB/s wr, 564 op/s 2024-09-21T06:45:19.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:19 smithi203 ceph-mon[33735]: pgmap v622: 129 pgs: 129 active+clean; 10 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 38 MiB/s wr, 755 op/s 2024-09-21T06:45:19.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:19 smithi067 ceph-mon[31232]: pgmap v622: 129 pgs: 129 active+clean; 10 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 38 MiB/s wr, 755 op/s 2024-09-21T06:45:19.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:19 smithi100 ceph-mon[33401]: pgmap v622: 129 pgs: 129 active+clean; 10 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 38 MiB/s wr, 755 op/s 2024-09-21T06:45:21.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:21 smithi203 ceph-mon[33735]: pgmap v623: 129 pgs: 129 active+clean; 10 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 33 MiB/s wr, 659 op/s 2024-09-21T06:45:21.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:21 smithi067 ceph-mon[31232]: pgmap v623: 129 pgs: 129 active+clean; 10 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 33 MiB/s wr, 659 op/s 2024-09-21T06:45:21.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:21 smithi100 ceph-mon[33401]: pgmap v623: 129 pgs: 129 active+clean; 10 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 33 MiB/s wr, 659 op/s 2024-09-21T06:45:23.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:23 smithi203 ceph-mon[33735]: pgmap v624: 129 pgs: 129 active+clean; 10 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 41 MiB/s wr, 825 op/s 2024-09-21T06:45:23.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:23 smithi067 ceph-mon[31232]: pgmap v624: 129 pgs: 129 active+clean; 10 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 41 MiB/s wr, 825 op/s 2024-09-21T06:45:23.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:23 smithi100 ceph-mon[33401]: pgmap v624: 129 pgs: 129 active+clean; 10 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 41 MiB/s wr, 825 op/s 2024-09-21T06:45:25.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:25 smithi067 ceph-mon[31232]: pgmap v625: 129 pgs: 129 active+clean; 10 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 34 MiB/s wr, 677 op/s 2024-09-21T06:45:25.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:25 smithi100 ceph-mon[33401]: pgmap v625: 129 pgs: 129 active+clean; 10 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 34 MiB/s wr, 677 op/s 2024-09-21T06:45:25.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:25 smithi203 ceph-mon[33735]: pgmap v625: 129 pgs: 129 active+clean; 10 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 34 MiB/s wr, 677 op/s 2024-09-21T06:45:27.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:27 smithi067 ceph-mon[31232]: pgmap v626: 129 pgs: 129 active+clean; 10 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 33 MiB/s wr, 662 op/s 2024-09-21T06:45:27.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:27 smithi100 ceph-mon[33401]: pgmap v626: 129 pgs: 129 active+clean; 10 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 33 MiB/s wr, 662 op/s 2024-09-21T06:45:27.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:27 smithi203 ceph-mon[33735]: pgmap v626: 129 pgs: 129 active+clean; 10 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 33 MiB/s wr, 662 op/s 2024-09-21T06:45:29.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:29 smithi067 ceph-mon[31232]: pgmap v627: 129 pgs: 129 active+clean; 10 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 41 MiB/s wr, 815 op/s 2024-09-21T06:45:29.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:29 smithi100 ceph-mon[33401]: pgmap v627: 129 pgs: 129 active+clean; 10 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 41 MiB/s wr, 815 op/s 2024-09-21T06:45:29.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:29 smithi203 ceph-mon[33735]: pgmap v627: 129 pgs: 129 active+clean; 10 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 41 MiB/s wr, 815 op/s 2024-09-21T06:45:31.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:31 smithi067 ceph-mon[31232]: pgmap v628: 129 pgs: 129 active+clean; 10 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 32 MiB/s wr, 642 op/s 2024-09-21T06:45:31.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:31 smithi100 ceph-mon[33401]: pgmap v628: 129 pgs: 129 active+clean; 10 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 32 MiB/s wr, 642 op/s 2024-09-21T06:45:31.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:31 smithi203 ceph-mon[33735]: pgmap v628: 129 pgs: 129 active+clean; 10 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 32 MiB/s wr, 642 op/s 2024-09-21T06:45:33.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:33 smithi067 ceph-mon[31232]: pgmap v629: 129 pgs: 129 active+clean; 10 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 40 MiB/s wr, 816 op/s 2024-09-21T06:45:33.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:33 smithi100 ceph-mon[33401]: pgmap v629: 129 pgs: 129 active+clean; 10 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 40 MiB/s wr, 816 op/s 2024-09-21T06:45:33.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:33 smithi203 ceph-mon[33735]: pgmap v629: 129 pgs: 129 active+clean; 10 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 40 MiB/s wr, 816 op/s 2024-09-21T06:45:35.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:35 smithi067 ceph-mon[31232]: pgmap v630: 129 pgs: 129 active+clean; 10 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 34 MiB/s wr, 687 op/s 2024-09-21T06:45:35.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:35 smithi100 ceph-mon[33401]: pgmap v630: 129 pgs: 129 active+clean; 10 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 34 MiB/s wr, 687 op/s 2024-09-21T06:45:35.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:35 smithi203 ceph-mon[33735]: pgmap v630: 129 pgs: 129 active+clean; 10 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 34 MiB/s wr, 687 op/s 2024-09-21T06:45:37.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:37 smithi067 ceph-mon[31232]: pgmap v631: 129 pgs: 129 active+clean; 10 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 33 MiB/s wr, 671 op/s 2024-09-21T06:45:37.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:37 smithi100 ceph-mon[33401]: pgmap v631: 129 pgs: 129 active+clean; 10 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 33 MiB/s wr, 671 op/s 2024-09-21T06:45:37.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:37 smithi203 ceph-mon[33735]: pgmap v631: 129 pgs: 129 active+clean; 10 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 33 MiB/s wr, 671 op/s 2024-09-21T06:45:39.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:39 smithi100 ceph-mon[33401]: pgmap v632: 129 pgs: 129 active+clean; 10 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 40 MiB/s wr, 825 op/s 2024-09-21T06:45:39.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:39 smithi203 ceph-mon[33735]: pgmap v632: 129 pgs: 129 active+clean; 10 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 40 MiB/s wr, 825 op/s 2024-09-21T06:45:39.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:39 smithi067 ceph-mon[31232]: pgmap v632: 129 pgs: 129 active+clean; 10 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 40 MiB/s wr, 825 op/s 2024-09-21T06:45:41.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:41 smithi100 ceph-mon[33401]: pgmap v633: 129 pgs: 129 active+clean; 10 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 32 MiB/s wr, 654 op/s 2024-09-21T06:45:41.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:41 smithi203 ceph-mon[33735]: pgmap v633: 129 pgs: 129 active+clean; 10 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 32 MiB/s wr, 654 op/s 2024-09-21T06:45:41.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:41 smithi067 ceph-mon[31232]: pgmap v633: 129 pgs: 129 active+clean; 10 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 32 MiB/s wr, 654 op/s 2024-09-21T06:45:43.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:43 smithi100 ceph-mon[33401]: pgmap v634: 129 pgs: 129 active+clean; 10 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 40 MiB/s wr, 831 op/s 2024-09-21T06:45:43.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:43 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:45:43.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:43 smithi203 ceph-mon[33735]: pgmap v634: 129 pgs: 129 active+clean; 10 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 40 MiB/s wr, 831 op/s 2024-09-21T06:45:43.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:43 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:45:43.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:43 smithi067 ceph-mon[31232]: pgmap v634: 129 pgs: 129 active+clean; 10 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 40 MiB/s wr, 831 op/s 2024-09-21T06:45:43.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:43 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:45:44.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:44 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:45:44.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:44 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:45:44.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:44 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:45:44.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:44 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:45:44.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:44 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:45:44.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:44 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:45:44.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:44 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:45:44.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:44 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:45:44.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:44 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:45:45.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:45 smithi203 ceph-mon[33735]: pgmap v635: 129 pgs: 129 active+clean; 10 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 33 MiB/s wr, 687 op/s 2024-09-21T06:45:46.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:45 smithi067 ceph-mon[31232]: pgmap v635: 129 pgs: 129 active+clean; 10 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 33 MiB/s wr, 687 op/s 2024-09-21T06:45:46.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:45 smithi100 ceph-mon[33401]: pgmap v635: 129 pgs: 129 active+clean; 10 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 33 MiB/s wr, 687 op/s 2024-09-21T06:45:47.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:47 smithi203 ceph-mon[33735]: pgmap v636: 129 pgs: 129 active+clean; 10 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 32 MiB/s wr, 669 op/s 2024-09-21T06:45:48.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:47 smithi067 ceph-mon[31232]: pgmap v636: 129 pgs: 129 active+clean; 10 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 32 MiB/s wr, 669 op/s 2024-09-21T06:45:48.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:47 smithi100 ceph-mon[33401]: pgmap v636: 129 pgs: 129 active+clean; 10 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 32 MiB/s wr, 669 op/s 2024-09-21T06:45:49.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:49 smithi203 ceph-mon[33735]: pgmap v637: 129 pgs: 129 active+clean; 10 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 39 MiB/s wr, 812 op/s 2024-09-21T06:45:50.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:49 smithi067 ceph-mon[31232]: pgmap v637: 129 pgs: 129 active+clean; 10 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 39 MiB/s wr, 812 op/s 2024-09-21T06:45:50.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:49 smithi100 ceph-mon[33401]: pgmap v637: 129 pgs: 129 active+clean; 10 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 39 MiB/s wr, 812 op/s 2024-09-21T06:45:51.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:51 smithi203 ceph-mon[33735]: pgmap v638: 129 pgs: 129 active+clean; 10 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 31 MiB/s wr, 634 op/s 2024-09-21T06:45:52.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:51 smithi067 ceph-mon[31232]: pgmap v638: 129 pgs: 129 active+clean; 10 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 31 MiB/s wr, 634 op/s 2024-09-21T06:45:52.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:51 smithi100 ceph-mon[33401]: pgmap v638: 129 pgs: 129 active+clean; 10 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 31 MiB/s wr, 634 op/s 2024-09-21T06:45:53.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:53 smithi203 ceph-mon[33735]: pgmap v639: 129 pgs: 129 active+clean; 10 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 39 MiB/s wr, 810 op/s 2024-09-21T06:45:54.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:53 smithi067 ceph-mon[31232]: pgmap v639: 129 pgs: 129 active+clean; 10 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 39 MiB/s wr, 810 op/s 2024-09-21T06:45:54.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:53 smithi100 ceph-mon[33401]: pgmap v639: 129 pgs: 129 active+clean; 10 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 39 MiB/s wr, 810 op/s 2024-09-21T06:45:55.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:55 smithi203 ceph-mon[33735]: pgmap v640: 129 pgs: 129 active+clean; 10 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 32 MiB/s wr, 675 op/s 2024-09-21T06:45:56.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:55 smithi067 ceph-mon[31232]: pgmap v640: 129 pgs: 129 active+clean; 10 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 32 MiB/s wr, 675 op/s 2024-09-21T06:45:56.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:55 smithi100 ceph-mon[33401]: pgmap v640: 129 pgs: 129 active+clean; 10 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 32 MiB/s wr, 675 op/s 2024-09-21T06:45:57.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:57 smithi203 ceph-mon[33735]: pgmap v641: 129 pgs: 129 active+clean; 10 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 32 MiB/s wr, 671 op/s 2024-09-21T06:45:58.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:57 smithi067 ceph-mon[31232]: pgmap v641: 129 pgs: 129 active+clean; 10 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 32 MiB/s wr, 671 op/s 2024-09-21T06:45:58.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:57 smithi100 ceph-mon[33401]: pgmap v641: 129 pgs: 129 active+clean; 10 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 32 MiB/s wr, 671 op/s 2024-09-21T06:45:59.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:45:59 smithi203 ceph-mon[33735]: pgmap v642: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 39 MiB/s wr, 830 op/s 2024-09-21T06:46:00.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:45:59 smithi067 ceph-mon[31232]: pgmap v642: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 39 MiB/s wr, 830 op/s 2024-09-21T06:46:00.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:45:59 smithi100 ceph-mon[33401]: pgmap v642: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 39 MiB/s wr, 830 op/s 2024-09-21T06:46:01.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:01 smithi203 ceph-mon[33735]: pgmap v643: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 31 MiB/s wr, 668 op/s 2024-09-21T06:46:02.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:01 smithi067 ceph-mon[31232]: pgmap v643: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 31 MiB/s wr, 668 op/s 2024-09-21T06:46:02.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:01 smithi100 ceph-mon[33401]: pgmap v643: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 31 MiB/s wr, 668 op/s 2024-09-21T06:46:03.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:03 smithi203 ceph-mon[33735]: pgmap v644: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 39 MiB/s wr, 850 op/s 2024-09-21T06:46:04.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:03 smithi067 ceph-mon[31232]: pgmap v644: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 39 MiB/s wr, 850 op/s 2024-09-21T06:46:04.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:03 smithi100 ceph-mon[33401]: pgmap v644: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 39 MiB/s wr, 850 op/s 2024-09-21T06:46:05.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:05 smithi203 ceph-mon[33735]: pgmap v645: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 33 MiB/s wr, 713 op/s 2024-09-21T06:46:06.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:05 smithi067 ceph-mon[31232]: pgmap v645: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 33 MiB/s wr, 713 op/s 2024-09-21T06:46:06.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:05 smithi100 ceph-mon[33401]: pgmap v645: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 33 MiB/s wr, 713 op/s 2024-09-21T06:46:07.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:07 smithi203 ceph-mon[33735]: pgmap v646: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 31 MiB/s wr, 689 op/s 2024-09-21T06:46:08.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:07 smithi067 ceph-mon[31232]: pgmap v646: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 31 MiB/s wr, 689 op/s 2024-09-21T06:46:08.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:07 smithi100 ceph-mon[33401]: pgmap v646: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 31 MiB/s wr, 689 op/s 2024-09-21T06:46:10.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:09 smithi067 ceph-mon[31232]: pgmap v647: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 38 MiB/s wr, 836 op/s 2024-09-21T06:46:10.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:09 smithi100 ceph-mon[33401]: pgmap v647: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 38 MiB/s wr, 836 op/s 2024-09-21T06:46:10.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:09 smithi203 ceph-mon[33735]: pgmap v647: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 38 MiB/s wr, 836 op/s 2024-09-21T06:46:12.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:11 smithi067 ceph-mon[31232]: pgmap v648: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 30 MiB/s wr, 655 op/s 2024-09-21T06:46:12.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:11 smithi100 ceph-mon[33401]: pgmap v648: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 30 MiB/s wr, 655 op/s 2024-09-21T06:46:12.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:11 smithi203 ceph-mon[33735]: pgmap v648: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 30 MiB/s wr, 655 op/s 2024-09-21T06:46:14.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:13 smithi067 ceph-mon[31232]: pgmap v649: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 38 MiB/s wr, 841 op/s 2024-09-21T06:46:14.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:13 smithi100 ceph-mon[33401]: pgmap v649: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 38 MiB/s wr, 841 op/s 2024-09-21T06:46:14.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:13 smithi203 ceph-mon[33735]: pgmap v649: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 38 MiB/s wr, 841 op/s 2024-09-21T06:46:16.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:15 smithi067 ceph-mon[31232]: pgmap v650: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 31 MiB/s wr, 697 op/s 2024-09-21T06:46:16.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:15 smithi100 ceph-mon[33401]: pgmap v650: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 31 MiB/s wr, 697 op/s 2024-09-21T06:46:16.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:15 smithi203 ceph-mon[33735]: pgmap v650: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 31 MiB/s wr, 697 op/s 2024-09-21T06:46:18.033 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:17 smithi203 ceph-mon[33735]: pgmap v651: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 30 MiB/s wr, 681 op/s 2024-09-21T06:46:18.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:17 smithi067 ceph-mon[31232]: pgmap v651: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 30 MiB/s wr, 681 op/s 2024-09-21T06:46:18.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:17 smithi100 ceph-mon[33401]: pgmap v651: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 30 MiB/s wr, 681 op/s 2024-09-21T06:46:20.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:19 smithi067 ceph-mon[31232]: pgmap v652: 129 pgs: 129 active+clean; 10 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 37 MiB/s wr, 840 op/s 2024-09-21T06:46:20.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:19 smithi100 ceph-mon[33401]: pgmap v652: 129 pgs: 129 active+clean; 10 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 37 MiB/s wr, 840 op/s 2024-09-21T06:46:20.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:19 smithi203 ceph-mon[33735]: pgmap v652: 129 pgs: 129 active+clean; 10 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 37 MiB/s wr, 840 op/s 2024-09-21T06:46:21.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:20 smithi203 ceph-mon[33735]: pgmap v653: 129 pgs: 129 active+clean; 10 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 29 MiB/s wr, 666 op/s 2024-09-21T06:46:21.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:20 smithi067 ceph-mon[31232]: pgmap v653: 129 pgs: 129 active+clean; 10 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 29 MiB/s wr, 666 op/s 2024-09-21T06:46:21.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:20 smithi100 ceph-mon[33401]: pgmap v653: 129 pgs: 129 active+clean; 10 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 29 MiB/s wr, 666 op/s 2024-09-21T06:46:22.574 INFO:tasks.quiescer.fs.[cephfs]:Total duration (49.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-21T06:46:22.574 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.5 2024-09-21T06:46:22.575 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:46:23.170 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:46:23.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:23 smithi203 ceph-mon[33735]: pgmap v654: 129 pgs: 129 active+clean; 10 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 37 MiB/s wr, 847 op/s 2024-09-21T06:46:23.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:23 smithi067 ceph-mon[31232]: pgmap v654: 129 pgs: 129 active+clean; 10 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 37 MiB/s wr, 847 op/s 2024-09-21T06:46:23.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:23 smithi100 ceph-mon[33401]: pgmap v654: 129 pgs: 129 active+clean; 10 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 37 MiB/s wr, 847 op/s 2024-09-21T06:46:25.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:25 smithi203 ceph-mon[33735]: pgmap v655: 129 pgs: 129 active+clean; 10 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 31 MiB/s wr, 700 op/s 2024-09-21T06:46:25.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:25 smithi067 ceph-mon[31232]: pgmap v655: 129 pgs: 129 active+clean; 10 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 31 MiB/s wr, 700 op/s 2024-09-21T06:46:25.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:25 smithi100 ceph-mon[33401]: pgmap v655: 129 pgs: 129 active+clean; 10 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 31 MiB/s wr, 700 op/s 2024-09-21T06:46:26.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.260+0000 7f583b144640 1 -- 172.21.15.67:0/3528230999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f583410c8c0 msgr2=0x7f583410ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:26.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.260+0000 7f583b144640 1 --2- 172.21.15.67:0/3528230999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f583410c8c0 0x7f583410ccd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5828009930 tx=0x7f582802a480 comp rx=0 tx=0).stop 2024-09-21T06:46:26.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.261+0000 7f583b144640 1 -- 172.21.15.67:0/3528230999 shutdown_connections 2024-09-21T06:46:26.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.261+0000 7f583b144640 1 --2- 172.21.15.67:0/3528230999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f583410d210 0x7f5834116ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:26.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.261+0000 7f583b144640 1 --2- 172.21.15.67:0/3528230999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f583410c8c0 0x7f583410ccd0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:26.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.261+0000 7f583b144640 1 --2- 172.21.15.67:0/3528230999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f583410a490 0x7f583410a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:26.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.261+0000 7f583b144640 1 -- 172.21.15.67:0/3528230999 >> 172.21.15.67:0/3528230999 conn(0x7f58340796d0 msgr2=0x7f583407baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:46:26.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.261+0000 7f583b144640 1 -- 172.21.15.67:0/3528230999 shutdown_connections 2024-09-21T06:46:26.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.261+0000 7f583b144640 1 -- 172.21.15.67:0/3528230999 wait complete. 2024-09-21T06:46:26.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.262+0000 7f583b144640 1 Processor -- start 2024-09-21T06:46:26.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.262+0000 7f583b144640 1 -- start start 2024-09-21T06:46:26.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.262+0000 7f583b144640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f583410a490 0x7f58341a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:26.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.262+0000 7f583b144640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f583410c8c0 0x7f58341a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:26.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.262+0000 7f583b144640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f583410d210 0x7f58341a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:26.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.262+0000 7f583b144640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f583411a040 con 0x7f583410a490 2024-09-21T06:46:26.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.262+0000 7f583b144640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5834119ec0 con 0x7f583410d210 2024-09-21T06:46:26.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.262+0000 7f583b144640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f583411a1c0 con 0x7f583410c8c0 2024-09-21T06:46:26.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.263+0000 7f58396e1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f583410d210 0x7f58341a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:26.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.263+0000 7f5838ee0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f583410a490 0x7f58341a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:26.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.263+0000 7f5838ee0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f583410a490 0x7f58341a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42350/0 (socket says 172.21.15.67:42350) 2024-09-21T06:46:26.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.263+0000 7f5838ee0640 1 -- 172.21.15.67:0/3930897223 learned_addr learned my addr 172.21.15.67:0/3930897223 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:46:26.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.263+0000 7f5833fff640 1 --2- 172.21.15.67:0/3930897223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f583410c8c0 0x7f58341a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:26.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.263+0000 7f5833fff640 1 -- 172.21.15.67:0/3930897223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f583410d210 msgr2=0x7f58341a63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:26.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.263+0000 7f5833fff640 1 --2- 172.21.15.67:0/3930897223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f583410d210 0x7f58341a63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:26.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.263+0000 7f5833fff640 1 -- 172.21.15.67:0/3930897223 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f583410a490 msgr2=0x7f58341a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:26.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.263+0000 7f5833fff640 1 --2- 172.21.15.67:0/3930897223 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f583410a490 0x7f58341a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:26.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.263+0000 7f5833fff640 1 -- 172.21.15.67:0/3930897223 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58341a6ab0 con 0x7f583410c8c0 2024-09-21T06:46:26.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.264+0000 7f5833fff640 1 --2- 172.21.15.67:0/3930897223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f583410c8c0 0x7f58341a1110 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f583406c6e0 tx=0x7f5828003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:46:26.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.264+0000 7f5831ffb640 1 -- 172.21.15.67:0/3930897223 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582803f070 con 0x7f583410c8c0 2024-09-21T06:46:26.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.264+0000 7f5831ffb640 1 -- 172.21.15.67:0/3930897223 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f582803baf0 con 0x7f583410c8c0 2024-09-21T06:46:26.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.265+0000 7f5831ffb640 1 -- 172.21.15.67:0/3930897223 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5828035030 con 0x7f583410c8c0 2024-09-21T06:46:26.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.265+0000 7f583b144640 1 -- 172.21.15.67:0/3930897223 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58341a6d40 con 0x7f583410c8c0 2024-09-21T06:46:26.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.265+0000 7f583b144640 1 -- 172.21.15.67:0/3930897223 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58341a71f0 con 0x7f583410c8c0 2024-09-21T06:46:26.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.266+0000 7f583b144640 1 -- 172.21.15.67:0/3930897223 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f583410b460 con 0x7f583410c8c0 2024-09-21T06:46:26.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.272+0000 7f5831ffb640 1 -- 172.21.15.67:0/3930897223 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f5828042070 con 0x7f583410c8c0 2024-09-21T06:46:26.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.274+0000 7f5831ffb640 1 --2- 172.21.15.67:0/3930897223 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f581c0b64e0 0x7f581c0b89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:26.081 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:46:26.081 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:46:26.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.274+0000 7f5831ffb640 1 -- 172.21.15.67:0/3930897223 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f58281086c0 con 0x7f583410c8c0 2024-09-21T06:46:26.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.274+0000 7f5838ee0640 1 --2- 172.21.15.67:0/3930897223 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f581c0b64e0 0x7f581c0b89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:26.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.274+0000 7f5831ffb640 1 -- 172.21.15.67:0/3930897223 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5828049030 con 0x7f583410c8c0 2024-09-21T06:46:26.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.275+0000 7f5838ee0640 1 --2- 172.21.15.67:0/3930897223 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f581c0b64e0 0x7f581c0b89f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f582400bc40 tx=0x7f5824005f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:46:26.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.689+0000 7f583b144640 1 -- 172.21.15.67:0/3930897223 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58341a1ca0 con 0x7f583410c8c0 2024-09-21T06:46:26.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.690+0000 7f5831ffb640 1 -- 172.21.15.67:0/3930897223 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7f58280ceb20 con 0x7f583410c8c0 2024-09-21T06:46:26.082 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:46:26.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.692+0000 7f583b144640 1 -- 172.21.15.67:0/3930897223 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f581c0b64e0 msgr2=0x7f581c0b89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:26.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.692+0000 7f583b144640 1 --2- 172.21.15.67:0/3930897223 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f581c0b64e0 0x7f581c0b89f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f582400bc40 tx=0x7f5824005f30 comp rx=0 tx=0).stop 2024-09-21T06:46:26.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.692+0000 7f583b144640 1 -- 172.21.15.67:0/3930897223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f583410c8c0 msgr2=0x7f58341a1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:26.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.692+0000 7f583b144640 1 --2- 172.21.15.67:0/3930897223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f583410c8c0 0x7f58341a1110 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f583406c6e0 tx=0x7f5828003b90 comp rx=0 tx=0).stop 2024-09-21T06:46:26.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.692+0000 7f583b144640 1 -- 172.21.15.67:0/3930897223 shutdown_connections 2024-09-21T06:46:26.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.692+0000 7f583b144640 1 --2- 172.21.15.67:0/3930897223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f583410d210 0x7f58341a63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:26.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.692+0000 7f583b144640 1 --2- 172.21.15.67:0/3930897223 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f581c0b64e0 0x7f581c0b89f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:26.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.692+0000 7f583b144640 1 --2- 172.21.15.67:0/3930897223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f583410c8c0 0x7f58341a1110 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:26.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.692+0000 7f583b144640 1 --2- 172.21.15.67:0/3930897223 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f583410a490 0x7f58341a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:26.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.692+0000 7f583b144640 1 -- 172.21.15.67:0/3930897223 >> 172.21.15.67:0/3930897223 conn(0x7f58340796d0 msgr2=0x7f583410f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:46:26.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.692+0000 7f583b144640 1 -- 172.21.15.67:0/3930897223 shutdown_connections 2024-09-21T06:46:26.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:25.692+0000 7f583b144640 1 -- 172.21.15.67:0/3930897223 wait complete. 2024-09-21T06:46:26.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:26 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3930897223' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:46:26.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:26 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3930897223' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:46:26.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:26 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3930897223' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:46:27.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:27 smithi067 ceph-mon[31232]: pgmap v656: 129 pgs: 129 active+clean; 10 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 30 MiB/s wr, 683 op/s 2024-09-21T06:46:27.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:27 smithi203 ceph-mon[33735]: pgmap v656: 129 pgs: 129 active+clean; 10 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 30 MiB/s wr, 683 op/s 2024-09-21T06:46:27.529 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0' 2024-09-21T06:46:27.530 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24671 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0 2024-09-21T06:46:27.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:27 smithi100 ceph-mon[33401]: pgmap v656: 129 pgs: 129 active+clean; 10 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 30 MiB/s wr, 683 op/s 2024-09-21T06:46:28.214 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:46:28.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.959+0000 7f80b9a3f640 1 -- 172.21.15.67:0/3871731680 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80b4102a80 msgr2=0x7f80b4102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:28.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.959+0000 7f80b9a3f640 1 --2- 172.21.15.67:0/3871731680 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80b4102a80 0x7f80b4102e90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f80a800d910 tx=0x7f80a80329c0 comp rx=0 tx=0).stop 2024-09-21T06:46:28.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.961+0000 7f80b9a3f640 1 -- 172.21.15.67:0/3871731680 shutdown_connections 2024-09-21T06:46:28.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.961+0000 7f80b9a3f640 1 --2- 172.21.15.67:0/3871731680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80b41083c0 0x7f80b41087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:28.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.961+0000 7f80b9a3f640 1 --2- 172.21.15.67:0/3871731680 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80b4108d10 0x7f80b410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:28.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.961+0000 7f80b9a3f640 1 --2- 172.21.15.67:0/3871731680 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80b4102a80 0x7f80b4102e90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:28.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.961+0000 7f80b9a3f640 1 -- 172.21.15.67:0/3871731680 >> 172.21.15.67:0/3871731680 conn(0x7f80b4079760 msgr2=0x7f80b4079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:46:28.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.962+0000 7f80b9a3f640 1 -- 172.21.15.67:0/3871731680 shutdown_connections 2024-09-21T06:46:28.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.962+0000 7f80b9a3f640 1 -- 172.21.15.67:0/3871731680 wait complete. 2024-09-21T06:46:28.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.963+0000 7f80b9a3f640 1 Processor -- start 2024-09-21T06:46:28.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.963+0000 7f80b9a3f640 1 -- start start 2024-09-21T06:46:28.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.963+0000 7f80b9a3f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80b4102a80 0x7f80b4074a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:28.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.963+0000 7f80b9a3f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80b41083c0 0x7f80b4074f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:28.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.963+0000 7f80b9a3f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80b4108d10 0x7f80b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:28.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.964+0000 7f80b9a3f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f80b406d170 con 0x7f80b4108d10 2024-09-21T06:46:28.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.964+0000 7f80b9a3f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f80b406cff0 con 0x7f80b4102a80 2024-09-21T06:46:28.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.964+0000 7f80b9a3f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f80b406d2f0 con 0x7f80b41083c0 2024-09-21T06:46:28.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.968+0000 7f80b37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80b4108d10 0x7f80b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:28.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.968+0000 7f80b37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80b4108d10 0x7f80b406ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47010/0 (socket says 172.21.15.67:47010) 2024-09-21T06:46:28.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.968+0000 7f80b37fe640 1 -- 172.21.15.67:0/2592997913 learned_addr learned my addr 172.21.15.67:0/2592997913 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:46:28.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.968+0000 7f80b2ffd640 1 --2- 172.21.15.67:0/2592997913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80b4102a80 0x7f80b4074a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:28.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.968+0000 7f80b27fc640 1 --2- 172.21.15.67:0/2592997913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80b41083c0 0x7f80b4074f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:28.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.969+0000 7f80b2ffd640 1 -- 172.21.15.67:0/2592997913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80b41083c0 msgr2=0x7f80b4074f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:28.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.969+0000 7f80b2ffd640 1 --2- 172.21.15.67:0/2592997913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80b41083c0 0x7f80b4074f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:28.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.969+0000 7f80b2ffd640 1 -- 172.21.15.67:0/2592997913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80b4108d10 msgr2=0x7f80b406ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:28.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.969+0000 7f80b2ffd640 1 --2- 172.21.15.67:0/2592997913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80b4108d10 0x7f80b406ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:28.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.969+0000 7f80b2ffd640 1 -- 172.21.15.67:0/2592997913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b406f600 con 0x7f80b4102a80 2024-09-21T06:46:28.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.971+0000 7f80b2ffd640 1 --2- 172.21.15.67:0/2592997913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80b4102a80 0x7f80b4074a10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f80a8000c00 tx=0x7f80a80055f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:46:28.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.972+0000 7f80b8a3d640 1 -- 172.21.15.67:0/2592997913 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8044070 con 0x7f80b4102a80 2024-09-21T06:46:28.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.972+0000 7f80b8a3d640 1 -- 172.21.15.67:0/2592997913 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80a8004d30 con 0x7f80b4102a80 2024-09-21T06:46:28.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.972+0000 7f80b8a3d640 1 -- 172.21.15.67:0/2592997913 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8014070 con 0x7f80b4102a80 2024-09-21T06:46:28.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.972+0000 7f80b9a3f640 1 -- 172.21.15.67:0/2592997913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b406f890 con 0x7f80b4102a80 2024-09-21T06:46:28.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.972+0000 7f80b9a3f640 1 -- 172.21.15.67:0/2592997913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b41be220 con 0x7f80b4102a80 2024-09-21T06:46:28.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.979+0000 7f80b8a3d640 1 -- 172.21.15.67:0/2592997913 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f80a803e070 con 0x7f80b4102a80 2024-09-21T06:46:28.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.980+0000 7f80b8a3d640 1 --2- 172.21.15.67:0/2592997913 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f809c0b6710 0x7f809c0b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:28.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.980+0000 7f80b27fc640 1 --2- 172.21.15.67:0/2592997913 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f809c0b6710 0x7f809c0b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:28.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.980+0000 7f80b8a3d640 1 -- 172.21.15.67:0/2592997913 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f80a81061c0 con 0x7f80b4102a80 2024-09-21T06:46:28.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.981+0000 7f80b27fc640 1 --2- 172.21.15.67:0/2592997913 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f809c0b6710 0x7f809c0b8c20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f80b4100c10 tx=0x7f80ac00d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:46:28.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.982+0000 7f80bffb3740 1 Processor -- start 2024-09-21T06:46:28.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.982+0000 7f80bffb3740 1 -- start start 2024-09-21T06:46:28.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.983+0000 7f80bffb3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c644c5840 0x558c6486c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:28.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.983+0000 7f80bffb3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c6449c980 0x558c6486cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:28.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.983+0000 7f80bffb3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c644a4b60 0x558c6486d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:28.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.983+0000 7f80bffb3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x558c647e2b90 con 0x558c644c5840 2024-09-21T06:46:28.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.983+0000 7f80bffb3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558c647d1fc0 con 0x558c644a4b60 2024-09-21T06:46:28.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.983+0000 7f80bffb3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558c647d5dd0 con 0x558c6449c980 2024-09-21T06:46:28.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.984+0000 7f80b2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c644c5840 0x558c6486c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:28.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.984+0000 7f80b2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c644c5840 0x558c6486c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47026/0 (socket says 172.21.15.67:47026) 2024-09-21T06:46:28.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.984+0000 7f80b2ffd640 1 -- 172.21.15.67:0/2475670993 learned_addr learned my addr 172.21.15.67:0/2475670993 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:46:28.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.985+0000 7f80b37fe640 1 --2- 172.21.15.67:0/2475670993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c644a4b60 0x558c6486d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:28.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.986+0000 7f80b27fc640 1 --2- 172.21.15.67:0/2475670993 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c6449c980 0x558c6486cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:28.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.986+0000 7f80b2ffd640 1 -- 172.21.15.67:0/2475670993 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c6449c980 msgr2=0x558c6486cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:46:28.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.986+0000 7f80b2ffd640 1 --2- 172.21.15.67:0/2475670993 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c6449c980 0x558c6486cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:28.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.986+0000 7f80b2ffd640 1 -- 172.21.15.67:0/2475670993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c644a4b60 msgr2=0x558c6486d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:46:28.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.986+0000 7f80b2ffd640 1 --2- 172.21.15.67:0/2475670993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c644a4b60 0x558c6486d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:28.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.986+0000 7f80b2ffd640 1 -- 172.21.15.67:0/2475670993 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c64759060 con 0x558c644c5840 2024-09-21T06:46:28.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.987+0000 7f80b2ffd640 1 --2- 172.21.15.67:0/2475670993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c644c5840 0x558c6486c600 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f80a80cbf60 tx=0x7f80a80329c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1659009a810994bc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:46:28.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.996+0000 7f8099ffb640 1 -- 172.21.15.67:0/2475670993 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a800d520 con 0x558c644c5840 2024-09-21T06:46:28.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.996+0000 7f8099ffb640 1 -- 172.21.15.67:0/2475670993 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80a80d0a00 con 0x558c644c5840 2024-09-21T06:46:28.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.996+0000 7f8099ffb640 1 -- 172.21.15.67:0/2475670993 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a80b71c0 con 0x558c644c5840 2024-09-21T06:46:28.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.997+0000 7f80bffb3740 1 -- 172.21.15.67:0/2475670993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c644c5840 msgr2=0x558c6486c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:28.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:28.997+0000 7f80bffb3740 1 --2- 172.21.15.67:0/2475670993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c644c5840 0x558c6486c600 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f80a80cbf60 tx=0x7f80a80329c0 comp rx=0 tx=0).stop 2024-09-21T06:46:29.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.001+0000 7f80bffb3740 1 -- 172.21.15.67:0/2475670993 shutdown_connections 2024-09-21T06:46:29.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.001+0000 7f80bffb3740 1 --2- 172.21.15.67:0/2475670993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c644a4b60 0x558c6486d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:29.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.001+0000 7f80bffb3740 1 --2- 172.21.15.67:0/2475670993 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c6449c980 0x558c6486cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:29.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.001+0000 7f80bffb3740 1 --2- 172.21.15.67:0/2475670993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c644c5840 0x558c6486c600 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:29.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.001+0000 7f80bffb3740 1 -- 172.21.15.67:0/2475670993 >> 172.21.15.67:0/2475670993 conn(0x558c644b9990 msgr2=0x558c6486a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:46:29.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.005+0000 7f80bffb3740 1 -- 172.21.15.67:0/2475670993 shutdown_connections 2024-09-21T06:46:29.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.005+0000 7f80bffb3740 1 -- 172.21.15.67:0/2475670993 wait complete. 2024-09-21T06:46:29.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.006+0000 7f80bffb3740 1 -- start start 2024-09-21T06:46:29.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.006+0000 7f80bffb3740 1 Processor -- start 2024-09-21T06:46:29.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.007+0000 7f80bffb3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c6449c980 0x558c64878210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:29.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.007+0000 7f80bffb3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c644a4b60 0x558c64878770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:29.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.007+0000 7f80bffb3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c644c5840 0x558c64883d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:29.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.007+0000 7f80bffb3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x558c646337a0 con 0x558c6449c980 2024-09-21T06:46:29.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.007+0000 7f80bffb3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558c647770a0 con 0x558c644a4b60 2024-09-21T06:46:29.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.007+0000 7f80bffb3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558c647e2b90 con 0x558c644c5840 2024-09-21T06:46:29.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.007+0000 7f80bffb3740 10 client.0 _ll_register_callbacks cb 0x558c647543c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:46:29.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.007+0000 7f80b37fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c644c5840 0x558c64883d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:29.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.008+0000 7f80b37fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c644c5840 0x558c64883d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.67:58054/0 (socket says 172.21.15.67:58054) 2024-09-21T06:46:29.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.008+0000 7f80b37fe640 1 -- 172.21.15.67:0/6451697 learned_addr learned my addr 172.21.15.67:0/6451697 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:46:29.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.008+0000 7f80b37fe640 1 -- 172.21.15.67:0/6451697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c644a4b60 msgr2=0x558c64878770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:46:29.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.008+0000 7f80b37fe640 1 --2- 172.21.15.67:0/6451697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c644a4b60 0x558c64878770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:29.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.008+0000 7f80b37fe640 1 -- 172.21.15.67:0/6451697 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c6449c980 msgr2=0x558c64878210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:46:29.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.008+0000 7f80b37fe640 1 --2- 172.21.15.67:0/6451697 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c6449c980 0x558c64878210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:29.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.008+0000 7f80b37fe640 1 -- 172.21.15.67:0/6451697 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c64765880 con 0x558c644c5840 2024-09-21T06:46:29.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.008+0000 7f80b37fe640 1 --2- 172.21.15.67:0/6451697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c644c5840 0x558c64883d40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f80a400c800 tx=0x7f80a400ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18a8293cadb67d94 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:46:29.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.009+0000 7f8092ffd640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T06:46:29.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.009+0000 7f8092ffd640 1 -- 172.21.15.67:0/6451697 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a400ba30 con 0x558c644c5840 2024-09-21T06:46:29.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.009+0000 7f80bffb3740 1 -- 172.21.15.67:0/6451697 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80a80d0a00 con 0x558c644c5840 2024-09-21T06:46:29.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.009+0000 7f8092ffd640 1 -- 172.21.15.67:0/6451697 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f80a400bbd0 con 0x558c644c5840 2024-09-21T06:46:29.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.010+0000 7f8092ffd640 1 -- 172.21.15.67:0/6451697 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f80a4027050 con 0x558c644c5840 2024-09-21T06:46:29.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.013+0000 7f80bffb3740 10 client.25495 fetch_fsmap learned FSMap version 21 2024-09-21T06:46:29.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.013+0000 7f80bffb3740 1 -- 172.21.15.67:0/6451697 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558c6453fb90 con 0x558c644c5840 2024-09-21T06:46:29.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.013+0000 7f8092ffd640 1 -- 172.21.15.67:0/6451697 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80a401e070 con 0x558c644c5840 2024-09-21T06:46:29.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.013+0000 7f8092ffd640 1 -- 172.21.15.67:0/6451697 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f80a40183e0 con 0x558c644c5840 2024-09-21T06:46:29.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.017+0000 7f80bffb3740 10 client.25495 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:46:29.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.017+0000 7f80bffb3740 10 client.25495 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:46:29.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.017+0000 7f80bffb3740 20 client.25495 populate_metadata read hostname 'smithi067' 2024-09-21T06:46:29.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.017+0000 7f80bffb3740 1 --2- 172.21.15.67:0/6451697 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x558c644841c0 0x558c6488a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:29.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.018+0000 7f80bffb3740 4 client.25495 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:46:29.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.018+0000 7f80bffb3740 1 -- 172.21.15.67:0/6451697 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f80a80d0a00 con 0x558c644841c0 2024-09-21T06:46:29.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.018+0000 7f80b2ffd640 1 --2- 172.21.15.67:0/6451697 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x558c644841c0 0x558c6488a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:29.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.025+0000 7f8092ffd640 10 client.25495 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:46:29.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.025+0000 7f80b2ffd640 1 --2- 172.21.15.67:0/6451697 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x558c644841c0 0x558c6488a000 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd89793ce2cab65d server_cookie=393e7bf6057f7520 in_seq=0 out_seq=0 2024-09-21T06:46:29.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.027+0000 7f8092ffd640 1 -- 172.21.15.67:0/6451697 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f80a80d0a00 con 0x558c644841c0 2024-09-21T06:46:29.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.027+0000 7f8092ffd640 10 client.25495 handle_command_reply: tid=0 2024-09-21T06:46:29.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.027+0000 7f80bffb3740 1 client.25495 shutdown 2024-09-21T06:46:29.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.031+0000 7f80bffb3740 1 -- 172.21.15.67:0/6451697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c644c5840 msgr2=0x558c64883d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:29.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.031+0000 7f80bffb3740 1 --2- 172.21.15.67:0/6451697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c644c5840 0x558c64883d40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f80a400c800 tx=0x7f80a400ccc0 comp rx=0 tx=0).stop 2024-09-21T06:46:29.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.031+0000 7f80bffb3740 1 -- 172.21.15.67:0/6451697 shutdown_connections 2024-09-21T06:46:29.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.031+0000 7f80bffb3740 1 --2- 172.21.15.67:0/6451697 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x558c644841c0 0x558c6488a000 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:29.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.031+0000 7f80bffb3740 1 --2- 172.21.15.67:0/6451697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c644a4b60 0x558c64878770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:29.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.031+0000 7f80bffb3740 1 --2- 172.21.15.67:0/6451697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c644c5840 0x558c64883d40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:29.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.031+0000 7f80bffb3740 1 --2- 172.21.15.67:0/6451697 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c6449c980 0x558c64878210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:29.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.031+0000 7f80bffb3740 1 -- 172.21.15.67:0/6451697 >> 172.21.15.67:0/6451697 conn(0x558c644b9990 msgr2=0x558c6486f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:46:29.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.032+0000 7f8092ffd640 0 client.25495 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:46:29.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.032+0000 7f80bffb3740 1 -- 172.21.15.67:0/6451697 shutdown_connections 2024-09-21T06:46:29.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.032+0000 7f80bffb3740 1 -- 172.21.15.67:0/6451697 wait complete. 2024-09-21T06:46:29.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.032+0000 7f80bffb3740 20 client.25495 trim_cache size 0 max 16384 2024-09-21T06:46:29.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.066+0000 7f80bffb3740 1 Processor -- start 2024-09-21T06:46:29.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.066+0000 7f80bffb3740 1 -- start start 2024-09-21T06:46:29.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.067+0000 7f80bffb3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c6449c980 0x558c64879ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:29.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.067+0000 7f80bffb3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c644a4b60 0x558c6486dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:29.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.067+0000 7f80bffb3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c644c5840 0x558c64875200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:29.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.068+0000 7f80b27fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c644a4b60 0x558c6486dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:29.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.068+0000 7f80b27fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c644a4b60 0x558c6486dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.67:58068/0 (socket says 172.21.15.67:58068) 2024-09-21T06:46:29.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.068+0000 7f80b2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c6449c980 0x558c64879ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:29.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.068+0000 7f80b2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c6449c980 0x558c64879ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.67:50246/0 (socket says 172.21.15.67:50246) 2024-09-21T06:46:29.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.068+0000 7f80b37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c644c5840 0x558c64875200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:29.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.069+0000 7f80b37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c644c5840 0x558c64875200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47052/0 (socket says 172.21.15.67:47052) 2024-09-21T06:46:29.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.069+0000 7f80b27fc640 1 -- 172.21.15.67:0/1792685869 learned_addr learned my addr 172.21.15.67:0/1792685869 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:46:29.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.069+0000 7f80bffb3740 1 -- 172.21.15.67:0/1792685869 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f80b41aede0 con 0x558c644c5840 2024-09-21T06:46:29.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.069+0000 7f80bffb3740 1 -- 172.21.15.67:0/1792685869 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f80b41b1660 con 0x558c6449c980 2024-09-21T06:46:29.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.069+0000 7f80bffb3740 1 -- 172.21.15.67:0/1792685869 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f80b41af450 con 0x558c644a4b60 2024-09-21T06:46:29.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.069+0000 7f80b2ffd640 1 -- 172.21.15.67:0/1792685869 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c644a4b60 msgr2=0x558c6486dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:46:29.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.069+0000 7f80b2ffd640 1 --2- 172.21.15.67:0/1792685869 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c644a4b60 0x558c6486dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:29.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.069+0000 7f80b2ffd640 1 -- 172.21.15.67:0/1792685869 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c644c5840 msgr2=0x558c64875200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:46:29.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.069+0000 7f80b2ffd640 1 --2- 172.21.15.67:0/1792685869 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c644c5840 0x558c64875200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:29.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.069+0000 7f80b2ffd640 1 -- 172.21.15.67:0/1792685869 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c64865ad0 con 0x558c6449c980 2024-09-21T06:46:29.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.069+0000 7f80b37fe640 1 --2- 172.21.15.67:0/1792685869 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c644c5840 0x558c64875200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:46:29.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.069+0000 7f80b2ffd640 1 --2- 172.21.15.67:0/1792685869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c6449c980 0x558c64879ef0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f80a80cbf60 tx=0x7f80a80cb920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e18682eb060984c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:46:29.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.070+0000 7f80997fa640 1 -- 172.21.15.67:0/1792685869 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a80e4020 con 0x558c6449c980 2024-09-21T06:46:29.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.070+0000 7f80997fa640 1 -- 172.21.15.67:0/1792685869 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80a800d520 con 0x558c6449c980 2024-09-21T06:46:29.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.070+0000 7f80997fa640 1 -- 172.21.15.67:0/1792685869 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a80b71c0 con 0x558c6449c980 2024-09-21T06:46:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.071+0000 7f80bffb3740 1 -- 172.21.15.67:0/1792685869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c6449c980 msgr2=0x558c64879ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.071+0000 7f80bffb3740 1 --2- 172.21.15.67:0/1792685869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c6449c980 0x558c64879ef0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f80a80cbf60 tx=0x7f80a80cb920 comp rx=0 tx=0).stop 2024-09-21T06:46:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.071+0000 7f80bffb3740 1 -- 172.21.15.67:0/1792685869 shutdown_connections 2024-09-21T06:46:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.071+0000 7f80bffb3740 1 --2- 172.21.15.67:0/1792685869 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c644c5840 0x558c64875200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.071+0000 7f80bffb3740 1 --2- 172.21.15.67:0/1792685869 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c644a4b60 0x558c6486dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.071+0000 7f80bffb3740 1 --2- 172.21.15.67:0/1792685869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c6449c980 0x558c64879ef0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.071+0000 7f80bffb3740 1 -- 172.21.15.67:0/1792685869 >> 172.21.15.67:0/1792685869 conn(0x558c644b9990 msgr2=0x558c6486d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:46:29.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.072+0000 7f80bffb3740 1 -- 172.21.15.67:0/1792685869 shutdown_connections 2024-09-21T06:46:29.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.075+0000 7f80bffb3740 1 -- 172.21.15.67:0/1792685869 wait complete. 2024-09-21T06:46:29.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.075+0000 7f80bffb3740 1 -- start start 2024-09-21T06:46:29.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.076+0000 7f80bffb3740 1 Processor -- start 2024-09-21T06:46:29.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.077+0000 7f80bffb3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c6449c980 0x558c6487d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:29.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.077+0000 7f80bffb3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c644a4b60 0x558c6487d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:29.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.077+0000 7f80b2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c6449c980 0x558c6487d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:29.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.077+0000 7f80b2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c6449c980 0x558c6487d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.67:50252/0 (socket says 172.21.15.67:50252) 2024-09-21T06:46:29.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.077+0000 7f80b2ffd640 1 -- 172.21.15.67:0/228913588 learned_addr learned my addr 172.21.15.67:0/228913588 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:46:29.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.077+0000 7f80bffb3740 1 --2- 172.21.15.67:0/228913588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c644c5840 0x558c648f49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:29.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.078+0000 7f80bffb3740 1 -- 172.21.15.67:0/228913588 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f80b41afb20 con 0x558c644a4b60 2024-09-21T06:46:29.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.078+0000 7f80b37fe640 1 --2- 172.21.15.67:0/228913588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c644c5840 0x558c648f49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:29.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.078+0000 7f80bffb3740 1 -- 172.21.15.67:0/228913588 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f80b41b0f90 con 0x558c6449c980 2024-09-21T06:46:29.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.078+0000 7f80bffb3740 1 -- 172.21.15.67:0/228913588 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f80b41b01f0 con 0x558c644c5840 2024-09-21T06:46:29.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.078+0000 7f80b37fe640 1 -- 172.21.15.67:0/228913588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c6449c980 msgr2=0x558c6487d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:46:29.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.078+0000 7f80b37fe640 1 --2- 172.21.15.67:0/228913588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c6449c980 0x558c6487d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:29.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.078+0000 7f80b37fe640 1 -- 172.21.15.67:0/228913588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c644a4b60 msgr2=0x558c6487d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:46:29.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.078+0000 7f80b37fe640 1 --2- 172.21.15.67:0/228913588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c644a4b60 0x558c6487d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:29.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.078+0000 7f80b37fe640 1 -- 172.21.15.67:0/228913588 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c648f4ee0 con 0x558c644c5840 2024-09-21T06:46:29.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.079+0000 7f80b2ffd640 1 --2- 172.21.15.67:0/228913588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c6449c980 0x558c6487d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:46:29.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.079+0000 7f80b37fe640 1 --2- 172.21.15.67:0/228913588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c644c5840 0x558c648f49a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f80a40187b0 tx=0x7f80a4018610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d55b216f4e7f173 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:46:29.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.079+0000 7f8092ffd640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T06:46:29.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.080+0000 7f8092ffd640 1 -- 172.21.15.67:0/228913588 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a4030060 con 0x558c644c5840 2024-09-21T06:46:29.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.080+0000 7f8092ffd640 1 -- 172.21.15.67:0/228913588 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f80a400bc30 con 0x558c644c5840 2024-09-21T06:46:29.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.079+0000 7f80bffb3740 10 client.0 _ll_register_callbacks cb 0x7f80a400d420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:46:29.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.080+0000 7f80bffb3740 1 -- 172.21.15.67:0/228913588 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80a800d520 con 0x558c644c5840 2024-09-21T06:46:29.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.081+0000 7f8092ffd640 1 -- 172.21.15.67:0/228913588 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f80a400be50 con 0x558c644c5840 2024-09-21T06:46:29.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.081+0000 7f80bffb3740 10 client.25501 fetch_fsmap learned FSMap version 21 2024-09-21T06:46:29.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.081+0000 7f80bffb3740 1 -- 172.21.15.67:0/228913588 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({fsmap=21}) -- 0x558c648f5e30 con 0x558c644c5840 2024-09-21T06:46:29.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.082+0000 7f8092ffd640 1 -- 172.21.15.67:0/228913588 <== mon.1 v2:172.21.15.203:3300/0 4 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f80a4019480 con 0x558c644c5840 2024-09-21T06:46:29.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.082+0000 7f80bffb3740 10 client.25501 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:46:29.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.082+0000 7f80bffb3740 10 client.25501 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:46:29.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.082+0000 7f80bffb3740 20 client.25501 populate_metadata read hostname 'smithi067' 2024-09-21T06:46:29.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.082+0000 7f80bffb3740 1 --2- 172.21.15.67:0/228913588 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x558c64459570 0x558c648f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:29.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.082+0000 7f80bffb3740 4 client.25501 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}] 2024-09-21T06:46:29.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.083+0000 7f80bffb3740 1 -- 172.21.15.67:0/228913588 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}) -- 0x7f80a800d520 con 0x558c64459570 2024-09-21T06:46:29.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.083+0000 7f80b2ffd640 1 --2- 172.21.15.67:0/228913588 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x558c64459570 0x558c648f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:29.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.083+0000 7f80b2ffd640 1 --2- 172.21.15.67:0/228913588 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x558c64459570 0x558c648f84d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc9766a07d4a9a88 server_cookie=aa1309f5679c3d1c in_seq=0 out_seq=0 2024-09-21T06:46:29.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.083+0000 7f8092ffd640 10 client.25501 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:46:29.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.119+0000 7f8092ffd640 1 -- 172.21.15.67:0/228913588 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f80a800d520 con 0x558c64459570 2024-09-21T06:46:29.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.119+0000 7f8092ffd640 10 client.25501 handle_command_reply: tid=0 2024-09-21T06:46:29.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.119+0000 7f80bffb3740 1 client.25501 shutdown 2024-09-21T06:46:29.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.119+0000 7f80bffb3740 1 -- 172.21.15.67:0/228913588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c644c5840 msgr2=0x558c648f49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:29.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.119+0000 7f80bffb3740 1 --2- 172.21.15.67:0/228913588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c644c5840 0x558c648f49a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f80a40187b0 tx=0x7f80a4018610 comp rx=0 tx=0).stop 2024-09-21T06:46:29.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.120+0000 7f80bffb3740 1 -- 172.21.15.67:0/228913588 shutdown_connections 2024-09-21T06:46:29.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.120+0000 7f80bffb3740 1 --2- 172.21.15.67:0/228913588 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x558c64459570 0x558c648f84d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:29.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.120+0000 7f80bffb3740 1 --2- 172.21.15.67:0/228913588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c644c5840 0x558c648f49a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:29.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.120+0000 7f80bffb3740 1 --2- 172.21.15.67:0/228913588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c644a4b60 0x558c6487d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:29.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.120+0000 7f80bffb3740 1 --2- 172.21.15.67:0/228913588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c6449c980 0x558c6487d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:29.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.120+0000 7f80bffb3740 1 -- 172.21.15.67:0/228913588 >> 172.21.15.67:0/228913588 conn(0x558c644b9990 msgr2=0x558c6486ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:46:29.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.120+0000 7f8092ffd640 0 client.25501 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:46:29.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.120+0000 7f80bffb3740 1 -- 172.21.15.67:0/228913588 shutdown_connections 2024-09-21T06:46:29.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.120+0000 7f80bffb3740 1 -- 172.21.15.67:0/228913588 wait complete. 2024-09-21T06:46:29.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.120+0000 7f80bffb3740 20 client.25501 trim_cache size 0 max 16384 2024-09-21T06:46:29.123 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-21T06:46:29.123 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 21, 2024-09-21T06:46:29.123 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-09-21T06:46:29.124 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 38, 2024-09-21T06:46:29.124 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-21T06:46:29.124 INFO:teuthology.orchestra.run.smithi067.stdout: "f868a83c": { 2024-09-21T06:46:29.124 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 38, 2024-09-21T06:46:29.124 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-21T06:46:29.124 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:46:29.124 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-21T06:46:29.124 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:46:29.124 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:46:29.124 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-21T06:46:29.124 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 144.5, 2024-09-21T06:46:29.124 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-21T06:46:29.124 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-21T06:46:29.124 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-21T06:46:29.124 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:46:29.125 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-21T06:46:29.125 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:46:29.125 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:46:29.125 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:46:29.125 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:46:29.125 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:46:29.125 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:46:29.125 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-21T06:46:29.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.121+0000 7f80b9a3f640 1 -- 172.21.15.67:0/2592997913 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f809c0b6710 msgr2=0x7f809c0b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:29.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.121+0000 7f80b9a3f640 1 --2- 172.21.15.67:0/2592997913 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f809c0b6710 0x7f809c0b8c20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f80b4100c10 tx=0x7f80ac00d040 comp rx=0 tx=0).stop 2024-09-21T06:46:29.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.121+0000 7f80b9a3f640 1 -- 172.21.15.67:0/2592997913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80b4102a80 msgr2=0x7f80b4074a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:29.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.121+0000 7f80b9a3f640 1 --2- 172.21.15.67:0/2592997913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80b4102a80 0x7f80b4074a10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f80a8000c00 tx=0x7f80a80055f0 comp rx=0 tx=0).stop 2024-09-21T06:46:29.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.121+0000 7f80b9a3f640 1 -- 172.21.15.67:0/2592997913 shutdown_connections 2024-09-21T06:46:29.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.121+0000 7f80b9a3f640 1 --2- 172.21.15.67:0/2592997913 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f809c0b6710 0x7f809c0b8c20 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:29.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.121+0000 7f80b9a3f640 1 --2- 172.21.15.67:0/2592997913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80b4108d10 0x7f80b406ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:29.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.121+0000 7f80b9a3f640 1 --2- 172.21.15.67:0/2592997913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80b41083c0 0x7f80b4074f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:29.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.121+0000 7f80b9a3f640 1 --2- 172.21.15.67:0/2592997913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80b4102a80 0x7f80b4074a10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:29.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.121+0000 7f80b9a3f640 1 -- 172.21.15.67:0/2592997913 >> 172.21.15.67:0/2592997913 conn(0x7f80b4079760 msgr2=0x7f80b4101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:46:29.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.122+0000 7f80b9a3f640 1 -- 172.21.15.67:0/2592997913 shutdown_connections 2024-09-21T06:46:29.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:29.122+0000 7f80b9a3f640 1 -- 172.21.15.67:0/2592997913 wait complete. 2024-09-21T06:46:29.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:29 smithi203 ceph-mon[33735]: pgmap v657: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 35 MiB/s wr, 803 op/s 2024-09-21T06:46:29.541 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:29 smithi067 ceph-mon[31232]: pgmap v657: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 35 MiB/s wr, 803 op/s 2024-09-21T06:46:29.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:29 smithi100 ceph-mon[33401]: pgmap v657: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 35 MiB/s wr, 803 op/s 2024-09-21T06:46:31.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:31 smithi203 ceph-mon[33735]: pgmap v658: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 27 MiB/s wr, 617 op/s 2024-09-21T06:46:31.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:31 smithi067 ceph-mon[31232]: pgmap v658: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 27 MiB/s wr, 617 op/s 2024-09-21T06:46:31.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:31 smithi100 ceph-mon[33401]: pgmap v658: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 27 MiB/s wr, 617 op/s 2024-09-21T06:46:32.045 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f868a83c', quiesce took 9.5 seconds. Will release after: 15.0 2024-09-21T06:46:33.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:33 smithi203 ceph-mon[33735]: pgmap v659: 129 pgs: 129 active+clean; 10 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 30 MiB/s wr, 679 op/s 2024-09-21T06:46:33.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:33 smithi067 ceph-mon[31232]: pgmap v659: 129 pgs: 129 active+clean; 10 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 30 MiB/s wr, 679 op/s 2024-09-21T06:46:33.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:33 smithi100 ceph-mon[33401]: pgmap v659: 129 pgs: 129 active+clean; 10 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 30 MiB/s wr, 679 op/s 2024-09-21T06:46:35.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:35 smithi203 ceph-mon[33735]: pgmap v660: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 21 MiB/s wr, 473 op/s 2024-09-21T06:46:35.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:35 smithi067 ceph-mon[31232]: pgmap v660: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 21 MiB/s wr, 473 op/s 2024-09-21T06:46:35.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:35 smithi100 ceph-mon[33401]: pgmap v660: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 21 MiB/s wr, 473 op/s 2024-09-21T06:46:37.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:37 smithi203 ceph-mon[33735]: pgmap v661: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 17 MiB/s wr, 395 op/s 2024-09-21T06:46:37.541 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:37 smithi067 ceph-mon[31232]: pgmap v661: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 17 MiB/s wr, 395 op/s 2024-09-21T06:46:37.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:37 smithi100 ceph-mon[33401]: pgmap v661: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 17 MiB/s wr, 395 op/s 2024-09-21T06:46:39.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:39 smithi203 ceph-mon[33735]: pgmap v662: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 15 MiB/s wr, 344 op/s 2024-09-21T06:46:39.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:39 smithi067 ceph-mon[31232]: pgmap v662: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 15 MiB/s wr, 344 op/s 2024-09-21T06:46:39.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:39 smithi100 ceph-mon[33401]: pgmap v662: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 15 MiB/s wr, 344 op/s 2024-09-21T06:46:41.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:41 smithi203 ceph-mon[33735]: pgmap v663: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.3 MiB/s wr, 160 op/s 2024-09-21T06:46:41.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:41 smithi067 ceph-mon[31232]: pgmap v663: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.3 MiB/s wr, 160 op/s 2024-09-21T06:46:41.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:41 smithi100 ceph-mon[33401]: pgmap v663: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.3 MiB/s wr, 160 op/s 2024-09-21T06:46:43.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:43 smithi203 ceph-mon[33735]: pgmap v664: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.8 MiB/s wr, 129 op/s 2024-09-21T06:46:43.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:43 smithi067 ceph-mon[31232]: pgmap v664: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.8 MiB/s wr, 129 op/s 2024-09-21T06:46:43.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:43 smithi100 ceph-mon[33401]: pgmap v664: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.8 MiB/s wr, 129 op/s 2024-09-21T06:46:44.517 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:44 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:46:44.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:44 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:46:44.655 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:44 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:46:45.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:45 smithi067 ceph-mon[31232]: pgmap v665: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1.3 MiB/s wr, 26 op/s 2024-09-21T06:46:45.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:45 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:46:45.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:45 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:46:45.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:45 smithi100 ceph-mon[33401]: pgmap v665: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1.3 MiB/s wr, 26 op/s 2024-09-21T06:46:45.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:45 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:46:45.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:45 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:46:45.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:45 smithi203 ceph-mon[33735]: pgmap v665: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1.3 MiB/s wr, 26 op/s 2024-09-21T06:46:45.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:45 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:46:45.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:45 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:46:46.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:46 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:46:46.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:46 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:46:46.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:46 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:46:46.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:46 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:46:46.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:46 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:46.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:46 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:46:46.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:46 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:46:46.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:46 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:46:46.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:46 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:46:46.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:46 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:46:46.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:46 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:46:46.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:46 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:46.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:46 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:46:46.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:46 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:46:46.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:46 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:46:46.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:46 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:46:46.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:46 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:46:46.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:46 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:46:46.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:46 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:46.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:46 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:46:46.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:46 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:46:47.045 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:46:47.320 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:46:47.440 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:47 smithi067 ceph-mon[31232]: pgmap v666: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 683 KiB/s wr, 13 op/s 2024-09-21T06:46:47.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:47 smithi100 ceph-mon[33401]: pgmap v666: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 683 KiB/s wr, 13 op/s 2024-09-21T06:46:47.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:47 smithi203 ceph-mon[33735]: pgmap v666: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 683 KiB/s wr, 13 op/s 2024-09-21T06:46:47.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.816+0000 7f904a140640 1 -- 172.21.15.67:0/3290530728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90441094a0 msgr2=0x7f90441098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:47.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.816+0000 7f904a140640 1 --2- 172.21.15.67:0/3290530728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90441094a0 0x7f90441098b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9038009900 tx=0x7f903802a410 comp rx=0 tx=0).stop 2024-09-21T06:46:47.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.816+0000 7f904a140640 1 -- 172.21.15.67:0/3290530728 shutdown_connections 2024-09-21T06:46:47.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.816+0000 7f904a140640 1 --2- 172.21.15.67:0/3290530728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9044109df0 0x7f9044110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:47.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.816+0000 7f904a140640 1 --2- 172.21.15.67:0/3290530728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90441094a0 0x7f90441098b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:47.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.816+0000 7f904a140640 1 --2- 172.21.15.67:0/3290530728 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9044103b60 0x7f9044103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:47.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.816+0000 7f904a140640 1 -- 172.21.15.67:0/3290530728 >> 172.21.15.67:0/3290530728 conn(0x7f904407b850 msgr2=0x7f904407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:46:47.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.816+0000 7f904a140640 1 -- 172.21.15.67:0/3290530728 shutdown_connections 2024-09-21T06:46:47.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.816+0000 7f904a140640 1 -- 172.21.15.67:0/3290530728 wait complete. 2024-09-21T06:46:47.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.817+0000 7f904a140640 1 Processor -- start 2024-09-21T06:46:47.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.817+0000 7f904a140640 1 -- start start 2024-09-21T06:46:47.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.817+0000 7f904a140640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9044103b60 0x7f9044075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:47.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.817+0000 7f904a140640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90441094a0 0x7f9044075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:47.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.817+0000 7f904a140640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9044109df0 0x7f9044076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:47.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.817+0000 7f904a140640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9044113960 con 0x7f90441094a0 2024-09-21T06:46:47.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.817+0000 7f904a140640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f90441137e0 con 0x7f9044109df0 2024-09-21T06:46:47.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.817+0000 7f904a140640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9044113ae0 con 0x7f9044103b60 2024-09-21T06:46:47.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.817+0000 7f9042ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90441094a0 0x7f9044075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.817+0000 7f9042ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90441094a0 0x7f9044075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51196/0 (socket says 172.21.15.67:51196) 2024-09-21T06:46:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.817+0000 7f9042ffd640 1 -- 172.21.15.67:0/1886037543 learned_addr learned my addr 172.21.15.67:0/1886037543 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:46:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.817+0000 7f9043fff640 1 --2- 172.21.15.67:0/1886037543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9044109df0 0x7f9044076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.817+0000 7f90437fe640 1 --2- 172.21.15.67:0/1886037543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9044103b60 0x7f9044075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.818+0000 7f9042ffd640 1 -- 172.21.15.67:0/1886037543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9044103b60 msgr2=0x7f9044075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.818+0000 7f9042ffd640 1 --2- 172.21.15.67:0/1886037543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9044103b60 0x7f9044075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.818+0000 7f9042ffd640 1 -- 172.21.15.67:0/1886037543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9044109df0 msgr2=0x7f9044076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.818+0000 7f9042ffd640 1 --2- 172.21.15.67:0/1886037543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9044109df0 0x7f9044076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.818+0000 7f9042ffd640 1 -- 172.21.15.67:0/1886037543 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90441a86b0 con 0x7f90441094a0 2024-09-21T06:46:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.818+0000 7f9042ffd640 1 --2- 172.21.15.67:0/1886037543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90441094a0 0x7f9044075c50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9038005940 tx=0x7f90380040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:46:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.818+0000 7f9040ff9640 1 -- 172.21.15.67:0/1886037543 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903803f030 con 0x7f90441094a0 2024-09-21T06:46:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.818+0000 7f9040ff9640 1 -- 172.21.15.67:0/1886037543 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9038003a80 con 0x7f90441094a0 2024-09-21T06:46:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.818+0000 7f904a140640 1 -- 172.21.15.67:0/1886037543 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90441a8940 con 0x7f90441094a0 2024-09-21T06:46:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.818+0000 7f9040ff9640 1 -- 172.21.15.67:0/1886037543 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9038035070 con 0x7f90441094a0 2024-09-21T06:46:47.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.818+0000 7f904a140640 1 -- 172.21.15.67:0/1886037543 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90441a8e20 con 0x7f90441094a0 2024-09-21T06:46:47.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.819+0000 7f904a140640 1 -- 172.21.15.67:0/1886037543 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9044103f70 con 0x7f90441094a0 2024-09-21T06:46:47.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.821+0000 7f9040ff9640 1 -- 172.21.15.67:0/1886037543 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f9038042030 con 0x7f90441094a0 2024-09-21T06:46:47.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.822+0000 7f9040ff9640 1 --2- 172.21.15.67:0/1886037543 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f902c0b6510 0x7f902c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:47.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.822+0000 7f90437fe640 1 --2- 172.21.15.67:0/1886037543 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f902c0b6510 0x7f902c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:47.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.822+0000 7f9040ff9640 1 -- 172.21.15.67:0/1886037543 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f9038107ef0 con 0x7f90441094a0 2024-09-21T06:46:47.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.823+0000 7f90437fe640 1 --2- 172.21.15.67:0/1886037543 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f902c0b6510 0x7f902c0b8a20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f90340058c0 tx=0x7f903400a250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:46:47.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:47.826+0000 7f9040ff9640 1 -- 172.21.15.67:0/1886037543 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f90380d2790 con 0x7f90441094a0 2024-09-21T06:46:48.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:48.050+0000 7f904a140640 1 -- 172.21.15.67:0/1886037543 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9044076a10 con 0x7f90441094a0 2024-09-21T06:46:48.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:48.051+0000 7f9040ff9640 1 -- 172.21.15.67:0/1886037543 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7f90380d76d0 con 0x7f90441094a0 2024-09-21T06:46:48.053 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:46:48.053 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:46:48.053 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:46:48.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:48.053+0000 7f904a140640 1 -- 172.21.15.67:0/1886037543 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f902c0b6510 msgr2=0x7f902c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:48.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:48.053+0000 7f904a140640 1 --2- 172.21.15.67:0/1886037543 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f902c0b6510 0x7f902c0b8a20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f90340058c0 tx=0x7f903400a250 comp rx=0 tx=0).stop 2024-09-21T06:46:48.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:48.053+0000 7f904a140640 1 -- 172.21.15.67:0/1886037543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90441094a0 msgr2=0x7f9044075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:48.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:48.053+0000 7f904a140640 1 --2- 172.21.15.67:0/1886037543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90441094a0 0x7f9044075c50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9038005940 tx=0x7f90380040e0 comp rx=0 tx=0).stop 2024-09-21T06:46:48.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:48.053+0000 7f904a140640 1 -- 172.21.15.67:0/1886037543 shutdown_connections 2024-09-21T06:46:48.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:48.053+0000 7f904a140640 1 --2- 172.21.15.67:0/1886037543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9044109df0 0x7f9044076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:48.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:48.053+0000 7f904a140640 1 --2- 172.21.15.67:0/1886037543 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f902c0b6510 0x7f902c0b8a20 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:48.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:48.053+0000 7f904a140640 1 --2- 172.21.15.67:0/1886037543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90441094a0 0x7f9044075c50 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:48.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:48.053+0000 7f904a140640 1 --2- 172.21.15.67:0/1886037543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9044103b60 0x7f9044075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:48.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:48.053+0000 7f904a140640 1 -- 172.21.15.67:0/1886037543 >> 172.21.15.67:0/1886037543 conn(0x7f904407b850 msgr2=0x7f9044105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:46:48.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:48.053+0000 7f904a140640 1 -- 172.21.15.67:0/1886037543 shutdown_connections 2024-09-21T06:46:48.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:48.053+0000 7f904a140640 1 -- 172.21.15.67:0/1886037543 wait complete. 2024-09-21T06:46:48.386 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:48 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1886037543' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:46:48.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:48 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1886037543' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:46:48.633 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:48 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1886037543' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:46:48.892 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db --set-id f868a83c --release --await' 2024-09-21T06:46:48.892 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24671 quiesce db --set-id f868a83c --release --await 2024-09-21T06:46:49.148 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:46:49.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:49 smithi067 ceph-mon[31232]: pgmap v667: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 523 KiB/s wr, 9 op/s 2024-09-21T06:46:49.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:49 smithi100 ceph-mon[33401]: pgmap v667: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 523 KiB/s wr, 9 op/s 2024-09-21T06:46:49.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.665+0000 7f5c0ce9c640 1 -- 172.21.15.67:0/1636411552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c08108da0 msgr2=0x7f5c0810f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:49.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.665+0000 7f5c0ce9c640 1 --2- 172.21.15.67:0/1636411552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c08108da0 0x7f5c0810f5d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5c00009ca0 tx=0x7f5c0002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.665+0000 7f5c0ce9c640 1 -- 172.21.15.67:0/1636411552 shutdown_connections 2024-09-21T06:46:49.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.665+0000 7f5c0ce9c640 1 --2- 172.21.15.67:0/1636411552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c08108450 0x7f5c08108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.665+0000 7f5c0ce9c640 1 --2- 172.21.15.67:0/1636411552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c08108da0 0x7f5c0810f5d0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.665+0000 7f5c0ce9c640 1 --2- 172.21.15.67:0/1636411552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c08102b10 0x7f5c08102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.665+0000 7f5c0ce9c640 1 -- 172.21.15.67:0/1636411552 >> 172.21.15.67:0/1636411552 conn(0x7f5c080796d0 msgr2=0x7f5c08079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:46:49.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.665+0000 7f5c0ce9c640 1 -- 172.21.15.67:0/1636411552 shutdown_connections 2024-09-21T06:46:49.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.665+0000 7f5c0ce9c640 1 -- 172.21.15.67:0/1636411552 wait complete. 2024-09-21T06:46:49.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.666+0000 7f5c0ce9c640 1 Processor -- start 2024-09-21T06:46:49.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.666+0000 7f5c0ce9c640 1 -- start start 2024-09-21T06:46:49.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.666+0000 7f5c0ce9c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c08102b10 0x7f5c0819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:49.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.666+0000 7f5c0ce9c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c08108450 0x7f5c0819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:49.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.666+0000 7f5c0ce9c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c08108da0 0x7f5c0819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:49.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.666+0000 7f5c0ce9c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5c08112960 con 0x7f5c08108450 2024-09-21T06:46:49.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.666+0000 7f5c0ce9c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5c081127e0 con 0x7f5c08102b10 2024-09-21T06:46:49.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.666+0000 7f5c0ce9c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5c08112ae0 con 0x7f5c08108da0 2024-09-21T06:46:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.666+0000 7f5c05d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c08108450 0x7f5c0819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.666+0000 7f5c05d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c08108450 0x7f5c0819a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51214/0 (socket says 172.21.15.67:51214) 2024-09-21T06:46:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.666+0000 7f5c05d9b640 1 -- 172.21.15.67:0/1356773460 learned_addr learned my addr 172.21.15.67:0/1356773460 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:46:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.666+0000 7f5c0659c640 1 --2- 172.21.15.67:0/1356773460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c08102b10 0x7f5c0819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.667+0000 7f5c06d9d640 1 --2- 172.21.15.67:0/1356773460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c08108da0 0x7f5c0819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.667+0000 7f5c05d9b640 1 -- 172.21.15.67:0/1356773460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c08108da0 msgr2=0x7f5c0819fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.667+0000 7f5c05d9b640 1 --2- 172.21.15.67:0/1356773460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c08108da0 0x7f5c0819fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.667+0000 7f5c05d9b640 1 -- 172.21.15.67:0/1356773460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c08102b10 msgr2=0x7f5c0819a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.667+0000 7f5c05d9b640 1 --2- 172.21.15.67:0/1356773460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c08102b10 0x7f5c0819a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.667+0000 7f5c05d9b640 1 -- 172.21.15.67:0/1356773460 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c081a02d0 con 0x7f5c08108450 2024-09-21T06:46:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.667+0000 7f5c05d9b640 1 --2- 172.21.15.67:0/1356773460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c08108450 0x7f5c0819a9c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00c8c0 tx=0x7f5bfc00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:46:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.667+0000 7f5bf77fe640 1 -- 172.21.15.67:0/1356773460 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc007bb0 con 0x7f5c08108450 2024-09-21T06:46:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.667+0000 7f5bf77fe640 1 -- 172.21.15.67:0/1356773460 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bfc007d50 con 0x7f5c08108450 2024-09-21T06:46:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.667+0000 7f5c0ce9c640 1 -- 172.21.15.67:0/1356773460 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c081a05c0 con 0x7f5c08108450 2024-09-21T06:46:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.667+0000 7f5bf77fe640 1 -- 172.21.15.67:0/1356773460 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc0186a0 con 0x7f5c08108450 2024-09-21T06:46:49.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.667+0000 7f5c0ce9c640 1 -- 172.21.15.67:0/1356773460 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c081a6de0 con 0x7f5c08108450 2024-09-21T06:46:49.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.670+0000 7f5c13410740 1 Processor -- start 2024-09-21T06:46:49.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.671+0000 7f5c13410740 1 -- start start 2024-09-21T06:46:49.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.671+0000 7f5bf77fe640 1 -- 172.21.15.67:0/1356773460 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f5bfc00fa00 con 0x7f5c08108450 2024-09-21T06:46:49.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.671+0000 7f5c13410740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ae2fe1520 0x563ae3387510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:49.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.671+0000 7f5c13410740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ae2fb8660 0x563ae3387a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:49.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.671+0000 7f5c13410740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ae2fc0840 0x563ae3387f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:49.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.671+0000 7f5c13410740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x563ae3272dc0 con 0x563ae2fb8660 2024-09-21T06:46:49.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.671+0000 7f5c13410740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x563ae32ff1f0 con 0x563ae2fe1520 2024-09-21T06:46:49.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.671+0000 7f5c13410740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563ae32ee620 con 0x563ae2fc0840 2024-09-21T06:46:49.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.671+0000 7f5c0659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ae2fb8660 0x563ae3387a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:49.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.671+0000 7f5c0659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ae2fb8660 0x563ae3387a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51216/0 (socket says 172.21.15.67:51216) 2024-09-21T06:46:49.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.671+0000 7f5c0659c640 1 -- 172.21.15.67:0/776665987 learned_addr learned my addr 172.21.15.67:0/776665987 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:46:49.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.671+0000 7f5c06d9d640 1 --2- 172.21.15.67:0/776665987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ae2fe1520 0x563ae3387510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:49.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.671+0000 7f5c05d9b640 1 --2- 172.21.15.67:0/776665987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ae2fc0840 0x563ae3387f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:49.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.672+0000 7f5c0659c640 1 -- 172.21.15.67:0/776665987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ae2fc0840 msgr2=0x563ae3387f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:46:49.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.672+0000 7f5c0659c640 1 --2- 172.21.15.67:0/776665987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ae2fc0840 0x563ae3387f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.672+0000 7f5c0659c640 1 -- 172.21.15.67:0/776665987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ae2fe1520 msgr2=0x563ae3387510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:46:49.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.672+0000 7f5c0659c640 1 --2- 172.21.15.67:0/776665987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ae2fe1520 0x563ae3387510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.672+0000 7f5c0659c640 1 -- 172.21.15.67:0/776665987 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ae32b6fd0 con 0x563ae2fb8660 2024-09-21T06:46:49.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.672+0000 7f5c05d9b640 1 --2- 172.21.15.67:0/776665987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ae2fc0840 0x563ae3387f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:46:49.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.672+0000 7f5c0659c640 1 --2- 172.21.15.67:0/776665987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ae2fb8660 0x563ae3387a50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f5bf800b760 tx=0x7f5bf800bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e356b41f953feaef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:46:49.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.672+0000 7f5bf77fe640 1 --2- 172.21.15.67:0/1356773460 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5bf00b65c0 0x7f5bf00b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:49.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.672+0000 7f5bf4ff9640 1 -- 172.21.15.67:0/776665987 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8003dd0 con 0x563ae2fb8660 2024-09-21T06:46:49.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.672+0000 7f5bf4ff9640 1 -- 172.21.15.67:0/776665987 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bf8015070 con 0x563ae2fb8660 2024-09-21T06:46:49.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.672+0000 7f5bf4ff9640 1 -- 172.21.15.67:0/776665987 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8004c80 con 0x563ae2fb8660 2024-09-21T06:46:49.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.672+0000 7f5c06d9d640 1 --2- 172.21.15.67:0/1356773460 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5bf00b65c0 0x7f5bf00b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:49.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.672+0000 7f5bf77fe640 1 -- 172.21.15.67:0/1356773460 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f5bfc0e7630 con 0x7f5c08108450 2024-09-21T06:46:49.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.673+0000 7f5c13410740 1 -- 172.21.15.67:0/776665987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ae2fb8660 msgr2=0x563ae3387a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:49.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.673+0000 7f5c13410740 1 --2- 172.21.15.67:0/776665987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ae2fb8660 0x563ae3387a50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f5bf800b760 tx=0x7f5bf800bc20 comp rx=0 tx=0).stop 2024-09-21T06:46:49.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.673+0000 7f5c06d9d640 1 --2- 172.21.15.67:0/1356773460 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5bf00b65c0 0x7f5bf00b8ad0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5c00006010 tx=0x7f5c00009800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:46:49.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.673+0000 7f5c13410740 1 -- 172.21.15.67:0/776665987 shutdown_connections 2024-09-21T06:46:49.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.673+0000 7f5c13410740 1 --2- 172.21.15.67:0/776665987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ae2fc0840 0x563ae3387f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.673+0000 7f5c13410740 1 --2- 172.21.15.67:0/776665987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ae2fb8660 0x563ae3387a50 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.673+0000 7f5c13410740 1 --2- 172.21.15.67:0/776665987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ae2fe1520 0x563ae3387510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.673+0000 7f5c13410740 1 -- 172.21.15.67:0/776665987 >> 172.21.15.67:0/776665987 conn(0x563ae2fd5910 msgr2=0x563ae3385400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:46:49.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.673+0000 7f5c13410740 1 -- 172.21.15.67:0/776665987 shutdown_connections 2024-09-21T06:46:49.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.673+0000 7f5c13410740 1 -- 172.21.15.67:0/776665987 wait complete. 2024-09-21T06:46:49.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.674+0000 7f5c13410740 1 -- start start 2024-09-21T06:46:49.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.674+0000 7f5c13410740 1 Processor -- start 2024-09-21T06:46:49.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.674+0000 7f5c13410740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ae2fb8660 0x563ae3392ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:49.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.674+0000 7f5c13410740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ae2fc0840 0x563ae33930e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:49.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.674+0000 7f5c13410740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ae2fe1520 0x563ae339ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:49.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.674+0000 7f5c13410740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x563ae3289300 con 0x563ae2fb8660 2024-09-21T06:46:49.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.674+0000 7f5c13410740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x563ae32f2430 con 0x563ae2fe1520 2024-09-21T06:46:49.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.674+0000 7f5c13410740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563ae314fbb0 con 0x563ae2fc0840 2024-09-21T06:46:49.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.674+0000 7f5c13410740 10 client.0 _ll_register_callbacks cb 0x563ae32a3120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:46:49.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.674+0000 7f5c0659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ae2fb8660 0x563ae3392ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:49.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.674+0000 7f5c0659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ae2fb8660 0x563ae3392ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51228/0 (socket says 172.21.15.67:51228) 2024-09-21T06:46:49.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.674+0000 7f5c0659c640 1 -- 172.21.15.67:0/119875198 learned_addr learned my addr 172.21.15.67:0/119875198 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:46:49.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.674+0000 7f5c06d9d640 1 --2- 172.21.15.67:0/119875198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ae2fe1520 0x563ae339ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:49.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.674+0000 7f5c05d9b640 1 --2- 172.21.15.67:0/119875198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ae2fc0840 0x563ae33930e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:49.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.675+0000 7f5c0659c640 1 -- 172.21.15.67:0/119875198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ae2fc0840 msgr2=0x563ae33930e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:46:49.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.675+0000 7f5c0659c640 1 --2- 172.21.15.67:0/119875198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ae2fc0840 0x563ae33930e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.675+0000 7f5c0659c640 1 -- 172.21.15.67:0/119875198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ae2fe1520 msgr2=0x563ae339ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:46:49.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.675+0000 7f5c0659c640 1 --2- 172.21.15.67:0/119875198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ae2fe1520 0x563ae339ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.675+0000 7f5c0659c640 1 -- 172.21.15.67:0/119875198 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ae3278780 con 0x563ae2fb8660 2024-09-21T06:46:49.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.675+0000 7f5c0659c640 1 --2- 172.21.15.67:0/119875198 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ae2fb8660 0x563ae3392ba0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5bf80044d0 tx=0x7f5bf800bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ca159b3d79e217 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:46:49.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.675+0000 7f5bedffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:46:49.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.675+0000 7f5bedffb640 1 -- 172.21.15.67:0/119875198 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8011440 con 0x563ae2fb8660 2024-09-21T06:46:49.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.675+0000 7f5c13410740 1 -- 172.21.15.67:0/119875198 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bf8015070 con 0x563ae2fb8660 2024-09-21T06:46:49.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.675+0000 7f5bedffb640 1 -- 172.21.15.67:0/119875198 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f5bf80115e0 con 0x563ae2fb8660 2024-09-21T06:46:49.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.675+0000 7f5bedffb640 1 -- 172.21.15.67:0/119875198 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5bf8030050 con 0x563ae2fb8660 2024-09-21T06:46:49.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.675+0000 7f5c13410740 10 client.15861 fetch_fsmap learned FSMap version 21 2024-09-21T06:46:49.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.675+0000 7f5c13410740 1 -- 172.21.15.67:0/119875198 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563ae304cb00 con 0x563ae2fb8660 2024-09-21T06:46:49.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.676+0000 7f5bedffb640 1 -- 172.21.15.67:0/119875198 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bf8015070 con 0x563ae2fb8660 2024-09-21T06:46:49.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.676+0000 7f5bedffb640 1 -- 172.21.15.67:0/119875198 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f5bf80257a0 con 0x563ae2fb8660 2024-09-21T06:46:49.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.676+0000 7f5c13410740 10 client.15861 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:46:49.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.676+0000 7f5c13410740 10 client.15861 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:46:49.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.676+0000 7f5c13410740 20 client.15861 populate_metadata read hostname 'smithi067' 2024-09-21T06:46:49.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.676+0000 7f5c13410740 1 --2- 172.21.15.67:0/119875198 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x563ae2f9fea0 0x563ae33a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:49.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.676+0000 7f5c13410740 4 client.15861 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:46:49.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.676+0000 7f5c13410740 1 -- 172.21.15.67:0/119875198 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ae3301140 con 0x563ae2f9fea0 2024-09-21T06:46:49.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.676+0000 7f5c05d9b640 1 --2- 172.21.15.67:0/119875198 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x563ae2f9fea0 0x563ae33a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:49.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.677+0000 7f5bedffb640 10 client.15861 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:46:49.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.677+0000 7f5c05d9b640 1 --2- 172.21.15.67:0/119875198 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x563ae2f9fea0 0x563ae33a4e20 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5260936d3b5d0140 server_cookie=e6b3ead4c1305db4 in_seq=0 out_seq=0 2024-09-21T06:46:49.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.677+0000 7f5bedffb640 1 -- 172.21.15.67:0/119875198 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563ae3301140 con 0x563ae2f9fea0 2024-09-21T06:46:49.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.677+0000 7f5bedffb640 10 client.15861 handle_command_reply: tid=0 2024-09-21T06:46:49.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.677+0000 7f5c13410740 1 client.15861 shutdown 2024-09-21T06:46:49.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.678+0000 7f5c13410740 1 -- 172.21.15.67:0/119875198 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ae2fb8660 msgr2=0x563ae3392ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:49.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.678+0000 7f5c13410740 1 --2- 172.21.15.67:0/119875198 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ae2fb8660 0x563ae3392ba0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5bf80044d0 tx=0x7f5bf800bee0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.678+0000 7f5c13410740 1 -- 172.21.15.67:0/119875198 shutdown_connections 2024-09-21T06:46:49.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.678+0000 7f5c13410740 1 --2- 172.21.15.67:0/119875198 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x563ae2f9fea0 0x563ae33a4e20 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.678+0000 7f5c13410740 1 --2- 172.21.15.67:0/119875198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ae2fe1520 0x563ae339ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.678+0000 7f5c13410740 1 --2- 172.21.15.67:0/119875198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ae2fc0840 0x563ae33930e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.678+0000 7f5bedffb640 0 client.15861 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:46:49.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.678+0000 7f5c13410740 1 --2- 172.21.15.67:0/119875198 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ae2fb8660 0x563ae3392ba0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.678+0000 7f5c13410740 1 -- 172.21.15.67:0/119875198 >> 172.21.15.67:0/119875198 conn(0x563ae2fd5910 msgr2=0x563ae338a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:46:49.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.678+0000 7f5c13410740 1 -- 172.21.15.67:0/119875198 shutdown_connections 2024-09-21T06:46:49.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.678+0000 7f5c13410740 1 -- 172.21.15.67:0/119875198 wait complete. 2024-09-21T06:46:49.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.678+0000 7f5c13410740 20 client.15861 trim_cache size 0 max 16384 2024-09-21T06:46:49.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.695+0000 7f5c13410740 1 Processor -- start 2024-09-21T06:46:49.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.695+0000 7f5c13410740 1 -- start start 2024-09-21T06:46:49.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.695+0000 7f5c13410740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ae2f9fea0 0x563ae338fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:49.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.695+0000 7f5c13410740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ae2fb8660 0x563ae33902d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:49.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.695+0000 7f5c13410740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ae2fc0840 0x563ae33aa500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:49.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.695+0000 7f5c13410740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5c081aeb00 con 0x563ae2fb8660 2024-09-21T06:46:49.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.695+0000 7f5c13410740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5c081b1380 con 0x563ae2f9fea0 2024-09-21T06:46:49.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.695+0000 7f5c13410740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5c081af170 con 0x563ae2fc0840 2024-09-21T06:46:49.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.696+0000 7f5c05d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ae2fb8660 0x563ae33902d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:49.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.696+0000 7f5c05d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ae2fb8660 0x563ae33902d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51242/0 (socket says 172.21.15.67:51242) 2024-09-21T06:46:49.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.696+0000 7f5c05d9b640 1 -- 172.21.15.67:0/2210853521 learned_addr learned my addr 172.21.15.67:0/2210853521 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:46:49.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.696+0000 7f5c0659c640 1 --2- 172.21.15.67:0/2210853521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ae2f9fea0 0x563ae338fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:49.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.696+0000 7f5c05d9b640 1 -- 172.21.15.67:0/2210853521 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ae2fc0840 msgr2=0x563ae33aa500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:46:49.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.696+0000 7f5c05d9b640 1 --2- 172.21.15.67:0/2210853521 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ae2fc0840 0x563ae33aa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.696+0000 7f5c05d9b640 1 -- 172.21.15.67:0/2210853521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ae2f9fea0 msgr2=0x563ae338fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:46:49.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.696+0000 7f5c05d9b640 1 --2- 172.21.15.67:0/2210853521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ae2f9fea0 0x563ae338fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.696+0000 7f5c05d9b640 1 -- 172.21.15.67:0/2210853521 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ae30c3960 con 0x563ae2fb8660 2024-09-21T06:46:49.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.696+0000 7f5c0659c640 1 --2- 172.21.15.67:0/2210853521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ae2f9fea0 0x563ae338fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:46:49.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.696+0000 7f5c05d9b640 1 --2- 172.21.15.67:0/2210853521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ae2fb8660 0x563ae33902d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00cfd0 tx=0x7f5bfc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a6ababe40904aa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:46:49.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.696+0000 7f5beffff640 1 -- 172.21.15.67:0/2210853521 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc0b1ed0 con 0x563ae2fb8660 2024-09-21T06:46:49.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.696+0000 7f5beffff640 1 -- 172.21.15.67:0/2210853521 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bfc0b6e10 con 0x563ae2fb8660 2024-09-21T06:46:49.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.696+0000 7f5beffff640 1 -- 172.21.15.67:0/2210853521 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc0a7af0 con 0x563ae2fb8660 2024-09-21T06:46:49.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.697+0000 7f5c13410740 1 -- 172.21.15.67:0/2210853521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ae2fb8660 msgr2=0x563ae33902d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:49.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.697+0000 7f5c13410740 1 --2- 172.21.15.67:0/2210853521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ae2fb8660 0x563ae33902d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00cfd0 tx=0x7f5bfc000bc0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.697+0000 7f5c13410740 1 -- 172.21.15.67:0/2210853521 shutdown_connections 2024-09-21T06:46:49.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.697+0000 7f5c13410740 1 --2- 172.21.15.67:0/2210853521 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ae2fc0840 0x563ae33aa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.697+0000 7f5c13410740 1 --2- 172.21.15.67:0/2210853521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ae2fb8660 0x563ae33902d0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.697+0000 7f5c13410740 1 --2- 172.21.15.67:0/2210853521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ae2f9fea0 0x563ae338fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.697+0000 7f5c13410740 1 -- 172.21.15.67:0/2210853521 >> 172.21.15.67:0/2210853521 conn(0x563ae2fd5910 msgr2=0x563ae3388f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:46:49.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.697+0000 7f5c13410740 1 -- 172.21.15.67:0/2210853521 shutdown_connections 2024-09-21T06:46:49.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.697+0000 7f5c13410740 1 -- 172.21.15.67:0/2210853521 wait complete. 2024-09-21T06:46:49.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.697+0000 7f5c13410740 1 -- start start 2024-09-21T06:46:49.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.698+0000 7f5c13410740 1 Processor -- start 2024-09-21T06:46:49.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.698+0000 7f5c13410740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ae2f9fea0 0x563ae3399510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:49.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.698+0000 7f5c13410740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ae2fb8660 0x563ae3399a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:49.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.698+0000 7f5c13410740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ae2fc0840 0x563ae340e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:49.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.698+0000 7f5c13410740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5c081af840 con 0x563ae2f9fea0 2024-09-21T06:46:49.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.698+0000 7f5c13410740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5c081b0cb0 con 0x563ae2fb8660 2024-09-21T06:46:49.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.698+0000 7f5c13410740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5c081aff10 con 0x563ae2fc0840 2024-09-21T06:46:49.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.698+0000 7f5c13410740 10 client.0 _ll_register_callbacks cb 0x7f5c00009440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:46:49.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.698+0000 7f5c0659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ae2f9fea0 0x563ae3399510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:49.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.698+0000 7f5c0659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ae2f9fea0 0x563ae3399510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51250/0 (socket says 172.21.15.67:51250) 2024-09-21T06:46:49.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.698+0000 7f5c0659c640 1 -- 172.21.15.67:0/3757173215 learned_addr learned my addr 172.21.15.67:0/3757173215 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:46:49.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.698+0000 7f5c05d9b640 1 --2- 172.21.15.67:0/3757173215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ae2fb8660 0x563ae3399a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:49.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.698+0000 7f5c06d9d640 1 --2- 172.21.15.67:0/3757173215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ae2fc0840 0x563ae340e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:49.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.698+0000 7f5c0659c640 1 -- 172.21.15.67:0/3757173215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ae2fc0840 msgr2=0x563ae340e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:46:49.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.698+0000 7f5c0659c640 1 --2- 172.21.15.67:0/3757173215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ae2fc0840 0x563ae340e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.698+0000 7f5c0659c640 1 -- 172.21.15.67:0/3757173215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ae2fb8660 msgr2=0x563ae3399a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:46:49.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.698+0000 7f5c0659c640 1 --2- 172.21.15.67:0/3757173215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ae2fb8660 0x563ae3399a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.698+0000 7f5c0659c640 1 -- 172.21.15.67:0/3757173215 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ae340ec50 con 0x563ae2f9fea0 2024-09-21T06:46:49.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.698+0000 7f5c06d9d640 1 --2- 172.21.15.67:0/3757173215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ae2fc0840 0x563ae340e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:46:49.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.699+0000 7f5c0659c640 1 --2- 172.21.15.67:0/3757173215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ae2f9fea0 0x563ae3399510 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5bf80039c0 tx=0x7f5bf8019420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1ebcf387970b95 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:46:49.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.699+0000 7f5bedffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:46:49.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.699+0000 7f5bedffb640 1 -- 172.21.15.67:0/3757173215 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf800b390 con 0x563ae2f9fea0 2024-09-21T06:46:49.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.699+0000 7f5bedffb640 1 -- 172.21.15.67:0/3757173215 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f5bf80042d0 con 0x563ae2f9fea0 2024-09-21T06:46:49.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.699+0000 7f5c13410740 1 -- 172.21.15.67:0/3757173215 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bfc0b6e10 con 0x563ae2f9fea0 2024-09-21T06:46:49.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.699+0000 7f5bedffb640 1 -- 172.21.15.67:0/3757173215 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5bf8044050 con 0x563ae2f9fea0 2024-09-21T06:46:49.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.699+0000 7f5c13410740 10 client.15867 fetch_fsmap learned FSMap version 21 2024-09-21T06:46:49.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.699+0000 7f5c13410740 1 -- 172.21.15.67:0/3757173215 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563ae340fef0 con 0x563ae2f9fea0 2024-09-21T06:46:49.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.699+0000 7f5bedffb640 1 -- 172.21.15.67:0/3757173215 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bfc0b6e10 con 0x563ae2f9fea0 2024-09-21T06:46:49.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.699+0000 7f5bedffb640 1 -- 172.21.15.67:0/3757173215 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f5bf8019bb0 con 0x563ae2f9fea0 2024-09-21T06:46:49.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.699+0000 7f5c13410740 10 client.15867 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:46:49.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.699+0000 7f5c13410740 10 client.15867 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:46:49.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.699+0000 7f5c13410740 20 client.15867 populate_metadata read hostname 'smithi067' 2024-09-21T06:46:49.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.700+0000 7f5c13410740 1 --2- 172.21.15.67:0/3757173215 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x563ae2fe1520 0x563ae34126e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:49.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.700+0000 7f5c13410740 4 client.15867 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f868a83c", "release": true, "await": true}] 2024-09-21T06:46:49.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.700+0000 7f5c13410740 1 -- 172.21.15.67:0/3757173215 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f868a83c", "release": true, "await": true}) -- 0x563ae32fcbf0 con 0x563ae2fe1520 2024-09-21T06:46:49.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.700+0000 7f5c05d9b640 1 --2- 172.21.15.67:0/3757173215 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x563ae2fe1520 0x563ae34126e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:49.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.700+0000 7f5c05d9b640 1 --2- 172.21.15.67:0/3757173215 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x563ae2fe1520 0x563ae34126e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=906ad6696a9c56ad server_cookie=7487b1431f0e48 in_seq=0 out_seq=0 2024-09-21T06:46:49.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.700+0000 7f5bedffb640 10 client.15867 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:46:49.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.703+0000 7f5bedffb640 1 -- 172.21.15.67:0/3757173215 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563ae32fcbf0 con 0x563ae2fe1520 2024-09-21T06:46:49.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.703+0000 7f5bedffb640 10 client.15867 handle_command_reply: tid=0 2024-09-21T06:46:49.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.703+0000 7f5c13410740 1 client.15867 shutdown 2024-09-21T06:46:49.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.704+0000 7f5c13410740 1 -- 172.21.15.67:0/3757173215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ae2f9fea0 msgr2=0x563ae3399510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:49.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.704+0000 7f5c13410740 1 --2- 172.21.15.67:0/3757173215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ae2f9fea0 0x563ae3399510 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5bf80039c0 tx=0x7f5bf8019420 comp rx=0 tx=0).stop 2024-09-21T06:46:49.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.704+0000 7f5c13410740 1 -- 172.21.15.67:0/3757173215 shutdown_connections 2024-09-21T06:46:49.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.704+0000 7f5c13410740 1 --2- 172.21.15.67:0/3757173215 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x563ae2fe1520 0x563ae34126e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.704+0000 7f5c13410740 1 --2- 172.21.15.67:0/3757173215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ae2fb8660 0x563ae3399a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.704+0000 7f5c13410740 1 --2- 172.21.15.67:0/3757173215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ae2fc0840 0x563ae340e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.704+0000 7f5bedffb640 0 client.15867 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:46:49.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.704+0000 7f5c13410740 1 --2- 172.21.15.67:0/3757173215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ae2f9fea0 0x563ae3399510 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.704+0000 7f5c13410740 1 -- 172.21.15.67:0/3757173215 >> 172.21.15.67:0/3757173215 conn(0x563ae2fd5910 msgr2=0x563ae3394820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:46:49.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.704+0000 7f5c13410740 1 -- 172.21.15.67:0/3757173215 shutdown_connections 2024-09-21T06:46:49.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.704+0000 7f5c13410740 1 -- 172.21.15.67:0/3757173215 wait complete. 2024-09-21T06:46:49.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.704+0000 7f5c13410740 20 client.15867 trim_cache size 0 max 16384 2024-09-21T06:46:49.709 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-21T06:46:49.709 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 21, 2024-09-21T06:46:49.709 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-09-21T06:46:49.709 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 40, 2024-09-21T06:46:49.709 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-21T06:46:49.709 INFO:teuthology.orchestra.run.smithi067.stdout: "f868a83c": { 2024-09-21T06:46:49.709 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 40, 2024-09-21T06:46:49.710 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-21T06:46:49.710 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:46:49.710 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-21T06:46:49.710 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:46:49.710 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:46:49.710 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-21T06:46:49.710 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 144.5, 2024-09-21T06:46:49.710 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-21T06:46:49.710 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-21T06:46:49.710 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-21T06:46:49.710 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:46:49.710 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-21T06:46:49.710 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:46:49.710 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:46:49.711 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:46:49.711 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:46:49.711 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:46:49.711 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:46:49.711 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-21T06:46:49.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.705+0000 7f5c0ce9c640 1 -- 172.21.15.67:0/1356773460 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5bf00b65c0 msgr2=0x7f5bf00b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:49.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.705+0000 7f5c0ce9c640 1 --2- 172.21.15.67:0/1356773460 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5bf00b65c0 0x7f5bf00b8ad0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5c00006010 tx=0x7f5c00009800 comp rx=0 tx=0).stop 2024-09-21T06:46:49.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.705+0000 7f5c0ce9c640 1 -- 172.21.15.67:0/1356773460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c08108450 msgr2=0x7f5c0819a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:49.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.705+0000 7f5c0ce9c640 1 --2- 172.21.15.67:0/1356773460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c08108450 0x7f5c0819a9c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00c8c0 tx=0x7f5bfc00cd80 comp rx=0 tx=0).stop 2024-09-21T06:46:49.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.705+0000 7f5c0ce9c640 1 -- 172.21.15.67:0/1356773460 shutdown_connections 2024-09-21T06:46:49.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.705+0000 7f5c0ce9c640 1 --2- 172.21.15.67:0/1356773460 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f5bf00b65c0 0x7f5bf00b8ad0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.705+0000 7f5c0ce9c640 1 --2- 172.21.15.67:0/1356773460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c08108da0 0x7f5c0819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.705+0000 7f5c0ce9c640 1 --2- 172.21.15.67:0/1356773460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c08108450 0x7f5c0819a9c0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.705+0000 7f5c0ce9c640 1 --2- 172.21.15.67:0/1356773460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c08102b10 0x7f5c0819a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:49.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.705+0000 7f5c0ce9c640 1 -- 172.21.15.67:0/1356773460 >> 172.21.15.67:0/1356773460 conn(0x7f5c080796d0 msgr2=0x7f5c08101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:46:49.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.705+0000 7f5c0ce9c640 1 -- 172.21.15.67:0/1356773460 shutdown_connections 2024-09-21T06:46:49.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:49.706+0000 7f5c0ce9c640 1 -- 172.21.15.67:0/1356773460 wait complete. 2024-09-21T06:46:49.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:49 smithi203 ceph-mon[33735]: pgmap v667: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 523 KiB/s wr, 9 op/s 2024-09-21T06:46:50.668 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f868a83c', total seconds elapsed: 28.1 2024-09-21T06:46:50.668 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.5 2024-09-21T06:46:50.669 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:46:51.339 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:46:51.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:51 smithi067 ceph-mon[31232]: pgmap v668: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 2 op/s 2024-09-21T06:46:51.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:51 smithi100 ceph-mon[33401]: pgmap v668: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 2 op/s 2024-09-21T06:46:51.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:51 smithi203 ceph-mon[33735]: pgmap v668: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 2 op/s 2024-09-21T06:46:52.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.070+0000 7f020684a640 1 -- 172.21.15.67:0/1310724651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01f80a5790 msgr2=0x7f01f80b7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:52.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.070+0000 7f020684a640 1 --2- 172.21.15.67:0/1310724651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01f80a5790 0x7f01f80b7620 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f01f0009900 tx=0x7f01f002a420 comp rx=0 tx=0).stop 2024-09-21T06:46:52.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.072+0000 7f020684a640 1 -- 172.21.15.67:0/1310724651 shutdown_connections 2024-09-21T06:46:52.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.072+0000 7f020684a640 1 --2- 172.21.15.67:0/1310724651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01f80b7b60 0x7f01f80b9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:52.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.072+0000 7f020684a640 1 --2- 172.21.15.67:0/1310724651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01f80a5790 0x7f01f80b7620 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:52.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.072+0000 7f020684a640 1 --2- 172.21.15.67:0/1310724651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01f80a4e40 0x7f01f80a5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:52.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.072+0000 7f020684a640 1 -- 172.21.15.67:0/1310724651 >> 172.21.15.67:0/1310724651 conn(0x7f01f801cc20 msgr2=0x7f01f801d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:46:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.073+0000 7f020684a640 1 -- 172.21.15.67:0/1310724651 shutdown_connections 2024-09-21T06:46:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.078+0000 7f020684a640 1 -- 172.21.15.67:0/1310724651 wait complete. 2024-09-21T06:46:52.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.080+0000 7f020684a640 1 Processor -- start 2024-09-21T06:46:52.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.080+0000 7f020684a640 1 -- start start 2024-09-21T06:46:52.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.080+0000 7f020684a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01f80a4e40 0x7f01f80b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:52.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.080+0000 7f020684a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01f80a5790 0x7f01f80b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:52.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.080+0000 7f020684a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01f80b7b60 0x7f01f80b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:52.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.080+0000 7f020684a640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f01f80bd610 con 0x7f01f80b7b60 2024-09-21T06:46:52.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.080+0000 7f020684a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f01f80bd490 con 0x7f01f80a4e40 2024-09-21T06:46:52.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.080+0000 7f020684a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f01f80bd790 con 0x7f01f80a5790 2024-09-21T06:46:52.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.081+0000 7f0206049640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01f80b7b60 0x7f01f80b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:52.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.081+0000 7f0206049640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01f80b7b60 0x7f01f80b0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51256/0 (socket says 172.21.15.67:51256) 2024-09-21T06:46:52.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.081+0000 7f0206049640 1 -- 172.21.15.67:0/3066303084 learned_addr learned my addr 172.21.15.67:0/3066303084 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:46:52.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.081+0000 7f0205047640 1 --2- 172.21.15.67:0/3066303084 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01f80a5790 0x7f01f80b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:52.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.082+0000 7f0205848640 1 --2- 172.21.15.67:0/3066303084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01f80a4e40 0x7f01f80b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:52.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.082+0000 7f0206049640 1 -- 172.21.15.67:0/3066303084 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01f80a5790 msgr2=0x7f01f80b01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:52.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.082+0000 7f0206049640 1 --2- 172.21.15.67:0/3066303084 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01f80a5790 0x7f01f80b01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:52.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.082+0000 7f0206049640 1 -- 172.21.15.67:0/3066303084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01f80a4e40 msgr2=0x7f01f80b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:52.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.082+0000 7f0206049640 1 --2- 172.21.15.67:0/3066303084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01f80a4e40 0x7f01f80b6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:52.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.082+0000 7f0206049640 1 -- 172.21.15.67:0/3066303084 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f80b5b60 con 0x7f01f80b7b60 2024-09-21T06:46:52.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.083+0000 7f0206049640 1 --2- 172.21.15.67:0/3066303084 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01f80b7b60 0x7f01f80b0820 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f020002ebd0 tx=0x7f020006de90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:46:52.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.084+0000 7f01f6ffd640 1 -- 172.21.15.67:0/3066303084 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02000729c0 con 0x7f01f80b7b60 2024-09-21T06:46:52.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.084+0000 7f020684a640 1 -- 172.21.15.67:0/3066303084 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f80b5e50 con 0x7f01f80b7b60 2024-09-21T06:46:52.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.084+0000 7f020684a640 1 -- 172.21.15.67:0/3066303084 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f80b6390 con 0x7f01f80b7b60 2024-09-21T06:46:52.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.085+0000 7f01f6ffd640 1 -- 172.21.15.67:0/3066303084 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0200078420 con 0x7f01f80b7b60 2024-09-21T06:46:52.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.086+0000 7f01f6ffd640 1 -- 172.21.15.67:0/3066303084 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02000775e0 con 0x7f01f80b7b60 2024-09-21T06:46:52.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.087+0000 7f01f4ff9640 1 -- 172.21.15.67:0/3066303084 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01f80a61c0 con 0x7f01f80b7b60 2024-09-21T06:46:52.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.090+0000 7f01f6ffd640 1 -- 172.21.15.67:0/3066303084 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f020007f030 con 0x7f01f80b7b60 2024-09-21T06:46:52.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.092+0000 7f01f6ffd640 1 --2- 172.21.15.67:0/3066303084 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f01ec0b6580 0x7f01ec0b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:52.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.093+0000 7f01f6ffd640 1 -- 172.21.15.67:0/3066303084 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f02000d55a0 con 0x7f01f80b7b60 2024-09-21T06:46:52.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.093+0000 7f0205848640 1 --2- 172.21.15.67:0/3066303084 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f01ec0b6580 0x7f01ec0b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:52.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.094+0000 7f0205848640 1 --2- 172.21.15.67:0/3066303084 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f01ec0b6580 0x7f01ec0b8a90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f01f80ab8f0 tx=0x7f01fc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:46:52.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.107+0000 7f01f6ffd640 1 -- 172.21.15.67:0/3066303084 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0200086060 con 0x7f01f80b7b60 2024-09-21T06:46:52.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.571+0000 7f01f4ff9640 1 -- 172.21.15.67:0/3066303084 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01f80b14c0 con 0x7f01f80b7b60 2024-09-21T06:46:52.578 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:46:52.578 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:46:52.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.572+0000 7f01f6ffd640 1 -- 172.21.15.67:0/3066303084 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7f0200111fb0 con 0x7f01f80b7b60 2024-09-21T06:46:52.579 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:46:52.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.576+0000 7f01f4ff9640 1 -- 172.21.15.67:0/3066303084 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f01ec0b6580 msgr2=0x7f01ec0b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:52.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.576+0000 7f01f4ff9640 1 --2- 172.21.15.67:0/3066303084 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f01ec0b6580 0x7f01ec0b8a90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f01f80ab8f0 tx=0x7f01fc008040 comp rx=0 tx=0).stop 2024-09-21T06:46:52.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.576+0000 7f01f4ff9640 1 -- 172.21.15.67:0/3066303084 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01f80b7b60 msgr2=0x7f01f80b0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:52.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.576+0000 7f01f4ff9640 1 --2- 172.21.15.67:0/3066303084 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01f80b7b60 0x7f01f80b0820 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f020002ebd0 tx=0x7f020006de90 comp rx=0 tx=0).stop 2024-09-21T06:46:52.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.577+0000 7f01f4ff9640 1 -- 172.21.15.67:0/3066303084 shutdown_connections 2024-09-21T06:46:52.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.577+0000 7f01f4ff9640 1 --2- 172.21.15.67:0/3066303084 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f01ec0b6580 0x7f01ec0b8a90 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:52.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.577+0000 7f01f4ff9640 1 --2- 172.21.15.67:0/3066303084 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01f80b7b60 0x7f01f80b0820 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:52.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.577+0000 7f01f4ff9640 1 --2- 172.21.15.67:0/3066303084 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01f80a5790 0x7f01f80b01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:52.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.577+0000 7f01f4ff9640 1 --2- 172.21.15.67:0/3066303084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01f80a4e40 0x7f01f80b6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:52.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.577+0000 7f01f4ff9640 1 -- 172.21.15.67:0/3066303084 >> 172.21.15.67:0/3066303084 conn(0x7f01f801cc20 msgr2=0x7f01f80ac270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:46:52.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.579+0000 7f01f4ff9640 1 -- 172.21.15.67:0/3066303084 shutdown_connections 2024-09-21T06:46:52.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:52.579+0000 7f01f4ff9640 1 -- 172.21.15.67:0/3066303084 wait complete. 2024-09-21T06:46:53.376 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0' 2024-09-21T06:46:53.376 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24671 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0 2024-09-21T06:46:53.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:53 smithi067 ceph-mon[31232]: pgmap v669: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 69 op/s 2024-09-21T06:46:53.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:53 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3066303084' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:46:53.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:53 smithi100 ceph-mon[33401]: pgmap v669: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 69 op/s 2024-09-21T06:46:53.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:53 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3066303084' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:46:53.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:53 smithi203 ceph-mon[33735]: pgmap v669: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 69 op/s 2024-09-21T06:46:53.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:53 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3066303084' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:46:53.994 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:46:54.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.754+0000 7f8303ec8640 1 -- 172.21.15.67:0/506946943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82fc10a490 msgr2=0x7f82fc10a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:54.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.754+0000 7f8303ec8640 1 --2- 172.21.15.67:0/506946943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82fc10a490 0x7f82fc10a8a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f82f0009890 tx=0x7f82f002a3b0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.755+0000 7f8303ec8640 1 -- 172.21.15.67:0/506946943 shutdown_connections 2024-09-21T06:46:54.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.755+0000 7f8303ec8640 1 --2- 172.21.15.67:0/506946943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82fc10c8c0 0x7f82fc10ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.755+0000 7f8303ec8640 1 --2- 172.21.15.67:0/506946943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82fc10d210 0x7f82fc116ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.755+0000 7f8303ec8640 1 --2- 172.21.15.67:0/506946943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82fc10a490 0x7f82fc10a8a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.755+0000 7f8303ec8640 1 -- 172.21.15.67:0/506946943 >> 172.21.15.67:0/506946943 conn(0x7f82fc0795f0 msgr2=0x7f82fc07ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:46:54.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.759+0000 7f8303ec8640 1 -- 172.21.15.67:0/506946943 shutdown_connections 2024-09-21T06:46:54.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.759+0000 7f8303ec8640 1 -- 172.21.15.67:0/506946943 wait complete. 2024-09-21T06:46:54.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.760+0000 7f8303ec8640 1 Processor -- start 2024-09-21T06:46:54.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.760+0000 7f8303ec8640 1 -- start start 2024-09-21T06:46:54.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.760+0000 7f8303ec8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82fc10a490 0x7f82fc074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:54.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.760+0000 7f8303ec8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82fc10c8c0 0x7f82fc06e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:54.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.760+0000 7f8303ec8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82fc10d210 0x7f82fc06e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:54.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.760+0000 7f8303ec8640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc119ff0 con 0x7f82fc10c8c0 2024-09-21T06:46:54.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.760+0000 7f8303ec8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc119e70 con 0x7f82fc10a490 2024-09-21T06:46:54.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.760+0000 7f8303ec8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc11a170 con 0x7f82fc10d210 2024-09-21T06:46:54.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.761+0000 7f8302465640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82fc10d210 0x7f82fc06e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:54.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.761+0000 7f8302465640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82fc10d210 0x7f82fc06e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.67:55186/0 (socket says 172.21.15.67:55186) 2024-09-21T06:46:54.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.761+0000 7f8302465640 1 -- 172.21.15.67:0/2960162233 learned_addr learned my addr 172.21.15.67:0/2960162233 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:46:54.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.762+0000 7f8301c64640 1 --2- 172.21.15.67:0/2960162233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82fc10a490 0x7f82fc074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:54.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.762+0000 7f8302465640 1 -- 172.21.15.67:0/2960162233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82fc10a490 msgr2=0x7f82fc074f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:54.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.762+0000 7f8302465640 1 --2- 172.21.15.67:0/2960162233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82fc10a490 0x7f82fc074f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.762+0000 7f8302465640 1 -- 172.21.15.67:0/2960162233 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82fc10c8c0 msgr2=0x7f82fc06e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:54.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.762+0000 7f8302465640 1 --2- 172.21.15.67:0/2960162233 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82fc10c8c0 0x7f82fc06e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.762+0000 7f8302465640 1 -- 172.21.15.67:0/2960162233 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82fc073b90 con 0x7f82fc10d210 2024-09-21T06:46:54.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.763+0000 7f8301c64640 1 --2- 172.21.15.67:0/2960162233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82fc10a490 0x7f82fc074f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:46:54.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.764+0000 7f8302465640 1 --2- 172.21.15.67:0/2960162233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82fc10d210 0x7f82fc06e870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00e970 tx=0x7f82ec00ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:46:54.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.767+0000 7f82faffd640 1 -- 172.21.15.67:0/2960162233 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec00ccc0 con 0x7f82fc10d210 2024-09-21T06:46:54.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.767+0000 7f8303ec8640 1 -- 172.21.15.67:0/2960162233 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82fc073e80 con 0x7f82fc10d210 2024-09-21T06:46:54.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.767+0000 7f8303ec8640 1 -- 172.21.15.67:0/2960162233 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82fc0743c0 con 0x7f82fc10d210 2024-09-21T06:46:54.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.767+0000 7f82faffd640 1 -- 172.21.15.67:0/2960162233 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82ec00ce60 con 0x7f82fc10d210 2024-09-21T06:46:54.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.767+0000 7f82faffd640 1 -- 172.21.15.67:0/2960162233 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec010430 con 0x7f82fc10d210 2024-09-21T06:46:54.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.773+0000 7f830a43c740 1 Processor -- start 2024-09-21T06:46:54.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.777+0000 7f82faffd640 1 -- 172.21.15.67:0/2960162233 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f82ec0046c0 con 0x7f82fc10d210 2024-09-21T06:46:54.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.778+0000 7f830a43c740 1 -- start start 2024-09-21T06:46:54.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.778+0000 7f830a43c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9e95a840 0x558e9ed01600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:54.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.778+0000 7f830a43c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e9e931980 0x558e9ed01b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:54.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.778+0000 7f830a43c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e9e939b60 0x558e9ed02080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:54.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.778+0000 7f830a43c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x558e9ec77b90 con 0x558e9e939b60 2024-09-21T06:46:54.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.778+0000 7f830a43c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558e9ec66fc0 con 0x558e9e95a840 2024-09-21T06:46:54.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.778+0000 7f830a43c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558e9ec6add0 con 0x558e9e931980 2024-09-21T06:46:54.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.779+0000 7f8301463640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9e95a840 0x558e9ed01600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:54.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.779+0000 7f8301463640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9e95a840 0x558e9ed01600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.67:43374/0 (socket says 172.21.15.67:43374) 2024-09-21T06:46:54.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.779+0000 7f8301463640 1 -- 172.21.15.67:0/451991201 learned_addr learned my addr 172.21.15.67:0/451991201 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:46:54.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.779+0000 7f8301463640 1 -- 172.21.15.67:0/451991201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e9e931980 msgr2=0x558e9ed01b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:46:54.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.779+0000 7f8301463640 1 --2- 172.21.15.67:0/451991201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e9e931980 0x558e9ed01b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.779+0000 7f8301463640 1 -- 172.21.15.67:0/451991201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e9e939b60 msgr2=0x558e9ed02080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:46:54.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.779+0000 7f8301463640 1 --2- 172.21.15.67:0/451991201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e9e939b60 0x558e9ed02080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.779+0000 7f8301463640 1 -- 172.21.15.67:0/451991201 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e9ebee060 con 0x558e9e95a840 2024-09-21T06:46:54.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.780+0000 7f82faffd640 1 --2- 172.21.15.67:0/2960162233 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f82e40b6650 0x7f82e40b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:54.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.780+0000 7f82faffd640 1 -- 172.21.15.67:0/2960162233 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f82ec0e7a40 con 0x7f82fc10d210 2024-09-21T06:46:54.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.780+0000 7f8301463640 1 --2- 172.21.15.67:0/451991201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9e95a840 0x558e9ed01600 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f82f4005fd0 tx=0x7f82f400e8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=855d068e2740e363 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:46:54.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.782+0000 7f82e3fff640 1 -- 172.21.15.67:0/451991201 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f4004b20 con 0x558e9e95a840 2024-09-21T06:46:54.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.782+0000 7f82e3fff640 1 -- 172.21.15.67:0/451991201 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82f4003e00 con 0x558e9e95a840 2024-09-21T06:46:54.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.782+0000 7f82e3fff640 1 -- 172.21.15.67:0/451991201 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f40047e0 con 0x558e9e95a840 2024-09-21T06:46:54.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.783+0000 7f8301c64640 1 --2- 172.21.15.67:0/2960162233 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f82e40b6650 0x7f82e40b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:54.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.783+0000 7f830a43c740 1 -- 172.21.15.67:0/451991201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9e95a840 msgr2=0x558e9ed01600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:54.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.783+0000 7f830a43c740 1 --2- 172.21.15.67:0/451991201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9e95a840 0x558e9ed01600 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f82f4005fd0 tx=0x7f82f400e8d0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.789+0000 7f830a43c740 1 -- 172.21.15.67:0/451991201 shutdown_connections 2024-09-21T06:46:54.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.790+0000 7f830a43c740 1 --2- 172.21.15.67:0/451991201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e9e939b60 0x558e9ed02080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.790+0000 7f830a43c740 1 --2- 172.21.15.67:0/451991201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e9e931980 0x558e9ed01b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.790+0000 7f830a43c740 1 --2- 172.21.15.67:0/451991201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9e95a840 0x558e9ed01600 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.790+0000 7f830a43c740 1 -- 172.21.15.67:0/451991201 >> 172.21.15.67:0/451991201 conn(0x558e9e94e990 msgr2=0x558e9ecff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:46:54.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.794+0000 7f830a43c740 1 -- 172.21.15.67:0/451991201 shutdown_connections 2024-09-21T06:46:54.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.798+0000 7f830a43c740 1 -- 172.21.15.67:0/451991201 wait complete. 2024-09-21T06:46:54.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.798+0000 7f830a43c740 1 -- start start 2024-09-21T06:46:54.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.798+0000 7f830a43c740 1 Processor -- start 2024-09-21T06:46:54.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.800+0000 7f830a43c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e9e931980 0x558e9ed0d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:54.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.800+0000 7f830a43c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9e939b60 0x558e9ed0d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:54.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.800+0000 7f830a43c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e9e95a840 0x558e9ed19160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:54.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.800+0000 7f830a43c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x558e9eac87a0 con 0x558e9e95a840 2024-09-21T06:46:54.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.800+0000 7f830a43c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558e9ec0c0a0 con 0x558e9e939b60 2024-09-21T06:46:54.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.800+0000 7f830a43c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558e9ed0fde0 con 0x558e9e931980 2024-09-21T06:46:54.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.800+0000 7f830a43c740 10 client.0 _ll_register_callbacks cb 0x558e9ebe93c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:46:54.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.801+0000 7f8301c64640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e9e931980 0x558e9ed0d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:54.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.801+0000 7f8301c64640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e9e931980 0x558e9ed0d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.67:55190/0 (socket says 172.21.15.67:55190) 2024-09-21T06:46:54.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.801+0000 7f8301c64640 1 -- 172.21.15.67:0/1180502652 learned_addr learned my addr 172.21.15.67:0/1180502652 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:46:54.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.801+0000 7f8301463640 1 --2- 172.21.15.67:0/1180502652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9e939b60 0x558e9ed0d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:54.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.802+0000 7f8302465640 1 --2- 172.21.15.67:0/1180502652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e9e95a840 0x558e9ed19160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:54.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.805+0000 7f8302465640 1 -- 172.21.15.67:0/1180502652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e9e931980 msgr2=0x558e9ed0d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:46:54.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.805+0000 7f8302465640 1 --2- 172.21.15.67:0/1180502652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e9e931980 0x558e9ed0d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.805+0000 7f8302465640 1 -- 172.21.15.67:0/1180502652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9e939b60 msgr2=0x558e9ed0d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:46:54.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.805+0000 7f8302465640 1 --2- 172.21.15.67:0/1180502652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9e939b60 0x558e9ed0d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.805+0000 7f8302465640 1 -- 172.21.15.67:0/1180502652 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e9ebfa880 con 0x558e9e95a840 2024-09-21T06:46:54.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.808+0000 7f8301c64640 1 --2- 172.21.15.67:0/2960162233 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f82e40b6650 0x7f82e40b8b60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f82f0000c00 tx=0x7f82f002cf70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:46:54.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.808+0000 7f8302465640 1 --2- 172.21.15.67:0/1180502652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e9e95a840 0x558e9ed19160 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00d010 tx=0x7f82ec0b70f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a224fd0bcc478e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:46:54.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.808+0000 7f82e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:46:54.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.808+0000 7f82e17fa640 1 -- 172.21.15.67:0/1180502652 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec0adea0 con 0x558e9e95a840 2024-09-21T06:46:54.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.808+0000 7f82e17fa640 1 -- 172.21.15.67:0/1180502652 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f82ec0aead0 con 0x558e9e95a840 2024-09-21T06:46:54.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.810+0000 7f830a43c740 1 -- 172.21.15.67:0/1180502652 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82f4003e00 con 0x558e9e95a840 2024-09-21T06:46:54.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.812+0000 7f82e17fa640 1 -- 172.21.15.67:0/1180502652 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f82ec0b2280 con 0x558e9e95a840 2024-09-21T06:46:54.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.812+0000 7f830a43c740 10 client.15876 fetch_fsmap learned FSMap version 21 2024-09-21T06:46:54.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.812+0000 7f830a43c740 1 -- 172.21.15.67:0/1180502652 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558e9e9d4b90 con 0x558e9e95a840 2024-09-21T06:46:54.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.813+0000 7f82e17fa640 1 -- 172.21.15.67:0/1180502652 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82f4003e00 con 0x558e9e95a840 2024-09-21T06:46:54.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.813+0000 7f82e17fa640 1 -- 172.21.15.67:0/1180502652 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f82ec0b35b0 con 0x558e9e95a840 2024-09-21T06:46:54.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.814+0000 7f830a43c740 10 client.15876 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:46:54.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.814+0000 7f830a43c740 10 client.15876 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:46:54.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.814+0000 7f830a43c740 20 client.15876 populate_metadata read hostname 'smithi067' 2024-09-21T06:46:54.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.814+0000 7f830a43c740 1 --2- 172.21.15.67:0/1180502652 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x558e9e9191c0 0x558e9ed1f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:54.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.814+0000 7f830a43c740 4 client.15876 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:46:54.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.814+0000 7f830a43c740 1 -- 172.21.15.67:0/1180502652 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e9ec79ae0 con 0x558e9e9191c0 2024-09-21T06:46:54.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.816+0000 7f8301c64640 1 --2- 172.21.15.67:0/1180502652 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x558e9e9191c0 0x558e9ed1f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:54.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.819+0000 7f8301c64640 1 --2- 172.21.15.67:0/1180502652 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x558e9e9191c0 0x558e9ed1f220 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0c5f1d443fb1bfa server_cookie=be2cd46c69abbd20 in_seq=0 out_seq=0 2024-09-21T06:46:54.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.820+0000 7f82e17fa640 10 client.15876 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:46:54.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.824+0000 7f82e17fa640 1 -- 172.21.15.67:0/1180502652 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558e9ec79ae0 con 0x558e9e9191c0 2024-09-21T06:46:54.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.824+0000 7f82e17fa640 10 client.15876 handle_command_reply: tid=0 2024-09-21T06:46:54.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.824+0000 7f830a43c740 1 client.15876 shutdown 2024-09-21T06:46:54.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.825+0000 7f830a43c740 1 -- 172.21.15.67:0/1180502652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e9e95a840 msgr2=0x558e9ed19160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:54.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.825+0000 7f830a43c740 1 --2- 172.21.15.67:0/1180502652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e9e95a840 0x558e9ed19160 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00d010 tx=0x7f82ec0b70f0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.826+0000 7f830a43c740 1 -- 172.21.15.67:0/1180502652 shutdown_connections 2024-09-21T06:46:54.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.826+0000 7f830a43c740 1 --2- 172.21.15.67:0/1180502652 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x558e9e9191c0 0x558e9ed1f220 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.826+0000 7f830a43c740 1 --2- 172.21.15.67:0/1180502652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9e939b60 0x558e9ed0d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.826+0000 7f830a43c740 1 --2- 172.21.15.67:0/1180502652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e9e95a840 0x558e9ed19160 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.826+0000 7f830a43c740 1 --2- 172.21.15.67:0/1180502652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e9e931980 0x558e9ed0d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.826+0000 7f830a43c740 1 -- 172.21.15.67:0/1180502652 >> 172.21.15.67:0/1180502652 conn(0x558e9e94e990 msgr2=0x558e9ed04790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:46:54.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.826+0000 7f82e17fa640 0 client.15876 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:46:54.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.826+0000 7f830a43c740 1 -- 172.21.15.67:0/1180502652 shutdown_connections 2024-09-21T06:46:54.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.827+0000 7f830a43c740 1 -- 172.21.15.67:0/1180502652 wait complete. 2024-09-21T06:46:54.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.830+0000 7f830a43c740 20 client.15876 trim_cache size 0 max 16384 2024-09-21T06:46:54.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.867+0000 7f830a43c740 1 Processor -- start 2024-09-21T06:46:54.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.867+0000 7f830a43c740 1 -- start start 2024-09-21T06:46:54.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.867+0000 7f830a43c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e9e931980 0x558e9ed02d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:54.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.868+0000 7f830a43c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e9e939b60 0x558e9ed0a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:54.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.868+0000 7f830a43c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9e95a840 0x558e9ed0a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:54.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.868+0000 7f830a43c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f82e800dbf0 con 0x558e9e939b60 2024-09-21T06:46:54.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.868+0000 7f830a43c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f82e8010470 con 0x558e9e95a840 2024-09-21T06:46:54.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.868+0000 7f830a43c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f82e800e260 con 0x558e9e931980 2024-09-21T06:46:54.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.870+0000 7f8301c64640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e9e931980 0x558e9ed02d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:54.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.870+0000 7f8301c64640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e9e931980 0x558e9ed02d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.67:55194/0 (socket says 172.21.15.67:55194) 2024-09-21T06:46:54.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.870+0000 7f8301c64640 1 -- 172.21.15.67:0/2492150414 learned_addr learned my addr 172.21.15.67:0/2492150414 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:46:54.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.871+0000 7f8301c64640 1 -- 172.21.15.67:0/2492150414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9e95a840 msgr2=0x558e9ed0a6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:46:54.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.871+0000 7f8301c64640 1 --2- 172.21.15.67:0/2492150414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9e95a840 0x558e9ed0a6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.871+0000 7f8301c64640 1 -- 172.21.15.67:0/2492150414 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e9e939b60 msgr2=0x558e9ed0a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:46:54.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.871+0000 7f8301c64640 1 --2- 172.21.15.67:0/2492150414 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e9e939b60 0x558e9ed0a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.871+0000 7f8301c64640 1 -- 172.21.15.67:0/2492150414 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e9ed04460 con 0x558e9e931980 2024-09-21T06:46:54.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.872+0000 7f8301c64640 1 --2- 172.21.15.67:0/2492150414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e9e931980 0x558e9ed02d40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f82f002a9a0 tx=0x7f82f003f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0e87f44dc587fb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:46:54.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.874+0000 7f82e37fe640 1 -- 172.21.15.67:0/2492150414 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f0037040 con 0x558e9e931980 2024-09-21T06:46:54.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.874+0000 7f82e37fe640 1 -- 172.21.15.67:0/2492150414 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82f0042040 con 0x558e9e931980 2024-09-21T06:46:54.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.874+0000 7f82e37fe640 1 -- 172.21.15.67:0/2492150414 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f00035f0 con 0x558e9e931980 2024-09-21T06:46:54.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.875+0000 7f830a43c740 1 -- 172.21.15.67:0/2492150414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e9e931980 msgr2=0x558e9ed02d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:54.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.875+0000 7f830a43c740 1 --2- 172.21.15.67:0/2492150414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e9e931980 0x558e9ed02d40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f82f002a9a0 tx=0x7f82f003f000 comp rx=0 tx=0).stop 2024-09-21T06:46:54.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.877+0000 7f830a43c740 1 -- 172.21.15.67:0/2492150414 shutdown_connections 2024-09-21T06:46:54.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.877+0000 7f830a43c740 1 --2- 172.21.15.67:0/2492150414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9e95a840 0x558e9ed0a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.877+0000 7f830a43c740 1 --2- 172.21.15.67:0/2492150414 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e9e939b60 0x558e9ed0a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.877+0000 7f830a43c740 1 --2- 172.21.15.67:0/2492150414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e9e931980 0x558e9ed02d40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.877+0000 7f830a43c740 1 -- 172.21.15.67:0/2492150414 >> 172.21.15.67:0/2492150414 conn(0x558e9e94e990 msgr2=0x558e9ed02630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:46:54.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.877+0000 7f830a43c740 1 -- 172.21.15.67:0/2492150414 shutdown_connections 2024-09-21T06:46:54.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.879+0000 7f830a43c740 1 -- 172.21.15.67:0/2492150414 wait complete. 2024-09-21T06:46:54.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.879+0000 7f830a43c740 1 -- start start 2024-09-21T06:46:54.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.880+0000 7f830a43c740 1 Processor -- start 2024-09-21T06:46:54.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.880+0000 7f830a43c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e9e931980 0x558e9ed15650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:54.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.880+0000 7f830a43c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9e939b60 0x558e9ed12b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:54.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.880+0000 7f830a43c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e9e95a840 0x558e9ed13060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:54.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.880+0000 7f830a43c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f82e800e930 con 0x558e9e931980 2024-09-21T06:46:54.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.880+0000 7f830a43c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f82e800fda0 con 0x558e9e939b60 2024-09-21T06:46:54.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.880+0000 7f830a43c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f82e800f000 con 0x558e9e95a840 2024-09-21T06:46:54.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.880+0000 7f830a43c740 10 client.0 _ll_register_callbacks cb 0x7f82ec009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:46:54.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.881+0000 7f8301463640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9e939b60 0x558e9ed12b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:54.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.881+0000 7f8302465640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e9e95a840 0x558e9ed13060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:54.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.881+0000 7f8301463640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9e939b60 0x558e9ed12b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.67:43390/0 (socket says 172.21.15.67:43390) 2024-09-21T06:46:54.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.881+0000 7f8301463640 1 -- 172.21.15.67:0/1768737301 learned_addr learned my addr 172.21.15.67:0/1768737301 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:46:54.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.881+0000 7f8301c64640 1 --2- 172.21.15.67:0/1768737301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e9e931980 0x558e9ed15650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:54.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.882+0000 7f8301463640 1 -- 172.21.15.67:0/1768737301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e9e95a840 msgr2=0x558e9ed13060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:46:54.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.882+0000 7f8301463640 1 --2- 172.21.15.67:0/1768737301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e9e95a840 0x558e9ed13060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.882+0000 7f8301463640 1 -- 172.21.15.67:0/1768737301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e9e931980 msgr2=0x558e9ed15650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:46:54.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.882+0000 7f8301463640 1 --2- 172.21.15.67:0/1768737301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e9e931980 0x558e9ed15650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.882+0000 7f8301463640 1 -- 172.21.15.67:0/1768737301 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e9ed135a0 con 0x558e9e939b60 2024-09-21T06:46:54.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.884+0000 7f8301463640 1 --2- 172.21.15.67:0/1768737301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9e939b60 0x558e9ed12b20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f82f400efd0 tx=0x7f82f4004de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=884871dd773799dd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:46:54.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.884+0000 7f82e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-21T06:46:54.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.884+0000 7f82e17fa640 1 -- 172.21.15.67:0/1768737301 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f401b070 con 0x558e9e939b60 2024-09-21T06:46:54.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.884+0000 7f82e17fa640 1 -- 172.21.15.67:0/1768737301 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f82f4004700 con 0x558e9e939b60 2024-09-21T06:46:54.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.888+0000 7f830a43c740 1 -- 172.21.15.67:0/1768737301 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82f0042040 con 0x558e9e939b60 2024-09-21T06:46:54.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.890+0000 7f82e17fa640 1 -- 172.21.15.67:0/1768737301 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f82f4003970 con 0x558e9e939b60 2024-09-21T06:46:54.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.892+0000 7f830a43c740 10 client.25586 fetch_fsmap learned FSMap version 21 2024-09-21T06:46:54.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.892+0000 7f830a43c740 1 -- 172.21.15.67:0/1768737301 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558e9ed8b8d0 con 0x558e9e939b60 2024-09-21T06:46:54.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.894+0000 7f82e17fa640 1 -- 172.21.15.67:0/1768737301 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82f0042040 con 0x558e9e939b60 2024-09-21T06:46:54.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.894+0000 7f82e17fa640 1 -- 172.21.15.67:0/1768737301 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f82f400b050 con 0x558e9e939b60 2024-09-21T06:46:54.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.894+0000 7f830a43c740 10 client.25586 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:46:54.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.894+0000 7f830a43c740 10 client.25586 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:46:54.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.894+0000 7f830a43c740 20 client.25586 populate_metadata read hostname 'smithi067' 2024-09-21T06:46:54.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.894+0000 7f830a43c740 1 --2- 172.21.15.67:0/1768737301 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x558e9e8ee570 0x558e9ed8e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:54.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.894+0000 7f830a43c740 4 client.25586 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}] 2024-09-21T06:46:54.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.894+0000 7f830a43c740 1 -- 172.21.15.67:0/1768737301 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}) -- 0x558e9ec75590 con 0x558e9e8ee570 2024-09-21T06:46:54.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.894+0000 7f8301c64640 1 --2- 172.21.15.67:0/1768737301 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x558e9e8ee570 0x558e9ed8e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:54.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.900+0000 7f8301c64640 1 --2- 172.21.15.67:0/1768737301 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x558e9e8ee570 0x558e9ed8e080 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c97487ecdd5a958 server_cookie=eed9ad90eec1260d in_seq=0 out_seq=0 2024-09-21T06:46:54.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.900+0000 7f82e17fa640 10 client.25586 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:46:54.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.941+0000 7f82e17fa640 1 -- 172.21.15.67:0/1768737301 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558e9ec75590 con 0x558e9e8ee570 2024-09-21T06:46:54.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.941+0000 7f82e17fa640 10 client.25586 handle_command_reply: tid=0 2024-09-21T06:46:54.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.941+0000 7f830a43c740 1 client.25586 shutdown 2024-09-21T06:46:54.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.942+0000 7f830a43c740 1 -- 172.21.15.67:0/1768737301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9e939b60 msgr2=0x558e9ed12b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:54.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.942+0000 7f830a43c740 1 --2- 172.21.15.67:0/1768737301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9e939b60 0x558e9ed12b20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f82f400efd0 tx=0x7f82f4004de0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.942+0000 7f830a43c740 1 -- 172.21.15.67:0/1768737301 shutdown_connections 2024-09-21T06:46:54.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.942+0000 7f830a43c740 1 --2- 172.21.15.67:0/1768737301 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x558e9e8ee570 0x558e9ed8e080 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.942+0000 7f830a43c740 1 --2- 172.21.15.67:0/1768737301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9e939b60 0x558e9ed12b20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.942+0000 7f830a43c740 1 --2- 172.21.15.67:0/1768737301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e9e95a840 0x558e9ed13060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.942+0000 7f830a43c740 1 --2- 172.21.15.67:0/1768737301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e9e931980 0x558e9ed15650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.942+0000 7f830a43c740 1 -- 172.21.15.67:0/1768737301 >> 172.21.15.67:0/1768737301 conn(0x558e9e94e990 msgr2=0x558e9ed04700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:46:54.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.943+0000 7f82e17fa640 0 client.25586 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:46:54.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.943+0000 7f830a43c740 1 -- 172.21.15.67:0/1768737301 shutdown_connections 2024-09-21T06:46:54.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.943+0000 7f830a43c740 1 -- 172.21.15.67:0/1768737301 wait complete. 2024-09-21T06:46:54.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.943+0000 7f830a43c740 20 client.25586 trim_cache size 0 max 16384 2024-09-21T06:46:54.946 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-21T06:46:54.946 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 21, 2024-09-21T06:46:54.947 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-09-21T06:46:54.947 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 42, 2024-09-21T06:46:54.947 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-21T06:46:54.947 INFO:teuthology.orchestra.run.smithi067.stdout: "6a9bb4bb": { 2024-09-21T06:46:54.947 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 42, 2024-09-21T06:46:54.947 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-21T06:46:54.947 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:46:54.947 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-21T06:46:54.947 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:46:54.947 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:46:54.947 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-21T06:46:54.948 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 144.5, 2024-09-21T06:46:54.948 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-21T06:46:54.948 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-21T06:46:54.949 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-21T06:46:54.949 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:46:54.950 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-21T06:46:54.950 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:46:54.950 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:46:54.950 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:46:54.950 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:46:54.950 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:46:54.950 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:46:54.950 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-21T06:46:54.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.944+0000 7f8303ec8640 1 -- 172.21.15.67:0/2960162233 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f82e40b6650 msgr2=0x7f82e40b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:54.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.944+0000 7f8303ec8640 1 --2- 172.21.15.67:0/2960162233 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f82e40b6650 0x7f82e40b8b60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f82f0000c00 tx=0x7f82f002cf70 comp rx=0 tx=0).stop 2024-09-21T06:46:54.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.945+0000 7f8303ec8640 1 -- 172.21.15.67:0/2960162233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82fc10d210 msgr2=0x7f82fc06e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:54.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.945+0000 7f8303ec8640 1 --2- 172.21.15.67:0/2960162233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82fc10d210 0x7f82fc06e870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00e970 tx=0x7f82ec00ed70 comp rx=0 tx=0).stop 2024-09-21T06:46:54.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.945+0000 7f8303ec8640 1 -- 172.21.15.67:0/2960162233 shutdown_connections 2024-09-21T06:46:54.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.945+0000 7f8303ec8640 1 --2- 172.21.15.67:0/2960162233 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f82e40b6650 0x7f82e40b8b60 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.945+0000 7f8303ec8640 1 --2- 172.21.15.67:0/2960162233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82fc10d210 0x7f82fc06e870 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.945+0000 7f8303ec8640 1 --2- 172.21.15.67:0/2960162233 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82fc10c8c0 0x7f82fc06e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.945+0000 7f8303ec8640 1 --2- 172.21.15.67:0/2960162233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82fc10a490 0x7f82fc074f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:54.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.945+0000 7f8303ec8640 1 -- 172.21.15.67:0/2960162233 >> 172.21.15.67:0/2960162233 conn(0x7f82fc0795f0 msgr2=0x7f82fc110420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:46:54.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.945+0000 7f8303ec8640 1 -- 172.21.15.67:0/2960162233 shutdown_connections 2024-09-21T06:46:54.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:46:54.945+0000 7f8303ec8640 1 -- 172.21.15.67:0/2960162233 wait complete. 2024-09-21T06:46:55.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:55 smithi100 ceph-mon[33401]: pgmap v670: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.9 MiB/s wr, 115 op/s 2024-09-21T06:46:55.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:55 smithi203 ceph-mon[33735]: pgmap v670: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.9 MiB/s wr, 115 op/s 2024-09-21T06:46:55.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:55 smithi067 ceph-mon[31232]: pgmap v670: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.9 MiB/s wr, 115 op/s 2024-09-21T06:46:55.897 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6a9bb4bb', quiesce took 5.2 seconds. Will release after: 19.3 2024-09-21T06:46:57.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:57 smithi203 ceph-mon[33735]: pgmap v671: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.2 MiB/s wr, 168 op/s 2024-09-21T06:46:57.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:57 smithi067 ceph-mon[31232]: pgmap v671: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.2 MiB/s wr, 168 op/s 2024-09-21T06:46:57.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:57 smithi100 ceph-mon[33401]: pgmap v671: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.2 MiB/s wr, 168 op/s 2024-09-21T06:46:59.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:46:59 smithi203 ceph-mon[33735]: pgmap v672: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 13 MiB/s wr, 295 op/s 2024-09-21T06:46:59.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:46:59 smithi067 ceph-mon[31232]: pgmap v672: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 13 MiB/s wr, 295 op/s 2024-09-21T06:46:59.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:46:59 smithi100 ceph-mon[33401]: pgmap v672: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 13 MiB/s wr, 295 op/s 2024-09-21T06:47:01.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:01 smithi203 ceph-mon[33735]: pgmap v673: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 13 MiB/s wr, 296 op/s 2024-09-21T06:47:01.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:01 smithi067 ceph-mon[31232]: pgmap v673: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 13 MiB/s wr, 296 op/s 2024-09-21T06:47:01.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:01 smithi100 ceph-mon[33401]: pgmap v673: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 13 MiB/s wr, 296 op/s 2024-09-21T06:47:03.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:03 smithi203 ceph-mon[33735]: pgmap v674: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 13 MiB/s wr, 303 op/s 2024-09-21T06:47:03.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:03 smithi067 ceph-mon[31232]: pgmap v674: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 13 MiB/s wr, 303 op/s 2024-09-21T06:47:03.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:03 smithi100 ceph-mon[33401]: pgmap v674: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 13 MiB/s wr, 303 op/s 2024-09-21T06:47:05.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:05 smithi203 ceph-mon[33735]: pgmap v675: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 11 MiB/s wr, 239 op/s 2024-09-21T06:47:05.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:05 smithi067 ceph-mon[31232]: pgmap v675: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 11 MiB/s wr, 239 op/s 2024-09-21T06:47:05.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:05 smithi100 ceph-mon[33401]: pgmap v675: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 11 MiB/s wr, 239 op/s 2024-09-21T06:47:07.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:07 smithi203 ceph-mon[33735]: pgmap v676: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.7 MiB/s wr, 194 op/s 2024-09-21T06:47:07.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:07 smithi067 ceph-mon[31232]: pgmap v676: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.7 MiB/s wr, 194 op/s 2024-09-21T06:47:07.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:07 smithi100 ceph-mon[33401]: pgmap v676: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.7 MiB/s wr, 194 op/s 2024-09-21T06:47:09.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:09 smithi203 ceph-mon[33735]: pgmap v677: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.5 MiB/s wr, 140 op/s 2024-09-21T06:47:09.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:09 smithi067 ceph-mon[31232]: pgmap v677: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.5 MiB/s wr, 140 op/s 2024-09-21T06:47:09.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:09 smithi100 ceph-mon[33401]: pgmap v677: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.5 MiB/s wr, 140 op/s 2024-09-21T06:47:11.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:11 smithi203 ceph-mon[33735]: pgmap v678: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 683 KiB/s wr, 12 op/s 2024-09-21T06:47:11.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:11 smithi067 ceph-mon[31232]: pgmap v678: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 683 KiB/s wr, 12 op/s 2024-09-21T06:47:11.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:11 smithi100 ceph-mon[33401]: pgmap v678: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 683 KiB/s wr, 12 op/s 2024-09-21T06:47:13.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:13 smithi203 ceph-mon[33735]: pgmap v679: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 651 KiB/s wr, 12 op/s 2024-09-21T06:47:13.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:13 smithi067 ceph-mon[31232]: pgmap v679: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 651 KiB/s wr, 12 op/s 2024-09-21T06:47:13.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:13 smithi100 ceph-mon[33401]: pgmap v679: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 651 KiB/s wr, 12 op/s 2024-09-21T06:47:15.201 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:47:15.478 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:47:15.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:15 smithi067 ceph-mon[31232]: pgmap v680: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 225 KiB/s wr, 5 op/s 2024-09-21T06:47:15.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:15 smithi203 ceph-mon[33735]: pgmap v680: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 225 KiB/s wr, 5 op/s 2024-09-21T06:47:15.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:15 smithi100 ceph-mon[33401]: pgmap v680: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 225 KiB/s wr, 5 op/s 2024-09-21T06:47:16.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.994+0000 7f40bd173640 1 -- 172.21.15.67:0/4030218602 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40b8109e10 msgr2=0x7f40b8110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:47:16.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.995+0000 7f40bd173640 1 --2- 172.21.15.67:0/4030218602 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40b8109e10 0x7f40b8110640 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f40b0009ca0 tx=0x7f40b002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:47:16.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.995+0000 7f40bd173640 1 -- 172.21.15.67:0/4030218602 shutdown_connections 2024-09-21T06:47:16.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.995+0000 7f40bd173640 1 --2- 172.21.15.67:0/4030218602 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40b8109e10 0x7f40b8110640 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:16.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.995+0000 7f40bd173640 1 --2- 172.21.15.67:0/4030218602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40b81094c0 0x7f40b81098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:16.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.995+0000 7f40bd173640 1 --2- 172.21.15.67:0/4030218602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b8103b80 0x7f40b8103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:16.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.995+0000 7f40bd173640 1 -- 172.21.15.67:0/4030218602 >> 172.21.15.67:0/4030218602 conn(0x7f40b80ff5d0 msgr2=0x7f40b8101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:47:16.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.995+0000 7f40bd173640 1 -- 172.21.15.67:0/4030218602 shutdown_connections 2024-09-21T06:47:16.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.995+0000 7f40bd173640 1 -- 172.21.15.67:0/4030218602 wait complete. 2024-09-21T06:47:16.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.995+0000 7f40bd173640 1 Processor -- start 2024-09-21T06:47:16.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.996+0000 7f40bd173640 1 -- start start 2024-09-21T06:47:16.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.996+0000 7f40bd173640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40b8103b80 0x7f40b819c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:16.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.996+0000 7f40bd173640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b81094c0 0x7f40b819cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:16.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.996+0000 7f40bd173640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40b8109e10 0x7f40b81a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:16.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.996+0000 7f40bd173640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f40b81139d0 con 0x7f40b8109e10 2024-09-21T06:47:16.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.996+0000 7f40bd173640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f40b8113850 con 0x7f40b81094c0 2024-09-21T06:47:16.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.996+0000 7f40bd173640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f40b8113b50 con 0x7f40b8103b80 2024-09-21T06:47:16.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.996+0000 7f40b759e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40b8109e10 0x7f40b81a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:16.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.996+0000 7f40b759e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40b8109e10 0x7f40b81a1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47614/0 (socket says 172.21.15.67:47614) 2024-09-21T06:47:16.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.996+0000 7f40b759e640 1 -- 172.21.15.67:0/4276475596 learned_addr learned my addr 172.21.15.67:0/4276475596 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:47:16.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.996+0000 7f40ae59c640 1 --2- 172.21.15.67:0/4276475596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b81094c0 0x7f40b819cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:16.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.996+0000 7f40b6d9d640 1 --2- 172.21.15.67:0/4276475596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40b8103b80 0x7f40b819c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:16.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.997+0000 7f40b759e640 1 -- 172.21.15.67:0/4276475596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40b8103b80 msgr2=0x7f40b819c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:47:16.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.997+0000 7f40b759e640 1 --2- 172.21.15.67:0/4276475596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40b8103b80 0x7f40b819c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:16.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.997+0000 7f40b759e640 1 -- 172.21.15.67:0/4276475596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b81094c0 msgr2=0x7f40b819cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:47:16.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.997+0000 7f40b759e640 1 --2- 172.21.15.67:0/4276475596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b81094c0 0x7f40b819cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:16.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.997+0000 7f40b759e640 1 -- 172.21.15.67:0/4276475596 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b81a2410 con 0x7f40b8109e10 2024-09-21T06:47:16.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.997+0000 7f40b759e640 1 --2- 172.21.15.67:0/4276475596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40b8109e10 0x7f40b81a1da0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f40b0005cc0 tx=0x7f40b0032c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:47:16.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.997+0000 7f40b4d99640 1 -- 172.21.15.67:0/4276475596 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0004040 con 0x7f40b8109e10 2024-09-21T06:47:16.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.997+0000 7f40bd173640 1 -- 172.21.15.67:0/4276475596 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b81a26a0 con 0x7f40b8109e10 2024-09-21T06:47:16.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.997+0000 7f40b4d99640 1 -- 172.21.15.67:0/4276475596 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f40b0004670 con 0x7f40b8109e10 2024-09-21T06:47:16.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.997+0000 7f40b4d99640 1 -- 172.21.15.67:0/4276475596 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0030cc0 con 0x7f40b8109e10 2024-09-21T06:47:16.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.997+0000 7f40bd173640 1 -- 172.21.15.67:0/4276475596 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b81a8f20 con 0x7f40b8109e10 2024-09-21T06:47:16.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:15.998+0000 7f40bd173640 1 -- 172.21.15.67:0/4276475596 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40b8103f90 con 0x7f40b8109e10 2024-09-21T06:47:16.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:16.000+0000 7f40b4d99640 1 -- 172.21.15.67:0/4276475596 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f40b00041e0 con 0x7f40b8109e10 2024-09-21T06:47:16.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:16.001+0000 7f40b4d99640 1 --2- 172.21.15.67:0/4276475596 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f40a40b6540 0x7f40a40b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:16.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:16.001+0000 7f40b6d9d640 1 --2- 172.21.15.67:0/4276475596 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f40a40b6540 0x7f40a40b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:16.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:16.001+0000 7f40b4d99640 1 -- 172.21.15.67:0/4276475596 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f40b0108c80 con 0x7f40b8109e10 2024-09-21T06:47:16.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:16.001+0000 7f40b6d9d640 1 --2- 172.21.15.67:0/4276475596 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f40a40b6540 0x7f40a40b8a50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f40a8005e10 tx=0x7f40a8005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:47:16.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:16.004+0000 7f40b4d99640 1 -- 172.21.15.67:0/4276475596 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f40b00cf140 con 0x7f40b8109e10 2024-09-21T06:47:16.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:16.235+0000 7f40bd173640 1 -- 172.21.15.67:0/4276475596 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40b819d690 con 0x7f40b8109e10 2024-09-21T06:47:16.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:16.236+0000 7f40b4d99640 1 -- 172.21.15.67:0/4276475596 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7f40b819d690 con 0x7f40b8109e10 2024-09-21T06:47:16.238 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:47:16.238 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:47:16.238 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:47:16.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:16.239+0000 7f40bd173640 1 -- 172.21.15.67:0/4276475596 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f40a40b6540 msgr2=0x7f40a40b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:47:16.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:16.239+0000 7f40bd173640 1 --2- 172.21.15.67:0/4276475596 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f40a40b6540 0x7f40a40b8a50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f40a8005e10 tx=0x7f40a8005da0 comp rx=0 tx=0).stop 2024-09-21T06:47:16.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:16.239+0000 7f40bd173640 1 -- 172.21.15.67:0/4276475596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40b8109e10 msgr2=0x7f40b81a1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:47:16.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:16.239+0000 7f40bd173640 1 --2- 172.21.15.67:0/4276475596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40b8109e10 0x7f40b81a1da0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f40b0005cc0 tx=0x7f40b0032c40 comp rx=0 tx=0).stop 2024-09-21T06:47:16.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:16.239+0000 7f40bd173640 1 -- 172.21.15.67:0/4276475596 shutdown_connections 2024-09-21T06:47:16.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:16.239+0000 7f40bd173640 1 --2- 172.21.15.67:0/4276475596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b81094c0 0x7f40b819cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:16.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:16.239+0000 7f40bd173640 1 --2- 172.21.15.67:0/4276475596 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f40a40b6540 0x7f40a40b8a50 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:16.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:16.239+0000 7f40bd173640 1 --2- 172.21.15.67:0/4276475596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40b8109e10 0x7f40b81a1da0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:16.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:16.239+0000 7f40bd173640 1 --2- 172.21.15.67:0/4276475596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40b8103b80 0x7f40b819c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:16.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:16.239+0000 7f40bd173640 1 -- 172.21.15.67:0/4276475596 >> 172.21.15.67:0/4276475596 conn(0x7f40b80ff5d0 msgr2=0x7f40b8108b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:47:16.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:16.239+0000 7f40bd173640 1 -- 172.21.15.67:0/4276475596 shutdown_connections 2024-09-21T06:47:16.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:16.239+0000 7f40bd173640 1 -- 172.21.15.67:0/4276475596 wait complete. 2024-09-21T06:47:16.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:16 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/4276475596' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:47:16.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:16 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/4276475596' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:47:16.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:16 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/4276475596' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:47:17.015 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db --set-id 6a9bb4bb --release --await' 2024-09-21T06:47:17.015 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24671 quiesce db --set-id 6a9bb4bb --release --await 2024-09-21T06:47:17.277 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:47:17.673 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:17 smithi067 ceph-mon[31232]: pgmap v681: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-21T06:47:17.705 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:17 smithi100 ceph-mon[33401]: pgmap v681: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-21T06:47:17.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:17 smithi203 ceph-mon[33735]: pgmap v681: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-21T06:47:17.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.770+0000 7f14a56fe640 1 -- 172.21.15.67:0/3087124057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14a0108d20 msgr2=0x7f14a010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:47:17.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.770+0000 7f14a56fe640 1 --2- 172.21.15.67:0/3087124057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14a0108d20 0x7f14a010f550 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1498009ca0 tx=0x7f149802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.771+0000 7f14a56fe640 1 -- 172.21.15.67:0/3087124057 shutdown_connections 2024-09-21T06:47:17.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.771+0000 7f14a56fe640 1 --2- 172.21.15.67:0/3087124057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14a0108d20 0x7f14a010f550 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.771+0000 7f14a56fe640 1 --2- 172.21.15.67:0/3087124057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14a01083d0 0x7f14a01087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.771+0000 7f14a56fe640 1 --2- 172.21.15.67:0/3087124057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14a0102ae0 0x7f14a0102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.771+0000 7f14a56fe640 1 -- 172.21.15.67:0/3087124057 >> 172.21.15.67:0/3087124057 conn(0x7f14a0079760 msgr2=0x7f14a0079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:47:17.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.771+0000 7f14a56fe640 1 -- 172.21.15.67:0/3087124057 shutdown_connections 2024-09-21T06:47:17.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.771+0000 7f14a56fe640 1 -- 172.21.15.67:0/3087124057 wait complete. 2024-09-21T06:47:17.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.771+0000 7f14a56fe640 1 Processor -- start 2024-09-21T06:47:17.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.771+0000 7f14a56fe640 1 -- start start 2024-09-21T06:47:17.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.772+0000 7f14a56fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14a0102ae0 0x7f14a0105110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:17.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.772+0000 7f14a56fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14a01083d0 0x7f14a0105650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:17.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.772+0000 7f14a56fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14a0108d20 0x7f14a0105b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:17.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.772+0000 7f14a56fe640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f14a01128f0 con 0x7f14a01083d0 2024-09-21T06:47:17.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.772+0000 7f14a56fe640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f14a0112770 con 0x7f14a0102ae0 2024-09-21T06:47:17.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.772+0000 7f14a56fe640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f14a0112a70 con 0x7f14a0108d20 2024-09-21T06:47:17.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.772+0000 7f1497fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14a01083d0 0x7f14a0105650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:17.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.772+0000 7f1497fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14a01083d0 0x7f14a0105650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35818/0 (socket says 172.21.15.67:35818) 2024-09-21T06:47:17.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.772+0000 7f1497fff640 1 -- 172.21.15.67:0/406883524 learned_addr learned my addr 172.21.15.67:0/406883524 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:47:17.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.772+0000 7f149effd640 1 --2- 172.21.15.67:0/406883524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14a0102ae0 0x7f14a0105110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:17.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.772+0000 7f149f7fe640 1 --2- 172.21.15.67:0/406883524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14a0108d20 0x7f14a0105b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:17.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.772+0000 7f1497fff640 1 -- 172.21.15.67:0/406883524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14a0108d20 msgr2=0x7f14a0105b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:47:17.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.772+0000 7f1497fff640 1 --2- 172.21.15.67:0/406883524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14a0108d20 0x7f14a0105b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.772+0000 7f1497fff640 1 -- 172.21.15.67:0/406883524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14a0102ae0 msgr2=0x7f14a0105110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:47:17.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.773+0000 7f1497fff640 1 --2- 172.21.15.67:0/406883524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14a0102ae0 0x7f14a0105110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.773+0000 7f1497fff640 1 -- 172.21.15.67:0/406883524 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a01a8680 con 0x7f14a01083d0 2024-09-21T06:47:17.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.773+0000 7f1497fff640 1 --2- 172.21.15.67:0/406883524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14a01083d0 0x7f14a0105650 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f148800b450 tx=0x7f148800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:47:17.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.773+0000 7f149cff9640 1 -- 172.21.15.67:0/406883524 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1488015020 con 0x7f14a01083d0 2024-09-21T06:47:17.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.773+0000 7f149cff9640 1 -- 172.21.15.67:0/406883524 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1488011d20 con 0x7f14a01083d0 2024-09-21T06:47:17.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.773+0000 7f14a56fe640 1 -- 172.21.15.67:0/406883524 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14a01a8970 con 0x7f14a01083d0 2024-09-21T06:47:17.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.773+0000 7f149cff9640 1 -- 172.21.15.67:0/406883524 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1488010e10 con 0x7f14a01083d0 2024-09-21T06:47:17.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.773+0000 7f14a56fe640 1 -- 172.21.15.67:0/406883524 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14a01a8eb0 con 0x7f14a01083d0 2024-09-21T06:47:17.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.776+0000 7f14abc72740 1 Processor -- start 2024-09-21T06:47:17.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.776+0000 7f14abc72740 1 -- start start 2024-09-21T06:47:17.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.776+0000 7f149cff9640 1 -- 172.21.15.67:0/406883524 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f1488018020 con 0x7f14a01083d0 2024-09-21T06:47:17.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.776+0000 7f14abc72740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d34d64d520 0x55d34d9f3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:17.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.776+0000 7f14abc72740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d34d624660 0x55d34d9f3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:17.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.776+0000 7f14abc72740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d34d62c840 0x55d34d9f3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:17.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.776+0000 7f14abc72740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d34d8dedc0 con 0x55d34d64d520 2024-09-21T06:47:17.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.776+0000 7f14abc72740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d34d96b1f0 con 0x55d34d62c840 2024-09-21T06:47:17.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.776+0000 7f14abc72740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d34d95a620 con 0x55d34d624660 2024-09-21T06:47:17.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.777+0000 7f149f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d34d64d520 0x55d34d9f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:17.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.777+0000 7f149f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d34d64d520 0x55d34d9f3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35822/0 (socket says 172.21.15.67:35822) 2024-09-21T06:47:17.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.777+0000 7f149f7fe640 1 -- 172.21.15.67:0/1246420703 learned_addr learned my addr 172.21.15.67:0/1246420703 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:47:17.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.777+0000 7f1497fff640 1 --2- 172.21.15.67:0/1246420703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d34d62c840 0x55d34d9f3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:17.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.777+0000 7f149f7fe640 1 -- 172.21.15.67:0/1246420703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d34d624660 msgr2=0x55d34d9f3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:47:17.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.777+0000 7f149f7fe640 1 --2- 172.21.15.67:0/1246420703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d34d624660 0x55d34d9f3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.777+0000 7f149f7fe640 1 -- 172.21.15.67:0/1246420703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d34d62c840 msgr2=0x55d34d9f3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:47:17.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.777+0000 7f149f7fe640 1 --2- 172.21.15.67:0/1246420703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d34d62c840 0x55d34d9f3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.777+0000 7f149f7fe640 1 -- 172.21.15.67:0/1246420703 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d34d922fd0 con 0x55d34d64d520 2024-09-21T06:47:17.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.777+0000 7f1497fff640 1 --2- 172.21.15.67:0/1246420703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d34d62c840 0x55d34d9f3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:47:17.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.777+0000 7f149cff9640 1 --2- 172.21.15.67:0/406883524 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f148c0b6510 0x7f148c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:17.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.777+0000 7f149f7fe640 1 --2- 172.21.15.67:0/1246420703 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d34d64d520 0x55d34d9f3510 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1498035000 tx=0x7f1498009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b1c48e74f8837 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:47:17.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.777+0000 7f149effd640 1 --2- 172.21.15.67:0/406883524 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f148c0b6510 0x7f148c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:17.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.777+0000 7f149cff9640 1 -- 172.21.15.67:0/406883524 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f14880e7100 con 0x7f14a01083d0 2024-09-21T06:47:17.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.778+0000 7f14957fa640 1 -- 172.21.15.67:0/1246420703 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1498031980 con 0x55d34d64d520 2024-09-21T06:47:17.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.778+0000 7f14957fa640 1 -- 172.21.15.67:0/1246420703 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1498036040 con 0x55d34d64d520 2024-09-21T06:47:17.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.778+0000 7f14957fa640 1 -- 172.21.15.67:0/1246420703 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14980044a0 con 0x55d34d64d520 2024-09-21T06:47:17.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.779+0000 7f14abc72740 1 -- 172.21.15.67:0/1246420703 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d34d64d520 msgr2=0x55d34d9f3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:47:17.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.779+0000 7f14abc72740 1 --2- 172.21.15.67:0/1246420703 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d34d64d520 0x55d34d9f3510 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1498035000 tx=0x7f1498009a20 comp rx=0 tx=0).stop 2024-09-21T06:47:17.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.779+0000 7f149effd640 1 --2- 172.21.15.67:0/406883524 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f148c0b6510 0x7f148c0b8a20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55d34d9f2190 tx=0x7f149000b790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:47:17.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.779+0000 7f14abc72740 1 -- 172.21.15.67:0/1246420703 shutdown_connections 2024-09-21T06:47:17.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.779+0000 7f14abc72740 1 --2- 172.21.15.67:0/1246420703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d34d62c840 0x55d34d9f3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.779+0000 7f14abc72740 1 --2- 172.21.15.67:0/1246420703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d34d624660 0x55d34d9f3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.779+0000 7f14abc72740 1 --2- 172.21.15.67:0/1246420703 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d34d64d520 0x55d34d9f3510 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.779+0000 7f14abc72740 1 -- 172.21.15.67:0/1246420703 >> 172.21.15.67:0/1246420703 conn(0x55d34d641910 msgr2=0x55d34d9f1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:47:17.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.779+0000 7f14abc72740 1 -- 172.21.15.67:0/1246420703 shutdown_connections 2024-09-21T06:47:17.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.779+0000 7f14abc72740 1 -- 172.21.15.67:0/1246420703 wait complete. 2024-09-21T06:47:17.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.779+0000 7f14abc72740 1 -- start start 2024-09-21T06:47:17.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.779+0000 7f14abc72740 1 Processor -- start 2024-09-21T06:47:17.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.780+0000 7f14abc72740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d34d624660 0x55d34d9febc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:17.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.780+0000 7f14abc72740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d34d62c840 0x55d34d9ff100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:17.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.780+0000 7f14abc72740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d34d64d520 0x55d34da0ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:17.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.780+0000 7f14abc72740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d34d8f5300 con 0x55d34d624660 2024-09-21T06:47:17.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.780+0000 7f14abc72740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d34d95e430 con 0x55d34d64d520 2024-09-21T06:47:17.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.780+0000 7f14abc72740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d34d7bbbb0 con 0x55d34d62c840 2024-09-21T06:47:17.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.780+0000 7f149effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d34d624660 0x55d34d9febc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:17.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.780+0000 7f149effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d34d624660 0x55d34d9febc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35828/0 (socket says 172.21.15.67:35828) 2024-09-21T06:47:17.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.780+0000 7f149effd640 1 -- 172.21.15.67:0/793847782 learned_addr learned my addr 172.21.15.67:0/793847782 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:47:17.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.780+0000 7f14abc72740 10 client.0 _ll_register_callbacks cb 0x55d34d90f120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:47:17.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.780+0000 7f149f7fe640 1 --2- 172.21.15.67:0/793847782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d34d64d520 0x55d34da0ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:17.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.780+0000 7f1497fff640 1 --2- 172.21.15.67:0/793847782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d34d62c840 0x55d34d9ff100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:17.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.780+0000 7f149effd640 1 -- 172.21.15.67:0/793847782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d34d62c840 msgr2=0x55d34d9ff100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:47:17.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.780+0000 7f149effd640 1 --2- 172.21.15.67:0/793847782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d34d62c840 0x55d34d9ff100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.780+0000 7f149effd640 1 -- 172.21.15.67:0/793847782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d34d64d520 msgr2=0x55d34da0ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:47:17.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.780+0000 7f149effd640 1 --2- 172.21.15.67:0/793847782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d34d64d520 0x55d34da0ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.780+0000 7f149effd640 1 -- 172.21.15.67:0/793847782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d34d8e4780 con 0x55d34d624660 2024-09-21T06:47:17.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.780+0000 7f149f7fe640 1 --2- 172.21.15.67:0/793847782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d34d64d520 0x55d34da0ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:47:17.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.780+0000 7f1497fff640 1 --2- 172.21.15.67:0/793847782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d34d62c840 0x55d34d9ff100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:47:17.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.780+0000 7f149effd640 1 --2- 172.21.15.67:0/793847782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d34d624660 0x55d34d9febc0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1490011010 tx=0x7f149000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f451802c56d1b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:47:17.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.780+0000 7f14867fc640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:47:17.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.781+0000 7f14867fc640 1 -- 172.21.15.67:0/793847782 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149001e830 con 0x55d34d624660 2024-09-21T06:47:17.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.781+0000 7f14867fc640 1 -- 172.21.15.67:0/793847782 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f149001e9d0 con 0x55d34d624660 2024-09-21T06:47:17.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.781+0000 7f14abc72740 1 -- 172.21.15.67:0/793847782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1498036040 con 0x55d34d624660 2024-09-21T06:47:17.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.781+0000 7f14867fc640 1 -- 172.21.15.67:0/793847782 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f149002f050 con 0x55d34d624660 2024-09-21T06:47:17.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.781+0000 7f14abc72740 10 client.15897 fetch_fsmap learned FSMap version 21 2024-09-21T06:47:17.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.781+0000 7f14abc72740 1 -- 172.21.15.67:0/793847782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d34d6b8b00 con 0x55d34d624660 2024-09-21T06:47:17.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.781+0000 7f14867fc640 1 -- 172.21.15.67:0/793847782 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1498036040 con 0x55d34d624660 2024-09-21T06:47:17.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.781+0000 7f14867fc640 1 -- 172.21.15.67:0/793847782 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f149001c3b0 con 0x55d34d624660 2024-09-21T06:47:17.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.781+0000 7f14abc72740 10 client.15897 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:47:17.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.781+0000 7f14abc72740 10 client.15897 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:47:17.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.781+0000 7f14abc72740 20 client.15897 populate_metadata read hostname 'smithi067' 2024-09-21T06:47:17.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.781+0000 7f14abc72740 1 --2- 172.21.15.67:0/793847782 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55d34d60bea0 0x55d34da10fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:17.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.781+0000 7f14abc72740 4 client.15897 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:47:17.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.782+0000 7f14abc72740 1 -- 172.21.15.67:0/793847782 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d34d96d140 con 0x55d34d60bea0 2024-09-21T06:47:17.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.782+0000 7f1497fff640 1 --2- 172.21.15.67:0/793847782 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55d34d60bea0 0x55d34da10fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:17.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.782+0000 7f1497fff640 1 --2- 172.21.15.67:0/793847782 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55d34d60bea0 0x55d34da10fa0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e932172f0a5b81ec server_cookie=7234c28102c9f9ae in_seq=0 out_seq=0 2024-09-21T06:47:17.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.782+0000 7f14867fc640 10 client.15897 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:47:17.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.783+0000 7f14867fc640 1 -- 172.21.15.67:0/793847782 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d34d96d140 con 0x55d34d60bea0 2024-09-21T06:47:17.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.783+0000 7f14867fc640 10 client.15897 handle_command_reply: tid=0 2024-09-21T06:47:17.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.783+0000 7f14abc72740 1 client.15897 shutdown 2024-09-21T06:47:17.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.783+0000 7f14abc72740 1 -- 172.21.15.67:0/793847782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d34d624660 msgr2=0x55d34d9febc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:47:17.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.783+0000 7f14abc72740 1 --2- 172.21.15.67:0/793847782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d34d624660 0x55d34d9febc0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1490011010 tx=0x7f149000b390 comp rx=0 tx=0).stop 2024-09-21T06:47:17.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.783+0000 7f14abc72740 1 -- 172.21.15.67:0/793847782 shutdown_connections 2024-09-21T06:47:17.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.783+0000 7f14abc72740 1 --2- 172.21.15.67:0/793847782 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55d34d60bea0 0x55d34da10fa0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.783+0000 7f14abc72740 1 --2- 172.21.15.67:0/793847782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d34d64d520 0x55d34da0ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.783+0000 7f14abc72740 1 --2- 172.21.15.67:0/793847782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d34d62c840 0x55d34d9ff100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.783+0000 7f14867fc640 0 client.15897 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:47:17.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.783+0000 7f14abc72740 1 --2- 172.21.15.67:0/793847782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d34d624660 0x55d34d9febc0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.783+0000 7f14abc72740 1 -- 172.21.15.67:0/793847782 >> 172.21.15.67:0/793847782 conn(0x55d34d641910 msgr2=0x55d34d9f66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:47:17.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.783+0000 7f14abc72740 1 -- 172.21.15.67:0/793847782 shutdown_connections 2024-09-21T06:47:17.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.783+0000 7f14abc72740 1 -- 172.21.15.67:0/793847782 wait complete. 2024-09-21T06:47:17.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.783+0000 7f14abc72740 20 client.15897 trim_cache size 0 max 16384 2024-09-21T06:47:17.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.801+0000 7f14abc72740 1 Processor -- start 2024-09-21T06:47:17.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.801+0000 7f14abc72740 1 -- start start 2024-09-21T06:47:17.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.801+0000 7f14abc72740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d34d60bea0 0x55d34d9f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:17.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.801+0000 7f14abc72740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d34d624660 0x55d34da0ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:17.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.801+0000 7f14abc72740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d34d62c840 0x55d34da00aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:17.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.801+0000 7f14abc72740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f14a01b0e00 con 0x55d34d624660 2024-09-21T06:47:17.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.801+0000 7f14abc72740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f14a01b3680 con 0x55d34d62c840 2024-09-21T06:47:17.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.801+0000 7f14abc72740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f14a01b1470 con 0x55d34d60bea0 2024-09-21T06:47:17.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.801+0000 7f1497fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d34d624660 0x55d34da0ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:17.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.801+0000 7f1497fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d34d624660 0x55d34da0ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35840/0 (socket says 172.21.15.67:35840) 2024-09-21T06:47:17.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.801+0000 7f1497fff640 1 -- 172.21.15.67:0/1102621165 learned_addr learned my addr 172.21.15.67:0/1102621165 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:47:17.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.801+0000 7f149effd640 1 --2- 172.21.15.67:0/1102621165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d34d60bea0 0x55d34d9f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:17.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.801+0000 7f149f7fe640 1 --2- 172.21.15.67:0/1102621165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d34d62c840 0x55d34da00aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:17.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.802+0000 7f1497fff640 1 -- 172.21.15.67:0/1102621165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d34d60bea0 msgr2=0x55d34d9f4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:47:17.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.802+0000 7f1497fff640 1 --2- 172.21.15.67:0/1102621165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d34d60bea0 0x55d34d9f4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.802+0000 7f1497fff640 1 -- 172.21.15.67:0/1102621165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d34d62c840 msgr2=0x55d34da00aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:47:17.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.802+0000 7f1497fff640 1 --2- 172.21.15.67:0/1102621165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d34d62c840 0x55d34da00aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.802+0000 7f1497fff640 1 -- 172.21.15.67:0/1102621165 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d34d72f960 con 0x55d34d624660 2024-09-21T06:47:17.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.802+0000 7f1497fff640 1 --2- 172.21.15.67:0/1102621165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d34d624660 0x55d34da0ad00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f14880a9490 tx=0x7f14880e4700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532f4803bd863915 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:47:17.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.802+0000 7f1494ff9640 1 -- 172.21.15.67:0/1102621165 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14880b19a0 con 0x55d34d624660 2024-09-21T06:47:17.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.802+0000 7f1494ff9640 1 -- 172.21.15.67:0/1102621165 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14880b68e0 con 0x55d34d624660 2024-09-21T06:47:17.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.802+0000 7f1494ff9640 1 -- 172.21.15.67:0/1102621165 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14880ac120 con 0x55d34d624660 2024-09-21T06:47:17.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.803+0000 7f14abc72740 1 -- 172.21.15.67:0/1102621165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d34d624660 msgr2=0x55d34da0ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:47:17.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.803+0000 7f14abc72740 1 --2- 172.21.15.67:0/1102621165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d34d624660 0x55d34da0ad00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f14880a9490 tx=0x7f14880e4700 comp rx=0 tx=0).stop 2024-09-21T06:47:17.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.803+0000 7f14abc72740 1 -- 172.21.15.67:0/1102621165 shutdown_connections 2024-09-21T06:47:17.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.803+0000 7f14abc72740 1 --2- 172.21.15.67:0/1102621165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d34d62c840 0x55d34da00aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.803+0000 7f14abc72740 1 --2- 172.21.15.67:0/1102621165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d34d624660 0x55d34da0ad00 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.803+0000 7f14abc72740 1 --2- 172.21.15.67:0/1102621165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d34d60bea0 0x55d34d9f4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.803+0000 7f14abc72740 1 -- 172.21.15.67:0/1102621165 >> 172.21.15.67:0/1102621165 conn(0x55d34d641910 msgr2=0x55d34d9fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:47:17.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.803+0000 7f14abc72740 1 -- 172.21.15.67:0/1102621165 shutdown_connections 2024-09-21T06:47:17.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.803+0000 7f14abc72740 1 -- 172.21.15.67:0/1102621165 wait complete. 2024-09-21T06:47:17.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.803+0000 7f14abc72740 1 -- start start 2024-09-21T06:47:17.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.803+0000 7f14abc72740 1 Processor -- start 2024-09-21T06:47:17.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.804+0000 7f14abc72740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d34d60bea0 0x55d34da03180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:17.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.804+0000 7f14abc72740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d34d624660 0x55d34da03740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:17.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.804+0000 7f14abc72740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d34d62c840 0x55d34da7a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:17.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.804+0000 7f14abc72740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f14a01b1b40 con 0x55d34d60bea0 2024-09-21T06:47:17.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.804+0000 7f14abc72740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f14a01b2fb0 con 0x55d34d624660 2024-09-21T06:47:17.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.804+0000 7f14abc72740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f14a01b2210 con 0x55d34d62c840 2024-09-21T06:47:17.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.804+0000 7f14abc72740 10 client.0 _ll_register_callbacks cb 0x7f1490005c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:47:17.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.804+0000 7f149effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d34d60bea0 0x55d34da03180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:17.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.804+0000 7f149effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d34d60bea0 0x55d34da03180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35854/0 (socket says 172.21.15.67:35854) 2024-09-21T06:47:17.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.804+0000 7f149effd640 1 -- 172.21.15.67:0/750869038 learned_addr learned my addr 172.21.15.67:0/750869038 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:47:17.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.804+0000 7f149f7fe640 1 --2- 172.21.15.67:0/750869038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d34d62c840 0x55d34da7a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:17.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.804+0000 7f1497fff640 1 --2- 172.21.15.67:0/750869038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d34d624660 0x55d34da03740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:17.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.804+0000 7f149effd640 1 -- 172.21.15.67:0/750869038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d34d62c840 msgr2=0x55d34da7a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:47:17.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.804+0000 7f149effd640 1 --2- 172.21.15.67:0/750869038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d34d62c840 0x55d34da7a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.804+0000 7f149effd640 1 -- 172.21.15.67:0/750869038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d34d624660 msgr2=0x55d34da03740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:47:17.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.804+0000 7f149effd640 1 --2- 172.21.15.67:0/750869038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d34d624660 0x55d34da03740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.804+0000 7f149effd640 1 -- 172.21.15.67:0/750869038 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d34da7aea0 con 0x55d34d60bea0 2024-09-21T06:47:17.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.804+0000 7f149f7fe640 1 --2- 172.21.15.67:0/750869038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d34d62c840 0x55d34da7a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:47:17.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.804+0000 7f149effd640 1 --2- 172.21.15.67:0/750869038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d34d60bea0 0x55d34da03180 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1490010d20 tx=0x7f149001cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b68e9759bcd4911 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:47:17.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.804+0000 7f14867fc640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:47:17.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.805+0000 7f14867fc640 1 -- 172.21.15.67:0/750869038 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1490005030 con 0x55d34d60bea0 2024-09-21T06:47:17.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.805+0000 7f14867fc640 1 -- 172.21.15.67:0/750869038 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f149002c070 con 0x55d34d60bea0 2024-09-21T06:47:17.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.805+0000 7f14abc72740 1 -- 172.21.15.67:0/750869038 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14880b68e0 con 0x55d34d60bea0 2024-09-21T06:47:17.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.805+0000 7f14867fc640 1 -- 172.21.15.67:0/750869038 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1490044050 con 0x55d34d60bea0 2024-09-21T06:47:17.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.805+0000 7f14abc72740 10 client.15903 fetch_fsmap learned FSMap version 21 2024-09-21T06:47:17.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.805+0000 7f14abc72740 1 -- 172.21.15.67:0/750869038 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d34da7c070 con 0x55d34d60bea0 2024-09-21T06:47:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.805+0000 7f14867fc640 1 -- 172.21.15.67:0/750869038 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14880b68e0 con 0x55d34d60bea0 2024-09-21T06:47:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.805+0000 7f14867fc640 1 -- 172.21.15.67:0/750869038 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f149001ea50 con 0x55d34d60bea0 2024-09-21T06:47:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.805+0000 7f14abc72740 10 client.15903 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:47:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.805+0000 7f14abc72740 10 client.15903 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:47:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.805+0000 7f14abc72740 20 client.15903 populate_metadata read hostname 'smithi067' 2024-09-21T06:47:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.805+0000 7f14abc72740 1 --2- 172.21.15.67:0/750869038 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55d34d64d520 0x55d34da7e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.805+0000 7f14abc72740 4 client.15903 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6a9bb4bb", "release": true, "await": true}] 2024-09-21T06:47:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.805+0000 7f14abc72740 1 -- 172.21.15.67:0/750869038 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6a9bb4bb", "release": true, "await": true}) -- 0x55d34d968bf0 con 0x55d34d64d520 2024-09-21T06:47:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.806+0000 7f1497fff640 1 --2- 172.21.15.67:0/750869038 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55d34d64d520 0x55d34da7e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.806+0000 7f1497fff640 1 --2- 172.21.15.67:0/750869038 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55d34d64d520 0x55d34da7e800 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=829ce9040c9ab8ad server_cookie=73b52efe296d4e5a in_seq=0 out_seq=0 2024-09-21T06:47:17.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.806+0000 7f14867fc640 10 client.15903 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:47:17.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.809+0000 7f14867fc640 1 -- 172.21.15.67:0/750869038 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d34d968bf0 con 0x55d34d64d520 2024-09-21T06:47:17.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.809+0000 7f14867fc640 10 client.15903 handle_command_reply: tid=0 2024-09-21T06:47:17.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.809+0000 7f14abc72740 1 client.15903 shutdown 2024-09-21T06:47:17.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.809+0000 7f14abc72740 1 -- 172.21.15.67:0/750869038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d34d60bea0 msgr2=0x55d34da03180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:47:17.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.809+0000 7f14abc72740 1 --2- 172.21.15.67:0/750869038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d34d60bea0 0x55d34da03180 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1490010d20 tx=0x7f149001cb90 comp rx=0 tx=0).stop 2024-09-21T06:47:17.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.809+0000 7f14abc72740 1 -- 172.21.15.67:0/750869038 shutdown_connections 2024-09-21T06:47:17.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.809+0000 7f14abc72740 1 --2- 172.21.15.67:0/750869038 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55d34d64d520 0x55d34da7e800 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.809+0000 7f14abc72740 1 --2- 172.21.15.67:0/750869038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d34d624660 0x55d34da03740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.809+0000 7f14867fc640 0 client.15903 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:47:17.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.809+0000 7f14abc72740 1 --2- 172.21.15.67:0/750869038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d34d62c840 0x55d34da7a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.809+0000 7f14abc72740 1 --2- 172.21.15.67:0/750869038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d34d60bea0 0x55d34da03180 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.809+0000 7f14abc72740 1 -- 172.21.15.67:0/750869038 >> 172.21.15.67:0/750869038 conn(0x55d34d641910 msgr2=0x55d34d9fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:47:17.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.810+0000 7f14abc72740 1 -- 172.21.15.67:0/750869038 shutdown_connections 2024-09-21T06:47:17.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.810+0000 7f14abc72740 1 -- 172.21.15.67:0/750869038 wait complete. 2024-09-21T06:47:17.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.810+0000 7f14abc72740 20 client.15903 trim_cache size 0 max 16384 2024-09-21T06:47:17.817 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-21T06:47:17.817 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 21, 2024-09-21T06:47:17.818 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-09-21T06:47:17.818 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 44, 2024-09-21T06:47:17.818 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-21T06:47:17.818 INFO:teuthology.orchestra.run.smithi067.stdout: "6a9bb4bb": { 2024-09-21T06:47:17.818 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 44, 2024-09-21T06:47:17.818 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-21T06:47:17.818 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:47:17.818 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-21T06:47:17.818 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:47:17.818 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:47:17.818 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-21T06:47:17.818 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 144.5, 2024-09-21T06:47:17.818 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-21T06:47:17.818 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-21T06:47:17.819 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-21T06:47:17.819 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-21T06:47:17.819 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-21T06:47:17.819 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-21T06:47:17.819 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:47:17.819 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:47:17.819 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:47:17.819 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:47:17.819 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:47:17.819 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-21T06:47:17.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.810+0000 7f14a56fe640 1 -- 172.21.15.67:0/406883524 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f148c0b6510 msgr2=0x7f148c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:47:17.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.810+0000 7f14a56fe640 1 --2- 172.21.15.67:0/406883524 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f148c0b6510 0x7f148c0b8a20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55d34d9f2190 tx=0x7f149000b790 comp rx=0 tx=0).stop 2024-09-21T06:47:17.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.810+0000 7f14a56fe640 1 -- 172.21.15.67:0/406883524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14a01083d0 msgr2=0x7f14a0105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:47:17.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.810+0000 7f14a56fe640 1 --2- 172.21.15.67:0/406883524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14a01083d0 0x7f14a0105650 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f148800b450 tx=0x7f148800b910 comp rx=0 tx=0).stop 2024-09-21T06:47:17.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.811+0000 7f14a56fe640 1 -- 172.21.15.67:0/406883524 shutdown_connections 2024-09-21T06:47:17.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.811+0000 7f14a56fe640 1 --2- 172.21.15.67:0/406883524 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f148c0b6510 0x7f148c0b8a20 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.811+0000 7f14a56fe640 1 --2- 172.21.15.67:0/406883524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14a0108d20 0x7f14a0105b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.811+0000 7f14a56fe640 1 --2- 172.21.15.67:0/406883524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14a01083d0 0x7f14a0105650 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.811+0000 7f14a56fe640 1 --2- 172.21.15.67:0/406883524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14a0102ae0 0x7f14a0105110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:17.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.811+0000 7f14a56fe640 1 -- 172.21.15.67:0/406883524 >> 172.21.15.67:0/406883524 conn(0x7f14a0079760 msgr2=0x7f14a0100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:47:17.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.811+0000 7f14a56fe640 1 -- 172.21.15.67:0/406883524 shutdown_connections 2024-09-21T06:47:17.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:47:17.811+0000 7f14a56fe640 1 -- 172.21.15.67:0/406883524 wait complete. 2024-09-21T06:47:18.601 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6a9bb4bb', total seconds elapsed: 27.9 2024-09-21T06:47:19.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:19 smithi203 ceph-mon[33735]: pgmap v682: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 75 KiB/s wr, 4 op/s 2024-09-21T06:47:19.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:19 smithi067 ceph-mon[31232]: pgmap v682: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 75 KiB/s wr, 4 op/s 2024-09-21T06:47:19.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:19 smithi100 ceph-mon[33401]: pgmap v682: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 75 KiB/s wr, 4 op/s 2024-09-21T06:47:21.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:21 smithi203 ceph-mon[33735]: pgmap v683: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 261 KiB/s wr, 7 op/s 2024-09-21T06:47:21.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:21 smithi067 ceph-mon[31232]: pgmap v683: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 261 KiB/s wr, 7 op/s 2024-09-21T06:47:21.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:21 smithi100 ceph-mon[33401]: pgmap v683: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 261 KiB/s wr, 7 op/s 2024-09-21T06:47:23.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:23 smithi203 ceph-mon[33735]: pgmap v684: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.9 MiB/s wr, 162 op/s 2024-09-21T06:47:23.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:23 smithi067 ceph-mon[31232]: pgmap v684: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.9 MiB/s wr, 162 op/s 2024-09-21T06:47:23.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:23 smithi100 ceph-mon[33401]: pgmap v684: 129 pgs: 129 active+clean; 10 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.9 MiB/s wr, 162 op/s 2024-09-21T06:47:25.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:25 smithi203 ceph-mon[33735]: pgmap v685: 129 pgs: 129 active+clean; 10 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 10 MiB/s wr, 239 op/s 2024-09-21T06:47:25.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:25 smithi067 ceph-mon[31232]: pgmap v685: 129 pgs: 129 active+clean; 10 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 10 MiB/s wr, 239 op/s 2024-09-21T06:47:25.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:25 smithi100 ceph-mon[33401]: pgmap v685: 129 pgs: 129 active+clean; 10 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 10 MiB/s wr, 239 op/s 2024-09-21T06:47:27.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:27 smithi203 ceph-mon[33735]: pgmap v686: 129 pgs: 129 active+clean; 10 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 13 MiB/s wr, 299 op/s 2024-09-21T06:47:27.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:27 smithi067 ceph-mon[31232]: pgmap v686: 129 pgs: 129 active+clean; 10 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 13 MiB/s wr, 299 op/s 2024-09-21T06:47:27.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:27 smithi100 ceph-mon[33401]: pgmap v686: 129 pgs: 129 active+clean; 10 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 13 MiB/s wr, 299 op/s 2024-09-21T06:47:29.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:29 smithi203 ceph-mon[33735]: pgmap v687: 129 pgs: 129 active+clean; 10 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 23 MiB/s wr, 533 op/s 2024-09-21T06:47:29.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:29 smithi067 ceph-mon[31232]: pgmap v687: 129 pgs: 129 active+clean; 10 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 23 MiB/s wr, 533 op/s 2024-09-21T06:47:29.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:29 smithi100 ceph-mon[33401]: pgmap v687: 129 pgs: 129 active+clean; 10 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 23 MiB/s wr, 533 op/s 2024-09-21T06:47:31.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:31 smithi203 ceph-mon[33735]: pgmap v688: 129 pgs: 129 active+clean; 10 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 24 MiB/s wr, 539 op/s 2024-09-21T06:47:31.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:31 smithi067 ceph-mon[31232]: pgmap v688: 129 pgs: 129 active+clean; 10 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 24 MiB/s wr, 539 op/s 2024-09-21T06:47:31.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:31 smithi100 ceph-mon[33401]: pgmap v688: 129 pgs: 129 active+clean; 10 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 24 MiB/s wr, 539 op/s 2024-09-21T06:47:33.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:33 smithi203 ceph-mon[33735]: pgmap v689: 129 pgs: 129 active+clean; 10 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 33 MiB/s wr, 749 op/s 2024-09-21T06:47:33.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:33 smithi067 ceph-mon[31232]: pgmap v689: 129 pgs: 129 active+clean; 10 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 33 MiB/s wr, 749 op/s 2024-09-21T06:47:33.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:33 smithi100 ceph-mon[33401]: pgmap v689: 129 pgs: 129 active+clean; 10 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 33 MiB/s wr, 749 op/s 2024-09-21T06:47:35.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:35 smithi203 ceph-mon[33735]: pgmap v690: 129 pgs: 129 active+clean; 10 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 30 MiB/s wr, 664 op/s 2024-09-21T06:47:35.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:35 smithi067 ceph-mon[31232]: pgmap v690: 129 pgs: 129 active+clean; 10 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 30 MiB/s wr, 664 op/s 2024-09-21T06:47:35.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:35 smithi100 ceph-mon[33401]: pgmap v690: 129 pgs: 129 active+clean; 10 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 30 MiB/s wr, 664 op/s 2024-09-21T06:47:37.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:37 smithi203 ceph-mon[33735]: pgmap v691: 129 pgs: 129 active+clean; 10 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 29 MiB/s wr, 651 op/s 2024-09-21T06:47:37.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:37 smithi067 ceph-mon[31232]: pgmap v691: 129 pgs: 129 active+clean; 10 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 29 MiB/s wr, 651 op/s 2024-09-21T06:47:37.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:37 smithi100 ceph-mon[33401]: pgmap v691: 129 pgs: 129 active+clean; 10 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 29 MiB/s wr, 651 op/s 2024-09-21T06:47:39.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:39 smithi203 ceph-mon[33735]: pgmap v692: 129 pgs: 129 active+clean; 10 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 36 MiB/s wr, 788 op/s 2024-09-21T06:47:39.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:39 smithi067 ceph-mon[31232]: pgmap v692: 129 pgs: 129 active+clean; 10 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 36 MiB/s wr, 788 op/s 2024-09-21T06:47:39.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:39 smithi100 ceph-mon[33401]: pgmap v692: 129 pgs: 129 active+clean; 10 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 36 MiB/s wr, 788 op/s 2024-09-21T06:47:41.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:41 smithi203 ceph-mon[33735]: pgmap v693: 129 pgs: 129 active+clean; 10 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 27 MiB/s wr, 593 op/s 2024-09-21T06:47:41.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:41 smithi067 ceph-mon[31232]: pgmap v693: 129 pgs: 129 active+clean; 10 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 27 MiB/s wr, 593 op/s 2024-09-21T06:47:41.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:41 smithi100 ceph-mon[33401]: pgmap v693: 129 pgs: 129 active+clean; 10 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 27 MiB/s wr, 593 op/s 2024-09-21T06:47:43.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:43 smithi203 ceph-mon[33735]: pgmap v694: 129 pgs: 129 active+clean; 10 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 35 MiB/s wr, 778 op/s 2024-09-21T06:47:43.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:43 smithi067 ceph-mon[31232]: pgmap v694: 129 pgs: 129 active+clean; 10 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 35 MiB/s wr, 778 op/s 2024-09-21T06:47:43.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:43 smithi100 ceph-mon[33401]: pgmap v694: 129 pgs: 129 active+clean; 10 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 35 MiB/s wr, 778 op/s 2024-09-21T06:47:45.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:45 smithi203 ceph-mon[33735]: pgmap v695: 129 pgs: 129 active+clean; 10 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 29 MiB/s wr, 635 op/s 2024-09-21T06:47:45.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:45 smithi067 ceph-mon[31232]: pgmap v695: 129 pgs: 129 active+clean; 10 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 29 MiB/s wr, 635 op/s 2024-09-21T06:47:45.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:45 smithi100 ceph-mon[33401]: pgmap v695: 129 pgs: 129 active+clean; 10 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 29 MiB/s wr, 635 op/s 2024-09-21T06:47:46.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:46 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:47:46.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:46 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:47:46.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:46 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:47:46.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:46 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:47:46.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:46 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:47:46.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:46 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:47:46.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:46 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:47:46.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:46 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:47:46.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:46 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:47:46.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:46 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:47:46.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:46 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:47:46.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:46 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:47:46.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:46 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:47:46.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:46 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:47:46.804 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:46 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:47:47.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:47 smithi203 ceph-mon[33735]: pgmap v696: 129 pgs: 129 active+clean; 10 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 29 MiB/s wr, 627 op/s 2024-09-21T06:47:47.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:47 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:47:47.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:47 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:47:47.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:47 smithi067 ceph-mon[31232]: pgmap v696: 129 pgs: 129 active+clean; 10 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 29 MiB/s wr, 627 op/s 2024-09-21T06:47:47.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:47 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:47:47.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:47 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:47:47.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:47 smithi100 ceph-mon[33401]: pgmap v696: 129 pgs: 129 active+clean; 10 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 29 MiB/s wr, 627 op/s 2024-09-21T06:47:47.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:47 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:47:47.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:47 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:47:48.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:48 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:47:48.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:48 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:47:48.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:48 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:47:48.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:48 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:47:48.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:48 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:47:48.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:48 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:47:48.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:48 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:47:48.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:48 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:47:48.803 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:48 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:47:50.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:49 smithi067 ceph-mon[31232]: pgmap v697: 129 pgs: 129 active+clean; 10 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 35 MiB/s wr, 754 op/s 2024-09-21T06:47:50.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:49 smithi100 ceph-mon[33401]: pgmap v697: 129 pgs: 129 active+clean; 10 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 35 MiB/s wr, 754 op/s 2024-09-21T06:47:50.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:49 smithi203 ceph-mon[33735]: pgmap v697: 129 pgs: 129 active+clean; 10 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 35 MiB/s wr, 754 op/s 2024-09-21T06:47:52.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:51 smithi067 ceph-mon[31232]: pgmap v698: 129 pgs: 129 active+clean; 10 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 27 MiB/s wr, 593 op/s 2024-09-21T06:47:52.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:51 smithi100 ceph-mon[33401]: pgmap v698: 129 pgs: 129 active+clean; 10 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 27 MiB/s wr, 593 op/s 2024-09-21T06:47:52.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:51 smithi203 ceph-mon[33735]: pgmap v698: 129 pgs: 129 active+clean; 10 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 27 MiB/s wr, 593 op/s 2024-09-21T06:47:54.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:53 smithi067 ceph-mon[31232]: pgmap v699: 129 pgs: 129 active+clean; 10 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 34 MiB/s wr, 745 op/s 2024-09-21T06:47:54.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:53 smithi100 ceph-mon[33401]: pgmap v699: 129 pgs: 129 active+clean; 10 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 34 MiB/s wr, 745 op/s 2024-09-21T06:47:54.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:53 smithi203 ceph-mon[33735]: pgmap v699: 129 pgs: 129 active+clean; 10 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 34 MiB/s wr, 745 op/s 2024-09-21T06:47:56.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:55 smithi067 ceph-mon[31232]: pgmap v700: 129 pgs: 129 active+clean; 10 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 29 MiB/s wr, 624 op/s 2024-09-21T06:47:56.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:55 smithi100 ceph-mon[33401]: pgmap v700: 129 pgs: 129 active+clean; 10 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 29 MiB/s wr, 624 op/s 2024-09-21T06:47:56.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:55 smithi203 ceph-mon[33735]: pgmap v700: 129 pgs: 129 active+clean; 10 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 29 MiB/s wr, 624 op/s 2024-09-21T06:47:58.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:57 smithi067 ceph-mon[31232]: pgmap v701: 129 pgs: 129 active+clean; 10 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 30 MiB/s wr, 646 op/s 2024-09-21T06:47:58.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:57 smithi100 ceph-mon[33401]: pgmap v701: 129 pgs: 129 active+clean; 10 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 30 MiB/s wr, 646 op/s 2024-09-21T06:47:58.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:57 smithi203 ceph-mon[33735]: pgmap v701: 129 pgs: 129 active+clean; 10 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 30 MiB/s wr, 646 op/s 2024-09-21T06:48:00.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:47:59 smithi067 ceph-mon[31232]: pgmap v702: 129 pgs: 129 active+clean; 10 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 35 MiB/s wr, 738 op/s 2024-09-21T06:48:00.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:47:59 smithi100 ceph-mon[33401]: pgmap v702: 129 pgs: 129 active+clean; 10 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 35 MiB/s wr, 738 op/s 2024-09-21T06:48:00.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:47:59 smithi203 ceph-mon[33735]: pgmap v702: 129 pgs: 129 active+clean; 10 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 35 MiB/s wr, 738 op/s 2024-09-21T06:48:02.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:01 smithi067 ceph-mon[31232]: pgmap v703: 129 pgs: 129 active+clean; 10 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 28 MiB/s wr, 584 op/s 2024-09-21T06:48:02.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:01 smithi100 ceph-mon[33401]: pgmap v703: 129 pgs: 129 active+clean; 10 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 28 MiB/s wr, 584 op/s 2024-09-21T06:48:02.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:01 smithi203 ceph-mon[33735]: pgmap v703: 129 pgs: 129 active+clean; 10 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 28 MiB/s wr, 584 op/s 2024-09-21T06:48:04.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:03 smithi100 ceph-mon[33401]: pgmap v704: 129 pgs: 129 active+clean; 10 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 35 MiB/s wr, 749 op/s 2024-09-21T06:48:04.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:03 smithi203 ceph-mon[33735]: pgmap v704: 129 pgs: 129 active+clean; 10 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 35 MiB/s wr, 749 op/s 2024-09-21T06:48:04.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:03 smithi067 ceph-mon[31232]: pgmap v704: 129 pgs: 129 active+clean; 10 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 35 MiB/s wr, 749 op/s 2024-09-21T06:48:05.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:04 smithi203 ceph-mon[33735]: pgmap v705: 129 pgs: 129 active+clean; 10 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 29 MiB/s wr, 626 op/s 2024-09-21T06:48:05.292 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:04 smithi067 ceph-mon[31232]: pgmap v705: 129 pgs: 129 active+clean; 10 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 29 MiB/s wr, 626 op/s 2024-09-21T06:48:05.303 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:04 smithi100 ceph-mon[33401]: pgmap v705: 129 pgs: 129 active+clean; 10 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 29 MiB/s wr, 626 op/s 2024-09-21T06:48:07.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:07 smithi067 ceph-mon[31232]: pgmap v706: 129 pgs: 129 active+clean; 10 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 30 MiB/s wr, 653 op/s 2024-09-21T06:48:07.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:07 smithi100 ceph-mon[33401]: pgmap v706: 129 pgs: 129 active+clean; 10 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 30 MiB/s wr, 653 op/s 2024-09-21T06:48:07.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:07 smithi203 ceph-mon[33735]: pgmap v706: 129 pgs: 129 active+clean; 10 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 30 MiB/s wr, 653 op/s 2024-09-21T06:48:08.047 INFO:tasks.workunit.client.0.smithi067.stdout:stopping iogen 2024-09-21T06:48:08.047 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'stopping iogen' 2024-09-21T06:48:08.047 INFO:tasks.workunit.client.0.smithi067.stderr:+ ./iogen -k 2024-09-21T06:48:08.054 INFO:tasks.workunit.client.0.smithi067.stdout:OK 2024-09-21T06:48:08.054 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo OK 2024-09-21T06:48:09.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:09 smithi067 ceph-mon[31232]: pgmap v707: 129 pgs: 129 active+clean; 10 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 33 MiB/s wr, 713 op/s 2024-09-21T06:48:09.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:09 smithi100 ceph-mon[33401]: pgmap v707: 129 pgs: 129 active+clean; 10 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 33 MiB/s wr, 713 op/s 2024-09-21T06:48:09.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:09 smithi203 ceph-mon[33735]: pgmap v707: 129 pgs: 129 active+clean; 10 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 33 MiB/s wr, 713 op/s 2024-09-21T06:48:10.239 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-21T06:48:10.239 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-21T06:48:10.974 INFO:tasks.workunit:Stopping ['suites/iogen.sh'] on client.0... 2024-09-21T06:48:10.974 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-21T06:48:11.303 DEBUG:teuthology.parallel:result is None 2024-09-21T06:48:11.303 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T06:48:11.341 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T06:48:11.341 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-21T06:48:11.351 INFO:tasks.quiescer:joining Quiescers 2024-09-21T06:48:11.351 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-21T06:48:11.351 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:48:11.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:11 smithi067 ceph-mon[31232]: pgmap v708: 129 pgs: 129 active+clean; 10 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 28 MiB/s wr, 599 op/s 2024-09-21T06:48:11.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:11 smithi100 ceph-mon[33401]: pgmap v708: 129 pgs: 129 active+clean; 10 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 28 MiB/s wr, 599 op/s 2024-09-21T06:48:11.667 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:48:11.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:11 smithi203 ceph-mon[33735]: pgmap v708: 129 pgs: 129 active+clean; 10 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 28 MiB/s wr, 599 op/s 2024-09-21T06:48:13.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:13 smithi067 ceph-mon[31232]: pgmap v709: 129 pgs: 129 active+clean; 10 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 33 MiB/s wr, 716 op/s 2024-09-21T06:48:13.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:13 smithi100 ceph-mon[33401]: pgmap v709: 129 pgs: 129 active+clean; 10 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 33 MiB/s wr, 716 op/s 2024-09-21T06:48:13.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:13 smithi203 ceph-mon[33735]: pgmap v709: 129 pgs: 129 active+clean; 10 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 33 MiB/s wr, 716 op/s 2024-09-21T06:48:15.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:15 smithi067 ceph-mon[31232]: pgmap v710: 129 pgs: 129 active+clean; 9.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 26 MiB/s wr, 558 op/s 2024-09-21T06:48:15.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:15 smithi100 ceph-mon[33401]: pgmap v710: 129 pgs: 129 active+clean; 9.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 26 MiB/s wr, 558 op/s 2024-09-21T06:48:15.732 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:15 smithi203 ceph-mon[33735]: pgmap v710: 129 pgs: 129 active+clean; 9.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 26 MiB/s wr, 558 op/s 2024-09-21T06:48:16.545 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:48:16.545 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:48:16.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.915+0000 7f39038c6640 1 -- 172.21.15.67:0/3188614014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38fc1084b0 msgr2=0x7f38fc10ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:16.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.915+0000 7f39038c6640 1 --2- 172.21.15.67:0/3188614014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38fc1084b0 0x7f38fc10ece0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f38f4009be0 tx=0x7f38f402a6f0 comp rx=0 tx=0).stop 2024-09-21T06:48:16.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.917+0000 7f39038c6640 1 -- 172.21.15.67:0/3188614014 shutdown_connections 2024-09-21T06:48:16.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.917+0000 7f39038c6640 1 --2- 172.21.15.67:0/3188614014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38fc1084b0 0x7f38fc10ece0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:16.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.917+0000 7f39038c6640 1 --2- 172.21.15.67:0/3188614014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38fc107b60 0x7f38fc107f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:16.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.917+0000 7f39038c6640 1 --2- 172.21.15.67:0/3188614014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38fc102270 0x7f38fc102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:16.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.917+0000 7f39038c6640 1 -- 172.21.15.67:0/3188614014 >> 172.21.15.67:0/3188614014 conn(0x7f38fc077cc0 msgr2=0x7f38fc078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:16.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.919+0000 7f39038c6640 1 -- 172.21.15.67:0/3188614014 shutdown_connections 2024-09-21T06:48:16.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.920+0000 7f39038c6640 1 -- 172.21.15.67:0/3188614014 wait complete. 2024-09-21T06:48:16.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.922+0000 7f39038c6640 1 Processor -- start 2024-09-21T06:48:16.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.922+0000 7f39038c6640 1 -- start start 2024-09-21T06:48:16.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.922+0000 7f39038c6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38fc102270 0x7f38fc10a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:16.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.922+0000 7f39038c6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38fc1084b0 0x7f38fc10a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:16.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.922+0000 7f39038c6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38fc1a1e80 0x7f38fc1a42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:16.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.922+0000 7f39038c6640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc06ec90 con 0x7f38fc1084b0 2024-09-21T06:48:16.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.922+0000 7f39038c6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc06eb10 con 0x7f38fc102270 2024-09-21T06:48:16.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.922+0000 7f39038c6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc06ee10 con 0x7f38fc1a1e80 2024-09-21T06:48:16.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.922+0000 7f3900e61640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38fc1084b0 0x7f38fc10a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:16.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.922+0000 7f3900e61640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38fc1084b0 0x7f38fc10a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57412/0 (socket says 172.21.15.67:57412) 2024-09-21T06:48:16.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.922+0000 7f3900e61640 1 -- 172.21.15.67:0/2822616076 learned_addr learned my addr 172.21.15.67:0/2822616076 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:48:16.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.923+0000 7f3901e63640 1 --2- 172.21.15.67:0/2822616076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38fc1a1e80 0x7f38fc1a42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:16.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.923+0000 7f3900e61640 1 -- 172.21.15.67:0/2822616076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38fc1a1e80 msgr2=0x7f38fc1a42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:16.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.923+0000 7f3900e61640 1 --2- 172.21.15.67:0/2822616076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38fc1a1e80 0x7f38fc1a42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:16.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.923+0000 7f3900e61640 1 -- 172.21.15.67:0/2822616076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38fc102270 msgr2=0x7f38fc10a490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:48:16.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.923+0000 7f3900e61640 1 --2- 172.21.15.67:0/2822616076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38fc102270 0x7f38fc10a490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:16.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.923+0000 7f3900e61640 1 -- 172.21.15.67:0/2822616076 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38fc194640 con 0x7f38fc1084b0 2024-09-21T06:48:16.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.923+0000 7f3901e63640 1 --2- 172.21.15.67:0/2822616076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38fc1a1e80 0x7f38fc1a42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:48:16.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.924+0000 7f3900e61640 1 --2- 172.21.15.67:0/2822616076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38fc1084b0 0x7f38fc10a9d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00b410 tx=0x7f38ec00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:16.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.925+0000 7f38f27fc640 1 -- 172.21.15.67:0/2822616076 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ec015020 con 0x7f38fc1084b0 2024-09-21T06:48:16.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.925+0000 7f38f27fc640 1 -- 172.21.15.67:0/2822616076 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f38ec011cc0 con 0x7f38fc1084b0 2024-09-21T06:48:16.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.925+0000 7f38f27fc640 1 -- 172.21.15.67:0/2822616076 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ec010cc0 con 0x7f38fc1084b0 2024-09-21T06:48:16.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.925+0000 7f39038c6640 1 -- 172.21.15.67:0/2822616076 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38fc1948d0 con 0x7f38fc1084b0 2024-09-21T06:48:16.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.925+0000 7f39038c6640 1 -- 172.21.15.67:0/2822616076 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38fc194d90 con 0x7f38fc1084b0 2024-09-21T06:48:16.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.926+0000 7f39038c6640 1 -- 172.21.15.67:0/2822616076 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38fc1034a0 con 0x7f38fc1084b0 2024-09-21T06:48:16.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.930+0000 7f38f27fc640 1 -- 172.21.15.67:0/2822616076 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f38ec018020 con 0x7f38fc1084b0 2024-09-21T06:48:16.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.931+0000 7f38f27fc640 1 --2- 172.21.15.67:0/2822616076 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f38e80b6640 0x7f38e80b8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:16.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.932+0000 7f3901662640 1 --2- 172.21.15.67:0/2822616076 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f38e80b6640 0x7f38e80b8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:16.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.932+0000 7f38f27fc640 1 -- 172.21.15.67:0/2822616076 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f38ec010e60 con 0x7f38fc1084b0 2024-09-21T06:48:16.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.933+0000 7f3901662640 1 --2- 172.21.15.67:0/2822616076 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f38e80b6640 0x7f38e80b8b50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f38fc10ef10 tx=0x7f38f801a270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:16.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:15.938+0000 7f38f27fc640 1 -- 172.21.15.67:0/2822616076 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f38ec01f020 con 0x7f38fc1084b0 2024-09-21T06:48:16.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:16.362+0000 7f39038c6640 1 -- 172.21.15.67:0/2822616076 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38fc10b790 con 0x7f38fc1084b0 2024-09-21T06:48:16.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:16.363+0000 7f38f27fc640 1 -- 172.21.15.67:0/2822616076 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7f38ec01f020 con 0x7f38fc1084b0 2024-09-21T06:48:16.549 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:48:16.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:16.366+0000 7f39038c6640 1 -- 172.21.15.67:0/2822616076 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f38e80b6640 msgr2=0x7f38e80b8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:16.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:16.366+0000 7f39038c6640 1 --2- 172.21.15.67:0/2822616076 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f38e80b6640 0x7f38e80b8b50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f38fc10ef10 tx=0x7f38f801a270 comp rx=0 tx=0).stop 2024-09-21T06:48:16.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:16.366+0000 7f39038c6640 1 -- 172.21.15.67:0/2822616076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38fc1084b0 msgr2=0x7f38fc10a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:16.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:16.366+0000 7f39038c6640 1 --2- 172.21.15.67:0/2822616076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38fc1084b0 0x7f38fc10a9d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00b410 tx=0x7f38ec00b8d0 comp rx=0 tx=0).stop 2024-09-21T06:48:16.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:16.366+0000 7f39038c6640 1 -- 172.21.15.67:0/2822616076 shutdown_connections 2024-09-21T06:48:16.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:16.366+0000 7f39038c6640 1 --2- 172.21.15.67:0/2822616076 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f38e80b6640 0x7f38e80b8b50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:16.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:16.366+0000 7f39038c6640 1 --2- 172.21.15.67:0/2822616076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38fc1a1e80 0x7f38fc1a42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:16.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:16.366+0000 7f39038c6640 1 --2- 172.21.15.67:0/2822616076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38fc1084b0 0x7f38fc10a9d0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:16.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:16.366+0000 7f39038c6640 1 --2- 172.21.15.67:0/2822616076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38fc102270 0x7f38fc10a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:16.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:16.366+0000 7f39038c6640 1 -- 172.21.15.67:0/2822616076 >> 172.21.15.67:0/2822616076 conn(0x7f38fc077cc0 msgr2=0x7f38fc105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:16.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:16.366+0000 7f39038c6640 1 -- 172.21.15.67:0/2822616076 shutdown_connections 2024-09-21T06:48:16.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:16.366+0000 7f39038c6640 1 -- 172.21.15.67:0/2822616076 wait complete. 2024-09-21T06:48:16.975 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:16 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2822616076' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:48:16.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:16 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2822616076' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:48:17.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:16 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2822616076' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:48:17.554 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db --cancel --all' 2024-09-21T06:48:17.554 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24671 quiesce db --cancel --all 2024-09-21T06:48:17.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:17 smithi067 ceph-mon[31232]: pgmap v711: 129 pgs: 129 active+clean; 7.7 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 22 MiB/s wr, 532 op/s 2024-09-21T06:48:17.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:17 smithi203 ceph-mon[33735]: pgmap v711: 129 pgs: 129 active+clean; 7.7 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 22 MiB/s wr, 532 op/s 2024-09-21T06:48:18.038 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:48:18.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:17 smithi100 ceph-mon[33401]: pgmap v711: 129 pgs: 129 active+clean; 7.7 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 22 MiB/s wr, 532 op/s 2024-09-21T06:48:18.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.589+0000 7f9dfdfa0640 1 -- 172.21.15.67:0/1419156757 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9df8102a60 msgr2=0x7f9df8102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:18.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.589+0000 7f9dfdfa0640 1 --2- 172.21.15.67:0/1419156757 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9df8102a60 0x7f9df8102e70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f9de8009900 tx=0x7f9de802a410 comp rx=0 tx=0).stop 2024-09-21T06:48:18.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.589+0000 7f9dfdfa0640 1 -- 172.21.15.67:0/1419156757 shutdown_connections 2024-09-21T06:48:18.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.589+0000 7f9dfdfa0640 1 --2- 172.21.15.67:0/1419156757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9df8108cf0 0x7f9df810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.589+0000 7f9dfdfa0640 1 --2- 172.21.15.67:0/1419156757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9df81083a0 0x7f9df81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.589+0000 7f9dfdfa0640 1 --2- 172.21.15.67:0/1419156757 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9df8102a60 0x7f9df8102e70 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.589+0000 7f9dfdfa0640 1 -- 172.21.15.67:0/1419156757 >> 172.21.15.67:0/1419156757 conn(0x7f9df8079730 msgr2=0x7f9df8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:18.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.589+0000 7f9dfdfa0640 1 -- 172.21.15.67:0/1419156757 shutdown_connections 2024-09-21T06:48:18.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.589+0000 7f9dfdfa0640 1 -- 172.21.15.67:0/1419156757 wait complete. 2024-09-21T06:48:18.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.590+0000 7f9dfdfa0640 1 Processor -- start 2024-09-21T06:48:18.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.590+0000 7f9dfdfa0640 1 -- start start 2024-09-21T06:48:18.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.590+0000 7f9dfdfa0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9df8102a60 0x7f9df819c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:18.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.590+0000 7f9dfdfa0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9df81083a0 0x7f9df819cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:18.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.590+0000 7f9dfdfa0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9df8108cf0 0x7f9df81a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:18.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.590+0000 7f9dfdfa0640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9df8113960 con 0x7f9df8108cf0 2024-09-21T06:48:18.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.590+0000 7f9dfdfa0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9df81137e0 con 0x7f9df8102a60 2024-09-21T06:48:18.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.590+0000 7f9dfdfa0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9df8113ae0 con 0x7f9df81083a0 2024-09-21T06:48:18.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.590+0000 7f9df7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9df8108cf0 0x7f9df81a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:18.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.591+0000 7f9df7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9df8108cf0 0x7f9df81a1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54576/0 (socket says 172.21.15.67:54576) 2024-09-21T06:48:18.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.591+0000 7f9df7fff640 1 -- 172.21.15.67:0/2377334906 learned_addr learned my addr 172.21.15.67:0/2377334906 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:48:18.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.591+0000 7f9df6ffd640 1 --2- 172.21.15.67:0/2377334906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9df81083a0 0x7f9df819cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:18.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.591+0000 7f9df77fe640 1 --2- 172.21.15.67:0/2377334906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9df8102a60 0x7f9df819c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:18.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.591+0000 7f9df7fff640 1 -- 172.21.15.67:0/2377334906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9df81083a0 msgr2=0x7f9df819cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:18.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.591+0000 7f9df7fff640 1 --2- 172.21.15.67:0/2377334906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9df81083a0 0x7f9df819cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.591+0000 7f9df7fff640 1 -- 172.21.15.67:0/2377334906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9df8102a60 msgr2=0x7f9df819c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:18.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.591+0000 7f9df7fff640 1 --2- 172.21.15.67:0/2377334906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9df8102a60 0x7f9df819c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.591+0000 7f9df7fff640 1 -- 172.21.15.67:0/2377334906 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df81a2440 con 0x7f9df8108cf0 2024-09-21T06:48:18.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.591+0000 7f9df77fe640 1 --2- 172.21.15.67:0/2377334906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9df8102a60 0x7f9df819c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:48:18.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.591+0000 7f9df7fff640 1 --2- 172.21.15.67:0/2377334906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9df8108cf0 0x7f9df81a1dd0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9df000cce0 tx=0x7f9df000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:18.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.591+0000 7f9df4ff9640 1 -- 172.21.15.67:0/2377334906 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df000f450 con 0x7f9df8108cf0 2024-09-21T06:48:18.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.591+0000 7f9df4ff9640 1 -- 172.21.15.67:0/2377334906 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9df000fa80 con 0x7f9df8108cf0 2024-09-21T06:48:18.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.591+0000 7f9dfdfa0640 1 -- 172.21.15.67:0/2377334906 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9df81a2730 con 0x7f9df8108cf0 2024-09-21T06:48:18.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.591+0000 7f9df4ff9640 1 -- 172.21.15.67:0/2377334906 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df0018ad0 con 0x7f9df8108cf0 2024-09-21T06:48:18.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.591+0000 7f9dfdfa0640 1 -- 172.21.15.67:0/2377334906 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9df81a8f50 con 0x7f9df8108cf0 2024-09-21T06:48:18.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.594+0000 7f9e04514740 1 Processor -- start 2024-09-21T06:48:18.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.594+0000 7f9df4ff9640 1 -- 172.21.15.67:0/2377334906 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f9df00213f0 con 0x7f9df8108cf0 2024-09-21T06:48:18.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.594+0000 7f9e04514740 1 -- start start 2024-09-21T06:48:18.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.594+0000 7f9e04514740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc450a52d0 0x55fc4544c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:18.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.594+0000 7f9e04514740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fc4507c410 0x55fc4544c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:18.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.594+0000 7f9e04514740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc450845f0 0x55fc4544cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:18.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.594+0000 7f9e04514740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55fc453c2dd0 con 0x55fc450845f0 2024-09-21T06:48:18.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.594+0000 7f9e04514740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55fc453b2200 con 0x55fc450a52d0 2024-09-21T06:48:18.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.594+0000 7f9e04514740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55fc453b6010 con 0x55fc4507c410 2024-09-21T06:48:18.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.595+0000 7f9df6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc450845f0 0x55fc4544cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:18.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.595+0000 7f9df6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc450845f0 0x55fc4544cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54578/0 (socket says 172.21.15.67:54578) 2024-09-21T06:48:18.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.595+0000 7f9df6ffd640 1 -- 172.21.15.67:0/2016504040 learned_addr learned my addr 172.21.15.67:0/2016504040 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:48:18.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.595+0000 7f9df6ffd640 1 --2- 172.21.15.67:0/2016504040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc450a52d0 0x55fc4544c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:18.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.595+0000 7f9df77fe640 1 --2- 172.21.15.67:0/2016504040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fc4507c410 0x55fc4544c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:18.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.595+0000 7f9df4ff9640 1 --2- 172.21.15.67:0/2377334906 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9de00b6540 0x7f9de00b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:18.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.595+0000 7f9df6ffd640 1 -- 172.21.15.67:0/2016504040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fc4507c410 msgr2=0x55fc4544c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:48:18.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.595+0000 7f9df6ffd640 1 --2- 172.21.15.67:0/2016504040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fc4507c410 0x55fc4544c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.595+0000 7f9df6ffd640 1 -- 172.21.15.67:0/2016504040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc450a52d0 msgr2=0x55fc4544c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:48:18.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.595+0000 7f9df6ffd640 1 --2- 172.21.15.67:0/2016504040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc450a52d0 0x55fc4544c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.595+0000 7f9df6ffd640 1 -- 172.21.15.67:0/2016504040 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc451108b0 con 0x55fc450845f0 2024-09-21T06:48:18.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.595+0000 7f9df77fe640 1 --2- 172.21.15.67:0/2016504040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fc4507c410 0x55fc4544c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:48:18.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.595+0000 7f9df7fff640 1 --2- 172.21.15.67:0/2377334906 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9de00b6540 0x7f9de00b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:18.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.595+0000 7f9df4ff9640 1 -- 172.21.15.67:0/2377334906 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f9df00e7f00 con 0x7f9df8108cf0 2024-09-21T06:48:18.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.595+0000 7f9df6ffd640 1 --2- 172.21.15.67:0/2016504040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc450845f0 0x55fc4544cd90 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9dec012770 tx=0x7f9dec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67c1db9f316d8ba2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:18.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.596+0000 7f9dddffb640 1 -- 172.21.15.67:0/2016504040 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dec012cf0 con 0x55fc450845f0 2024-09-21T06:48:18.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.596+0000 7f9dddffb640 1 -- 172.21.15.67:0/2016504040 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9dec0044c0 con 0x55fc450845f0 2024-09-21T06:48:18.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.596+0000 7f9dddffb640 1 -- 172.21.15.67:0/2016504040 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dec0035f0 con 0x55fc450845f0 2024-09-21T06:48:18.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.596+0000 7f9e04514740 1 -- 172.21.15.67:0/2016504040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc450845f0 msgr2=0x55fc4544cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:18.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.596+0000 7f9df7fff640 1 --2- 172.21.15.67:0/2377334906 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9de00b6540 0x7f9de00b8a50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9df00aaf30 tx=0x7f9df00a2570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:18.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.596+0000 7f9e04514740 1 --2- 172.21.15.67:0/2016504040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc450845f0 0x55fc4544cd90 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9dec012770 tx=0x7f9dec004730 comp rx=0 tx=0).stop 2024-09-21T06:48:18.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.597+0000 7f9e04514740 1 -- 172.21.15.67:0/2016504040 shutdown_connections 2024-09-21T06:48:18.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.597+0000 7f9e04514740 1 --2- 172.21.15.67:0/2016504040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc450845f0 0x55fc4544cd90 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.597+0000 7f9e04514740 1 --2- 172.21.15.67:0/2016504040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fc4507c410 0x55fc4544c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.597+0000 7f9e04514740 1 --2- 172.21.15.67:0/2016504040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc450a52d0 0x55fc4544c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.597+0000 7f9e04514740 1 -- 172.21.15.67:0/2016504040 >> 172.21.15.67:0/2016504040 conn(0x55fc450996c0 msgr2=0x55fc4544a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:18.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.597+0000 7f9e04514740 1 -- 172.21.15.67:0/2016504040 shutdown_connections 2024-09-21T06:48:18.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.597+0000 7f9e04514740 1 -- 172.21.15.67:0/2016504040 wait complete. 2024-09-21T06:48:18.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.597+0000 7f9e04514740 1 -- start start 2024-09-21T06:48:18.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.597+0000 7f9e04514740 1 Processor -- start 2024-09-21T06:48:18.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.598+0000 7f9e04514740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fc4507c410 0x55fc45457f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:18.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.598+0000 7f9e04514740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc450845f0 0x55fc45458480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:18.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.598+0000 7f9e04514740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc450a52d0 0x55fc45463e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:18.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.598+0000 7f9e04514740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55fc45213810 con 0x55fc450845f0 2024-09-21T06:48:18.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.598+0000 7f9e04514740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55fc4534d080 con 0x55fc450a52d0 2024-09-21T06:48:18.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.598+0000 7f9e04514740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55fc4545aaf0 con 0x55fc4507c410 2024-09-21T06:48:18.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.598+0000 7f9e04514740 10 client.0 _ll_register_callbacks cb 0x55fc4535d960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:48:18.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.598+0000 7f9df77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc450845f0 0x55fc45458480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:18.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.598+0000 7f9df77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc450845f0 0x55fc45458480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54584/0 (socket says 172.21.15.67:54584) 2024-09-21T06:48:18.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.598+0000 7f9df77fe640 1 -- 172.21.15.67:0/911982533 learned_addr learned my addr 172.21.15.67:0/911982533 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:48:18.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.598+0000 7f9df6ffd640 1 --2- 172.21.15.67:0/911982533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc450a52d0 0x55fc45463e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:18.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.598+0000 7f9df6ffd640 1 --2- 172.21.15.67:0/911982533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fc4507c410 0x55fc45457f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:18.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.598+0000 7f9df77fe640 1 -- 172.21.15.67:0/911982533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fc4507c410 msgr2=0x55fc45457f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:48:18.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.598+0000 7f9df77fe640 1 --2- 172.21.15.67:0/911982533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fc4507c410 0x55fc45457f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.598+0000 7f9df77fe640 1 -- 172.21.15.67:0/911982533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc450a52d0 msgr2=0x55fc45463e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:48:18.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.598+0000 7f9df77fe640 1 --2- 172.21.15.67:0/911982533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc450a52d0 0x55fc45463e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.598+0000 7f9df77fe640 1 -- 172.21.15.67:0/911982533 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc45357460 con 0x55fc450845f0 2024-09-21T06:48:18.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.598+0000 7f9df77fe640 1 --2- 172.21.15.67:0/911982533 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc450845f0 0x55fc45458480 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f9de802aa00 tx=0x7f9de80052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb4c9f02f9a8b3f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:18.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.598+0000 7f9dd6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:48:18.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.599+0000 7f9dd6ffd640 1 -- 172.21.15.67:0/911982533 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de8035030 con 0x55fc450845f0 2024-09-21T06:48:18.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.599+0000 7f9e04514740 1 -- 172.21.15.67:0/911982533 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fc453c4d20 con 0x55fc450845f0 2024-09-21T06:48:18.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.599+0000 7f9dd6ffd640 1 -- 172.21.15.67:0/911982533 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f9de80048c0 con 0x55fc450845f0 2024-09-21T06:48:18.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.599+0000 7f9dd6ffd640 1 -- 172.21.15.67:0/911982533 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9de804d050 con 0x55fc450845f0 2024-09-21T06:48:18.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.599+0000 7f9e04514740 10 client.15921 fetch_fsmap learned FSMap version 21 2024-09-21T06:48:18.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.599+0000 7f9e04514740 1 -- 172.21.15.67:0/911982533 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55fc45466bd0 con 0x55fc450845f0 2024-09-21T06:48:18.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.599+0000 7f9dd6ffd640 1 -- 172.21.15.67:0/911982533 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55fc453c4d20 con 0x55fc450845f0 2024-09-21T06:48:18.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.599+0000 7f9dd6ffd640 1 -- 172.21.15.67:0/911982533 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f9de8003ce0 con 0x55fc450845f0 2024-09-21T06:48:18.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.599+0000 7f9e04514740 10 client.15921 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:48:18.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.599+0000 7f9e04514740 10 client.15921 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:48:18.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.599+0000 7f9e04514740 20 client.15921 populate_metadata read hostname 'smithi067' 2024-09-21T06:48:18.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.599+0000 7f9e04514740 1 --2- 172.21.15.67:0/911982533 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55fc45063c50 0x55fc4546a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:18.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.599+0000 7f9e04514740 4 client.15921 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:48:18.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.599+0000 7f9e04514740 1 -- 172.21.15.67:0/911982533 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc453c07d0 con 0x55fc45063c50 2024-09-21T06:48:18.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.600+0000 7f9df6ffd640 1 --2- 172.21.15.67:0/911982533 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55fc45063c50 0x55fc4546a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:18.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.600+0000 7f9df6ffd640 1 --2- 172.21.15.67:0/911982533 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55fc45063c50 0x55fc4546a070 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cafc139faaebab4e server_cookie=b0f299f0cae9955c in_seq=0 out_seq=0 2024-09-21T06:48:18.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.600+0000 7f9dd6ffd640 10 client.15921 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:48:18.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.601+0000 7f9dd6ffd640 1 -- 172.21.15.67:0/911982533 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fc453c07d0 con 0x55fc45063c50 2024-09-21T06:48:18.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.601+0000 7f9dd6ffd640 10 client.15921 handle_command_reply: tid=0 2024-09-21T06:48:18.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.601+0000 7f9e04514740 1 client.15921 shutdown 2024-09-21T06:48:18.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.601+0000 7f9e04514740 1 -- 172.21.15.67:0/911982533 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc450845f0 msgr2=0x55fc45458480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:18.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.601+0000 7f9e04514740 1 --2- 172.21.15.67:0/911982533 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc450845f0 0x55fc45458480 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f9de802aa00 tx=0x7f9de80052e0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.601+0000 7f9e04514740 1 -- 172.21.15.67:0/911982533 shutdown_connections 2024-09-21T06:48:18.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.601+0000 7f9e04514740 1 --2- 172.21.15.67:0/911982533 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55fc45063c50 0x55fc4546a070 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.601+0000 7f9e04514740 1 --2- 172.21.15.67:0/911982533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc450a52d0 0x55fc45463e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.601+0000 7f9dd6ffd640 0 client.15921 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:48:18.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.601+0000 7f9e04514740 1 --2- 172.21.15.67:0/911982533 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc450845f0 0x55fc45458480 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.601+0000 7f9e04514740 1 --2- 172.21.15.67:0/911982533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fc4507c410 0x55fc45457f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.601+0000 7f9e04514740 1 -- 172.21.15.67:0/911982533 >> 172.21.15.67:0/911982533 conn(0x55fc450996c0 msgr2=0x55fc4544f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:18.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.601+0000 7f9e04514740 1 -- 172.21.15.67:0/911982533 shutdown_connections 2024-09-21T06:48:18.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.601+0000 7f9e04514740 1 -- 172.21.15.67:0/911982533 wait complete. 2024-09-21T06:48:18.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.602+0000 7f9e04514740 20 client.15921 trim_cache size 0 max 16384 2024-09-21T06:48:18.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.619+0000 7f9e04514740 1 Processor -- start 2024-09-21T06:48:18.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.619+0000 7f9e04514740 1 -- start start 2024-09-21T06:48:18.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.619+0000 7f9e04514740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc45063c50 0x55fc45463d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:18.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.619+0000 7f9e04514740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4507c410 0x55fc45464250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:18.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.619+0000 7f9e04514740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fc450845f0 0x55fc45454ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:18.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.619+0000 7f9e04514740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9df81b0b20 con 0x55fc45063c50 2024-09-21T06:48:18.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.619+0000 7f9e04514740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9df81b33a0 con 0x55fc4507c410 2024-09-21T06:48:18.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.619+0000 7f9e04514740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9df81b1190 con 0x55fc450845f0 2024-09-21T06:48:18.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.619+0000 7f9df6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc45063c50 0x55fc45463d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:18.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.619+0000 7f9df6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc45063c50 0x55fc45463d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54592/0 (socket says 172.21.15.67:54592) 2024-09-21T06:48:18.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.619+0000 7f9df6ffd640 1 -- 172.21.15.67:0/225986174 learned_addr learned my addr 172.21.15.67:0/225986174 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:48:18.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.619+0000 7f9df77fe640 1 --2- 172.21.15.67:0/225986174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4507c410 0x55fc45464250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:18.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.619+0000 7f9df6ffd640 1 --2- 172.21.15.67:0/225986174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fc450845f0 0x55fc45454ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:18.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.619+0000 7f9df6ffd640 1 -- 172.21.15.67:0/225986174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fc450845f0 msgr2=0x55fc45454ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:48:18.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.619+0000 7f9df6ffd640 1 --2- 172.21.15.67:0/225986174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fc450845f0 0x55fc45454ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.619+0000 7f9df6ffd640 1 -- 172.21.15.67:0/225986174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4507c410 msgr2=0x55fc45464250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:48:18.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.619+0000 7f9df6ffd640 1 --2- 172.21.15.67:0/225986174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4507c410 0x55fc45464250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.620+0000 7f9df6ffd640 1 -- 172.21.15.67:0/225986174 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc4544e600 con 0x55fc45063c50 2024-09-21T06:48:18.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.620+0000 7f9df6ffd640 1 --2- 172.21.15.67:0/225986174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc45063c50 0x55fc45463d10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9dec012740 tx=0x7f9dec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf6708cc4d823ed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:18.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.620+0000 7f9ddd7fa640 1 -- 172.21.15.67:0/225986174 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dec010740 con 0x55fc45063c50 2024-09-21T06:48:18.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.620+0000 7f9ddd7fa640 1 -- 172.21.15.67:0/225986174 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9dec016040 con 0x55fc45063c50 2024-09-21T06:48:18.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.620+0000 7f9ddd7fa640 1 -- 172.21.15.67:0/225986174 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dec036b60 con 0x55fc45063c50 2024-09-21T06:48:18.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.621+0000 7f9e04514740 1 -- 172.21.15.67:0/225986174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc45063c50 msgr2=0x55fc45463d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:18.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.621+0000 7f9e04514740 1 --2- 172.21.15.67:0/225986174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc45063c50 0x55fc45463d10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9dec012740 tx=0x7f9dec004730 comp rx=0 tx=0).stop 2024-09-21T06:48:18.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.621+0000 7f9e04514740 1 -- 172.21.15.67:0/225986174 shutdown_connections 2024-09-21T06:48:18.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.621+0000 7f9e04514740 1 --2- 172.21.15.67:0/225986174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4507c410 0x55fc45464250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.621+0000 7f9e04514740 1 --2- 172.21.15.67:0/225986174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fc450845f0 0x55fc45454ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.621+0000 7f9e04514740 1 --2- 172.21.15.67:0/225986174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc45063c50 0x55fc45463d10 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.621+0000 7f9e04514740 1 -- 172.21.15.67:0/225986174 >> 172.21.15.67:0/225986174 conn(0x55fc450996c0 msgr2=0x55fc45459220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:18.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.621+0000 7f9e04514740 1 -- 172.21.15.67:0/225986174 shutdown_connections 2024-09-21T06:48:18.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.621+0000 7f9e04514740 1 -- 172.21.15.67:0/225986174 wait complete. 2024-09-21T06:48:18.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.621+0000 7f9e04514740 1 -- start start 2024-09-21T06:48:18.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.621+0000 7f9e04514740 1 Processor -- start 2024-09-21T06:48:18.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.622+0000 7f9e04514740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc45063c50 0x55fc45460650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:18.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.622+0000 7f9e04514740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4507c410 0x55fc4545cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:18.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.622+0000 7f9e04514740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fc450845f0 0x55fc4545d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:18.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.622+0000 7f9e04514740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9df81b1860 con 0x55fc45063c50 2024-09-21T06:48:18.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.622+0000 7f9e04514740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9df81b2cd0 con 0x55fc4507c410 2024-09-21T06:48:18.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.622+0000 7f9e04514740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9df81b1f30 con 0x55fc450845f0 2024-09-21T06:48:18.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.622+0000 7f9e04514740 10 client.0 _ll_register_callbacks cb 0x7f9de80045b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:48:18.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.622+0000 7f9df6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc45063c50 0x55fc45460650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:18.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.622+0000 7f9df6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc45063c50 0x55fc45460650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54598/0 (socket says 172.21.15.67:54598) 2024-09-21T06:48:18.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.622+0000 7f9df6ffd640 1 -- 172.21.15.67:0/656447532 learned_addr learned my addr 172.21.15.67:0/656447532 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:48:18.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.622+0000 7f9df77fe640 1 --2- 172.21.15.67:0/656447532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4507c410 0x55fc4545cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:18.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.622+0000 7f9df6ffd640 1 --2- 172.21.15.67:0/656447532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fc450845f0 0x55fc4545d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:18.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.622+0000 7f9df6ffd640 1 -- 172.21.15.67:0/656447532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fc450845f0 msgr2=0x55fc4545d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:48:18.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.622+0000 7f9df6ffd640 1 --2- 172.21.15.67:0/656447532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fc450845f0 0x55fc4545d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.622+0000 7f9df6ffd640 1 -- 172.21.15.67:0/656447532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4507c410 msgr2=0x55fc4545cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:48:18.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.622+0000 7f9df6ffd640 1 --2- 172.21.15.67:0/656447532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4507c410 0x55fc4545cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.622+0000 7f9df6ffd640 1 -- 172.21.15.67:0/656447532 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc4545d8d0 con 0x55fc45063c50 2024-09-21T06:48:18.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.622+0000 7f9df6ffd640 1 --2- 172.21.15.67:0/656447532 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc45063c50 0x55fc45460650 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9dec00b450 tx=0x7f9dec012f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06a119985ae0924 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:18.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.622+0000 7f9dd6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-21T06:48:18.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.623+0000 7f9dd6ffd640 1 -- 172.21.15.67:0/656447532 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dec036480 con 0x55fc45063c50 2024-09-21T06:48:18.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.623+0000 7f9dd6ffd640 1 -- 172.21.15.67:0/656447532 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f9dec036620 con 0x55fc45063c50 2024-09-21T06:48:18.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.623+0000 7f9e04514740 1 -- 172.21.15.67:0/656447532 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dec016040 con 0x55fc45063c50 2024-09-21T06:48:18.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.623+0000 7f9dd6ffd640 1 -- 172.21.15.67:0/656447532 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9dec036820 con 0x55fc45063c50 2024-09-21T06:48:18.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.623+0000 7f9e04514740 10 client.15927 fetch_fsmap learned FSMap version 21 2024-09-21T06:48:18.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.623+0000 7f9e04514740 1 -- 172.21.15.67:0/656447532 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55fc454d5990 con 0x55fc45063c50 2024-09-21T06:48:18.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.623+0000 7f9dd6ffd640 1 -- 172.21.15.67:0/656447532 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9dec016040 con 0x55fc45063c50 2024-09-21T06:48:18.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.623+0000 7f9dd6ffd640 1 -- 172.21.15.67:0/656447532 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f9dec01f990 con 0x55fc45063c50 2024-09-21T06:48:18.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.623+0000 7f9e04514740 10 client.15927 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:48:18.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.623+0000 7f9e04514740 10 client.15927 resolve_mds: validated gid 24671 aka daemon mds.b 2024-09-21T06:48:18.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.623+0000 7f9e04514740 20 client.15927 populate_metadata read hostname 'smithi067' 2024-09-21T06:48:18.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.623+0000 7f9e04514740 1 --2- 172.21.15.67:0/656447532 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55fc450a52d0 0x55fc454d7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:18.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.623+0000 7f9e04514740 4 client.15927 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-21T06:48:18.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.623+0000 7f9e04514740 1 -- 172.21.15.67:0/656447532 --> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55fc453bdc00 con 0x55fc450a52d0 2024-09-21T06:48:18.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.624+0000 7f9df77fe640 1 --2- 172.21.15.67:0/656447532 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55fc450a52d0 0x55fc454d7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:18.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.624+0000 7f9dd6ffd640 10 client.15927 ms_handle_connect on v2:172.21.15.100:6838/778060936 2024-09-21T06:48:18.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.624+0000 7f9df77fe640 1 --2- 172.21.15.67:0/656447532 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55fc450a52d0 0x55fc454d7ec0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dbb365fe4f23e7e server_cookie=b0ebf9ea651bfff4 in_seq=0 out_seq=0 2024-09-21T06:48:18.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.624+0000 7f9dd6ffd640 1 -- 172.21.15.67:0/656447532 <== mds.0 v2:172.21.15.100:6838/778060936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55fc453bdc00 con 0x55fc450a52d0 2024-09-21T06:48:18.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.624+0000 7f9dd6ffd640 10 client.15927 handle_command_reply: tid=0 2024-09-21T06:48:18.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.624+0000 7f9e04514740 1 client.15927 shutdown 2024-09-21T06:48:18.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.624+0000 7f9e04514740 1 -- 172.21.15.67:0/656447532 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc45063c50 msgr2=0x55fc45460650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:18.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.624+0000 7f9e04514740 1 --2- 172.21.15.67:0/656447532 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc45063c50 0x55fc45460650 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9dec00b450 tx=0x7f9dec012f70 comp rx=0 tx=0).stop 2024-09-21T06:48:18.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.625+0000 7f9e04514740 1 -- 172.21.15.67:0/656447532 shutdown_connections 2024-09-21T06:48:18.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.625+0000 7f9e04514740 1 --2- 172.21.15.67:0/656447532 >> [v2:172.21.15.100:6838/778060936,v1:172.21.15.100:6839/778060936] conn(0x55fc450a52d0 0x55fc454d7ec0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.625+0000 7f9e04514740 1 --2- 172.21.15.67:0/656447532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4507c410 0x55fc4545cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.625+0000 7f9e04514740 1 --2- 172.21.15.67:0/656447532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fc450845f0 0x55fc4545d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.625+0000 7f9dd6ffd640 0 client.15927 ms_handle_reset on v2:172.21.15.100:6838/778060936 2024-09-21T06:48:18.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.625+0000 7f9e04514740 1 --2- 172.21.15.67:0/656447532 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc45063c50 0x55fc45460650 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.625+0000 7f9e04514740 1 -- 172.21.15.67:0/656447532 >> 172.21.15.67:0/656447532 conn(0x55fc450996c0 msgr2=0x55fc45456580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:18.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.625+0000 7f9e04514740 1 -- 172.21.15.67:0/656447532 shutdown_connections 2024-09-21T06:48:18.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.625+0000 7f9e04514740 1 -- 172.21.15.67:0/656447532 wait complete. 2024-09-21T06:48:18.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.625+0000 7f9e04514740 20 client.15927 trim_cache size 0 max 16384 2024-09-21T06:48:18.632 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-21T06:48:18.632 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 21, 2024-09-21T06:48:18.632 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-09-21T06:48:18.632 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 44, 2024-09-21T06:48:18.632 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": {} 2024-09-21T06:48:18.632 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-21T06:48:18.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.625+0000 7f9dfdfa0640 1 -- 172.21.15.67:0/2377334906 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9de00b6540 msgr2=0x7f9de00b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:18.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.625+0000 7f9dfdfa0640 1 --2- 172.21.15.67:0/2377334906 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9de00b6540 0x7f9de00b8a50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9df00aaf30 tx=0x7f9df00a2570 comp rx=0 tx=0).stop 2024-09-21T06:48:18.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.625+0000 7f9dfdfa0640 1 -- 172.21.15.67:0/2377334906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9df8108cf0 msgr2=0x7f9df81a1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:18.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.625+0000 7f9dfdfa0640 1 --2- 172.21.15.67:0/2377334906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9df8108cf0 0x7f9df81a1dd0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9df000cce0 tx=0x7f9df000a620 comp rx=0 tx=0).stop 2024-09-21T06:48:18.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.626+0000 7f9dfdfa0640 1 -- 172.21.15.67:0/2377334906 shutdown_connections 2024-09-21T06:48:18.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.626+0000 7f9dfdfa0640 1 --2- 172.21.15.67:0/2377334906 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9de00b6540 0x7f9de00b8a50 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.626+0000 7f9dfdfa0640 1 --2- 172.21.15.67:0/2377334906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9df8108cf0 0x7f9df81a1dd0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.626+0000 7f9dfdfa0640 1 --2- 172.21.15.67:0/2377334906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9df81083a0 0x7f9df819cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.626+0000 7f9dfdfa0640 1 --2- 172.21.15.67:0/2377334906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9df8102a60 0x7f9df819c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:18.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.626+0000 7f9dfdfa0640 1 -- 172.21.15.67:0/2377334906 >> 172.21.15.67:0/2377334906 conn(0x7f9df8079730 msgr2=0x7f9df81012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:18.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.626+0000 7f9dfdfa0640 1 -- 172.21.15.67:0/2377334906 shutdown_connections 2024-09-21T06:48:18.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:18.626+0000 7f9dfdfa0640 1 -- 172.21.15.67:0/2377334906 wait complete. 2024-09-21T06:48:19.356 INFO:tasks.quiescer:done joining Quiescers 2024-09-21T06:48:19.356 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-21T06:48:19.366 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:48:19.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.603+0000 7febdfc27640 1 -- 172.21.15.67:0/3128452205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febd8103fa0 msgr2=0x7febd810a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:19.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.603+0000 7febdfc27640 1 --2- 172.21.15.67:0/3128452205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febd8103fa0 0x7febd810a940 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7febd0009ca0 tx=0x7febd002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:48:19.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.604+0000 7febdfc27640 1 -- 172.21.15.67:0/3128452205 shutdown_connections 2024-09-21T06:48:19.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.604+0000 7febdfc27640 1 --2- 172.21.15.67:0/3128452205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febd8103fa0 0x7febd810a940 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:19.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.604+0000 7febdfc27640 1 --2- 172.21.15.67:0/3128452205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febd8103600 0x7febd8103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:19.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.604+0000 7febdfc27640 1 --2- 172.21.15.67:0/3128452205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febd8102580 0x7febd81029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:19.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.604+0000 7febdfc27640 1 -- 172.21.15.67:0/3128452205 >> 172.21.15.67:0/3128452205 conn(0x7febd80fdee0 msgr2=0x7febd8100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:19.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.604+0000 7febdfc27640 1 -- 172.21.15.67:0/3128452205 shutdown_connections 2024-09-21T06:48:19.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.604+0000 7febdfc27640 1 -- 172.21.15.67:0/3128452205 wait complete. 2024-09-21T06:48:19.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.606+0000 7febdfc27640 1 Processor -- start 2024-09-21T06:48:19.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.607+0000 7febdfc27640 1 -- start start 2024-09-21T06:48:19.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.607+0000 7febdfc27640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febd8102580 0x7febd8198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:19.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.607+0000 7febdfc27640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febd8103600 0x7febd8199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:19.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.607+0000 7febdfc27640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febd8103fa0 0x7febd8199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:19.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.607+0000 7febdfc27640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7febd810e010 con 0x7febd8102580 2024-09-21T06:48:19.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.607+0000 7febdfc27640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7febd810de90 con 0x7febd8103600 2024-09-21T06:48:19.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.607+0000 7febdfc27640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7febd810e190 con 0x7febd8103fa0 2024-09-21T06:48:19.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.607+0000 7febdd9c3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febd8102580 0x7febd8198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:19.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.607+0000 7febdd9c3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febd8102580 0x7febd8198cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54628/0 (socket says 172.21.15.67:54628) 2024-09-21T06:48:19.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.607+0000 7febdd9c3640 1 -- 172.21.15.67:0/2284860709 learned_addr learned my addr 172.21.15.67:0/2284860709 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:48:19.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.607+0000 7febdd1c2640 1 --2- 172.21.15.67:0/2284860709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febd8103600 0x7febd8199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:19.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.607+0000 7febde1c4640 1 --2- 172.21.15.67:0/2284860709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febd8103fa0 0x7febd8199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:19.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.607+0000 7febdd9c3640 1 -- 172.21.15.67:0/2284860709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febd8103fa0 msgr2=0x7febd8199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:19.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.607+0000 7febdd9c3640 1 --2- 172.21.15.67:0/2284860709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febd8103fa0 0x7febd8199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:19.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.607+0000 7febdd9c3640 1 -- 172.21.15.67:0/2284860709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febd8103600 msgr2=0x7febd8199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:19.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.607+0000 7febdd9c3640 1 --2- 172.21.15.67:0/2284860709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febd8103600 0x7febd8199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:19.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.607+0000 7febdd9c3640 1 -- 172.21.15.67:0/2284860709 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd819a710 con 0x7febd8102580 2024-09-21T06:48:19.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.608+0000 7febdd9c3640 1 --2- 172.21.15.67:0/2284860709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febd8102580 0x7febd8198cd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7febd400e9f0 tx=0x7febd400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:19.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.608+0000 7febceffd640 1 -- 172.21.15.67:0/2284860709 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd400cd80 con 0x7febd8102580 2024-09-21T06:48:19.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.608+0000 7febceffd640 1 -- 172.21.15.67:0/2284860709 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7febd40108c0 con 0x7febd8102580 2024-09-21T06:48:19.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.608+0000 7febceffd640 1 -- 172.21.15.67:0/2284860709 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd40187c0 con 0x7febd8102580 2024-09-21T06:48:19.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.608+0000 7febdfc27640 1 -- 172.21.15.67:0/2284860709 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd819aa00 con 0x7febd8102580 2024-09-21T06:48:19.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.608+0000 7febdfc27640 1 -- 172.21.15.67:0/2284860709 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd81a10f0 con 0x7febd8102580 2024-09-21T06:48:19.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.609+0000 7febdfc27640 1 -- 172.21.15.67:0/2284860709 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febd819ac60 con 0x7febd8102580 2024-09-21T06:48:19.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.611+0000 7febceffd640 1 -- 172.21.15.67:0/2284860709 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7febd4010430 con 0x7febd8102580 2024-09-21T06:48:19.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.611+0000 7febceffd640 1 --2- 172.21.15.67:0/2284860709 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7febc40b6660 0x7febc40b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:19.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.612+0000 7febdd1c2640 1 --2- 172.21.15.67:0/2284860709 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7febc40b6660 0x7febc40b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:19.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.612+0000 7febceffd640 1 -- 172.21.15.67:0/2284860709 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7febd40e79f0 con 0x7febd8102580 2024-09-21T06:48:19.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.612+0000 7febdd1c2640 1 --2- 172.21.15.67:0/2284860709 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7febc40b6660 0x7febc40b8b70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7febc8005d00 tx=0x7febc800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:19.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.616+0000 7febceffd640 1 -- 172.21.15.67:0/2284860709 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7febd40adeb0 con 0x7febd8102580 2024-09-21T06:48:19.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.805+0000 7febdfc27640 1 -- 172.21.15.67:0/2284860709 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7febd802e9b0 con 0x7febd8102580 2024-09-21T06:48:19.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.807+0000 7febceffd640 1 -- 172.21.15.67:0/2284860709 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24954 (secure 0 0 0) 0x7febd40b2290 con 0x7febd8102580 2024-09-21T06:48:19.809 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:48:19.810 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":85,"fsid":"2a791768-77e2-11ef-baf5-efdc52797490","created":"2024-09-21T06:26:15.222283+0000","modified":"2024-09-21T06:36:05.617384+0000","last_up_change":"2024-09-21T06:33:40.886310+0000","last_in_change":"2024-09-21T06:33:17.339486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:29:50.212825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:34:29.082881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:34:31.301094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:34:37.681356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"18778364-82ed-41c0-92da-ce44cdad7358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6803","nonce":2310759946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6805","nonce":2310759946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6809","nonce":2310759946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6807","nonce":2310759946}]},"public_addr":"172.21.15.67:6803/2310759946","cluster_addr":"172.21.15.67:6805/2310759946","heartbeat_back_addr":"172.21.15.67:6809/2310759946","heartbeat_front_addr":"172.21.15.67:6807/2310759946","state":["exists","up"]},{"osd":1,"uuid":"84e8d0c2-1c45-4673-ad49-f61df667d7f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6801","nonce":161807364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6803","nonce":161807364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6807","nonce":161807364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6805","nonce":161807364}]},"public_addr":"172.21.15.100:6801/161807364","cluster_addr":"172.21.15.100:6803/161807364","heartbeat_back_addr":"172.21.15.100:6807/161807364","heartbeat_front_addr":"172.21.15.100:6805/161807364","state":["exists","up"]},{"osd":2,"uuid":"e9a4d919-a779-4a1b-9b5b-d03a65f28ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6801","nonce":2754787169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6803","nonce":2754787169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6807","nonce":2754787169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6805","nonce":2754787169}]},"public_addr":"172.21.15.203:6801/2754787169","cluster_addr":"172.21.15.203:6803/2754787169","heartbeat_back_addr":"172.21.15.203:6807/2754787169","heartbeat_front_addr":"172.21.15.203:6805/2754787169","state":["exists","up"]},{"osd":3,"uuid":"3f7d9abd-587c-47ad-bf43-11f7dcdf71c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6811","nonce":4007517951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6813","nonce":4007517951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6817","nonce":4007517951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6815","nonce":4007517951}]},"public_addr":"172.21.15.67:6811/4007517951","cluster_addr":"172.21.15.67:6813/4007517951","heartbeat_back_addr":"172.21.15.67:6817/4007517951","heartbeat_front_addr":"172.21.15.67:6815/4007517951","state":["exists","up"]},{"osd":4,"uuid":"94f3a075-6d8a-4907-8418-a8513b35779a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6809","nonce":3773646501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6811","nonce":3773646501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6815","nonce":3773646501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6813","nonce":3773646501}]},"public_addr":"172.21.15.100:6809/3773646501","cluster_addr":"172.21.15.100:6811/3773646501","heartbeat_back_addr":"172.21.15.100:6815/3773646501","heartbeat_front_addr":"172.21.15.100:6813/3773646501","state":["exists","up"]},{"osd":5,"uuid":"5d842edf-06ad-49ae-ac71-863e5657270c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6809","nonce":2096171166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6811","nonce":2096171166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6815","nonce":2096171166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6813","nonce":2096171166}]},"public_addr":"172.21.15.203:6809/2096171166","cluster_addr":"172.21.15.203:6811/2096171166","heartbeat_back_addr":"172.21.15.203:6815/2096171166","heartbeat_front_addr":"172.21.15.203:6813/2096171166","state":["exists","up"]},{"osd":6,"uuid":"85b9beed-c93d-48ba-bc7a-c4cd816fbdff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6819","nonce":3229936280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6821","nonce":3229936280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6825","nonce":3229936280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6823","nonce":3229936280}]},"public_addr":"172.21.15.67:6819/3229936280","cluster_addr":"172.21.15.67:6821/3229936280","heartbeat_back_addr":"172.21.15.67:6825/3229936280","heartbeat_front_addr":"172.21.15.67:6823/3229936280","state":["exists","up"]},{"osd":7,"uuid":"86de30c2-b8ee-4c9d-9c17-8a56517934ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6817","nonce":1622917909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6819","nonce":1622917909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6823","nonce":1622917909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6821","nonce":1622917909}]},"public_addr":"172.21.15.100:6817/1622917909","cluster_addr":"172.21.15.100:6819/1622917909","heartbeat_back_addr":"172.21.15.100:6823/1622917909","heartbeat_front_addr":"172.21.15.100:6821/1622917909","state":["exists","up"]},{"osd":8,"uuid":"0e96cdad-a135-4115-9e2d-9a28306700e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6817","nonce":3452445943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6819","nonce":3452445943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6823","nonce":3452445943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6821","nonce":3452445943}]},"public_addr":"172.21.15.203:6817/3452445943","cluster_addr":"172.21.15.203:6819/3452445943","heartbeat_back_addr":"172.21.15.203:6823/3452445943","heartbeat_front_addr":"172.21.15.203:6821/3452445943","state":["exists","up"]},{"osd":9,"uuid":"95704ed6-6286-4094-9c6d-23919c7005ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6827","nonce":2998307533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6829","nonce":2998307533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6833","nonce":2998307533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6831","nonce":2998307533}]},"public_addr":"172.21.15.67:6827/2998307533","cluster_addr":"172.21.15.67:6829/2998307533","heartbeat_back_addr":"172.21.15.67:6833/2998307533","heartbeat_front_addr":"172.21.15.67:6831/2998307533","state":["exists","up"]},{"osd":10,"uuid":"bd2a7d33-b935-46c8-95c3-d76f28242ffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6825","nonce":2819057142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6827","nonce":2819057142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6831","nonce":2819057142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6829","nonce":2819057142}]},"public_addr":"172.21.15.100:6825/2819057142","cluster_addr":"172.21.15.100:6827/2819057142","heartbeat_back_addr":"172.21.15.100:6831/2819057142","heartbeat_front_addr":"172.21.15.100:6829/2819057142","state":["exists","up"]},{"osd":11,"uuid":"ea591507-10e2-4c08-8d29-d38ba9be96ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6825","nonce":566502600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6827","nonce":566502600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6831","nonce":566502600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6829","nonce":566502600}]},"public_addr":"172.21.15.203:6825/566502600","cluster_addr":"172.21.15.203:6827/566502600","heartbeat_back_addr":"172.21.15.203:6831/566502600","heartbeat_front_addr":"172.21.15.203:6829/566502600","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:28:58.945733+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:29:23.597118+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:29:48.303563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:30:14.139139+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:30:39.021508+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:04.981241+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:30.853841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:56.452967+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:32:22.239036+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:32:47.998803+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:33:13.850759+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:33:39.282150+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6800/1181476774":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/3100321138":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/2309364298":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/3458717076":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/4110206171":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6801/1181476774":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/2910038982":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6801/2622729103":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6800/2622729103":"2024-09-22T06:26:37.041377+0000","172.21.15.67:0/3346784528":"2024-09-22T06:26:37.041377+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:48:19.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.809+0000 7febdfc27640 1 -- 172.21.15.67:0/2284860709 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7febc40b6660 msgr2=0x7febc40b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:19.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.809+0000 7febdfc27640 1 --2- 172.21.15.67:0/2284860709 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7febc40b6660 0x7febc40b8b70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7febc8005d00 tx=0x7febc800b410 comp rx=0 tx=0).stop 2024-09-21T06:48:19.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.809+0000 7febdfc27640 1 -- 172.21.15.67:0/2284860709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febd8102580 msgr2=0x7febd8198cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:19.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.809+0000 7febdfc27640 1 --2- 172.21.15.67:0/2284860709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febd8102580 0x7febd8198cd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7febd400e9f0 tx=0x7febd400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:48:19.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.809+0000 7febdfc27640 1 -- 172.21.15.67:0/2284860709 shutdown_connections 2024-09-21T06:48:19.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.809+0000 7febdfc27640 1 --2- 172.21.15.67:0/2284860709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febd8103600 0x7febd8199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:19.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.809+0000 7febdfc27640 1 --2- 172.21.15.67:0/2284860709 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7febc40b6660 0x7febc40b8b70 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:19.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.809+0000 7febdfc27640 1 --2- 172.21.15.67:0/2284860709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febd8103fa0 0x7febd8199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:19.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.809+0000 7febdfc27640 1 --2- 172.21.15.67:0/2284860709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febd8102580 0x7febd8198cd0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:19.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.809+0000 7febdfc27640 1 -- 172.21.15.67:0/2284860709 >> 172.21.15.67:0/2284860709 conn(0x7febd80fdee0 msgr2=0x7febd8106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:19.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.809+0000 7febdfc27640 1 -- 172.21.15.67:0/2284860709 shutdown_connections 2024-09-21T06:48:19.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.809+0000 7febdfc27640 1 -- 172.21.15.67:0/2284860709 wait complete. 2024-09-21T06:48:19.823 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T06:29:50.212825+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T06:34:29.082881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T06:34:31.301094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-21T06:34:37.681356+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-21T06:48:19.824 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T06:48:19.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.938+0000 7f036af62640 1 -- 172.21.15.67:0/2311416916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0364101d20 msgr2=0x7f0364102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:19.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.938+0000 7f036af62640 1 --2- 172.21.15.67:0/2311416916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0364101d20 0x7f0364102190 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0358009900 tx=0x7f035802a410 comp rx=0 tx=0).stop 2024-09-21T06:48:19.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.938+0000 7f036af62640 1 -- 172.21.15.67:0/2311416916 shutdown_connections 2024-09-21T06:48:19.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.938+0000 7f036af62640 1 --2- 172.21.15.67:0/2311416916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0364103740 0x7f036410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:19.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.938+0000 7f036af62640 1 --2- 172.21.15.67:0/2311416916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0364102da0 0x7f03641031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:19.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.938+0000 7f036af62640 1 --2- 172.21.15.67:0/2311416916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0364101d20 0x7f0364102190 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:19.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.938+0000 7f036af62640 1 -- 172.21.15.67:0/2311416916 >> 172.21.15.67:0/2311416916 conn(0x7f03640fd660 msgr2=0x7f03640ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:19.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.938+0000 7f036af62640 1 -- 172.21.15.67:0/2311416916 shutdown_connections 2024-09-21T06:48:19.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.938+0000 7f036af62640 1 -- 172.21.15.67:0/2311416916 wait complete. 2024-09-21T06:48:19.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.939+0000 7f036af62640 1 Processor -- start 2024-09-21T06:48:19.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.939+0000 7f036af62640 1 -- start start 2024-09-21T06:48:19.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.939+0000 7f036af62640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0364101d20 0x7f036410d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:19.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.939+0000 7f036af62640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0364102da0 0x7f036410ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:19.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.939+0000 7f036af62640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0364103740 0x7f036410bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:19.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.939+0000 7f036af62640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f036410f1d0 con 0x7f0364102da0 2024-09-21T06:48:19.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.939+0000 7f036af62640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f036410f050 con 0x7f0364101d20 2024-09-21T06:48:19.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.939+0000 7f036af62640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f036410f350 con 0x7f0364103740 2024-09-21T06:48:19.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.939+0000 7f0363fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0364102da0 0x7f036410ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:19.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.939+0000 7f0363fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0364102da0 0x7f036410ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54642/0 (socket says 172.21.15.67:54642) 2024-09-21T06:48:19.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.940+0000 7f0363fff640 1 -- 172.21.15.67:0/3980505772 learned_addr learned my addr 172.21.15.67:0/3980505772 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:48:19.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.940+0000 7f0368cfe640 1 --2- 172.21.15.67:0/3980505772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0364101d20 0x7f036410d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:19.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.940+0000 7f03694ff640 1 --2- 172.21.15.67:0/3980505772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0364103740 0x7f036410bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:19.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.940+0000 7f0363fff640 1 -- 172.21.15.67:0/3980505772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0364103740 msgr2=0x7f036410bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:19.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.940+0000 7f0363fff640 1 --2- 172.21.15.67:0/3980505772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0364103740 0x7f036410bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:19.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.940+0000 7f0363fff640 1 -- 172.21.15.67:0/3980505772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0364101d20 msgr2=0x7f036410d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:19.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.940+0000 7f0363fff640 1 --2- 172.21.15.67:0/3980505772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0364101d20 0x7f036410d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:19.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.940+0000 7f0363fff640 1 -- 172.21.15.67:0/3980505772 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f036410c6d0 con 0x7f0364102da0 2024-09-21T06:48:19.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.940+0000 7f0363fff640 1 --2- 172.21.15.67:0/3980505772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0364102da0 0x7f036410ba60 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f035000c890 tx=0x7f035000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:19.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.940+0000 7f0361ffb640 1 -- 172.21.15.67:0/3980505772 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0350004220 con 0x7f0364102da0 2024-09-21T06:48:19.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.940+0000 7f0361ffb640 1 -- 172.21.15.67:0/3980505772 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0350007e40 con 0x7f0364102da0 2024-09-21T06:48:19.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.940+0000 7f036af62640 1 -- 172.21.15.67:0/3980505772 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03641a3ea0 con 0x7f0364102da0 2024-09-21T06:48:19.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.940+0000 7f0361ffb640 1 -- 172.21.15.67:0/3980505772 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0350004a00 con 0x7f0364102da0 2024-09-21T06:48:19.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.940+0000 7f036af62640 1 -- 172.21.15.67:0/3980505772 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03641a42b0 con 0x7f0364102da0 2024-09-21T06:48:19.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.941+0000 7f036af62640 1 -- 172.21.15.67:0/3980505772 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0364108ef0 con 0x7f0364102da0 2024-09-21T06:48:19.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.943+0000 7f0361ffb640 1 -- 172.21.15.67:0/3980505772 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f03500079b0 con 0x7f0364102da0 2024-09-21T06:48:19.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.944+0000 7f0361ffb640 1 --2- 172.21.15.67:0/3980505772 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f03540b6590 0x7f03540b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:19.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.944+0000 7f0368cfe640 1 --2- 172.21.15.67:0/3980505772 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f03540b6590 0x7f03540b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:19.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.944+0000 7f0361ffb640 1 -- 172.21.15.67:0/3980505772 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f03500e7750 con 0x7f0364102da0 2024-09-21T06:48:19.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.944+0000 7f0368cfe640 1 --2- 172.21.15.67:0/3980505772 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f03540b6590 0x7f03540b8aa0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f03580057c0 tx=0x7f0358000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:19.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:19.947+0000 7f0361ffb640 1 -- 172.21.15.67:0/3980505772 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0350014030 con 0x7f0364102da0 2024-09-21T06:48:19.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:19 smithi203 ceph-mon[33735]: pgmap v712: 129 pgs: 129 active+clean; 3.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 18 MiB/s wr, 525 op/s 2024-09-21T06:48:20.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:19 smithi067 ceph-mon[31232]: pgmap v712: 129 pgs: 129 active+clean; 3.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 18 MiB/s wr, 525 op/s 2024-09-21T06:48:20.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:19 smithi100 ceph-mon[33401]: pgmap v712: 129 pgs: 129 active+clean; 3.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 18 MiB/s wr, 525 op/s 2024-09-21T06:48:20.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.135+0000 7f036af62640 1 -- 172.21.15.67:0/3980505772 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f036402e890 con 0x7f0364102da0 2024-09-21T06:48:20.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.135+0000 7f0361ffb640 1 -- 172.21.15.67:0/3980505772 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) ==== 93+0+10 (secure 0 0 0) 0x7f03500adc90 con 0x7f0364102da0 2024-09-21T06:48:20.138 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-09-21T06:48:20.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.138+0000 7f036af62640 1 -- 172.21.15.67:0/3980505772 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f03540b6590 msgr2=0x7f03540b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:20.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.138+0000 7f036af62640 1 --2- 172.21.15.67:0/3980505772 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f03540b6590 0x7f03540b8aa0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f03580057c0 tx=0x7f0358000bc0 comp rx=0 tx=0).stop 2024-09-21T06:48:20.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.138+0000 7f036af62640 1 -- 172.21.15.67:0/3980505772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0364102da0 msgr2=0x7f036410ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:20.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.138+0000 7f036af62640 1 --2- 172.21.15.67:0/3980505772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0364102da0 0x7f036410ba60 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f035000c890 tx=0x7f035000cd50 comp rx=0 tx=0).stop 2024-09-21T06:48:20.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.138+0000 7f036af62640 1 -- 172.21.15.67:0/3980505772 shutdown_connections 2024-09-21T06:48:20.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.138+0000 7f036af62640 1 --2- 172.21.15.67:0/3980505772 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f03540b6590 0x7f03540b8aa0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:20.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.138+0000 7f036af62640 1 --2- 172.21.15.67:0/3980505772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0364103740 0x7f036410bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:20.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.138+0000 7f036af62640 1 --2- 172.21.15.67:0/3980505772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0364102da0 0x7f036410ba60 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:20.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.138+0000 7f036af62640 1 --2- 172.21.15.67:0/3980505772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0364101d20 0x7f036410d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:20.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.138+0000 7f036af62640 1 -- 172.21.15.67:0/3980505772 >> 172.21.15.67:0/3980505772 conn(0x7f03640fd660 msgr2=0x7f03641060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:20.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.138+0000 7f036af62640 1 -- 172.21.15.67:0/3980505772 shutdown_connections 2024-09-21T06:48:20.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.138+0000 7f036af62640 1 -- 172.21.15.67:0/3980505772 wait complete. 2024-09-21T06:48:20.152 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T06:48:20.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.267+0000 7f1c6a157640 1 -- 172.21.15.67:0/2297247518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c64101a80 msgr2=0x7f1c64101e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:20.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.267+0000 7f1c6a157640 1 --2- 172.21.15.67:0/2297247518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c64101a80 0x7f1c64101e90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1c54009900 tx=0x7f1c5402a410 comp rx=0 tx=0).stop 2024-09-21T06:48:20.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.267+0000 7f1c6a157640 1 -- 172.21.15.67:0/2297247518 shutdown_connections 2024-09-21T06:48:20.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.267+0000 7f1c6a157640 1 --2- 172.21.15.67:0/2297247518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c64106bd0 0x7f1c64108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:20.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.267+0000 7f1c6a157640 1 --2- 172.21.15.67:0/2297247518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c64109530 0x7f1c6410b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:20.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.267+0000 7f1c6a157640 1 --2- 172.21.15.67:0/2297247518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c64101a80 0x7f1c64101e90 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:20.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.267+0000 7f1c6a157640 1 -- 172.21.15.67:0/2297247518 >> 172.21.15.67:0/2297247518 conn(0x7f1c640fd660 msgr2=0x7f1c640ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:20.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.267+0000 7f1c6a157640 1 -- 172.21.15.67:0/2297247518 shutdown_connections 2024-09-21T06:48:20.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.267+0000 7f1c6a157640 1 -- 172.21.15.67:0/2297247518 wait complete. 2024-09-21T06:48:20.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.267+0000 7f1c6a157640 1 Processor -- start 2024-09-21T06:48:20.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.268+0000 7f1c6a157640 1 -- start start 2024-09-21T06:48:20.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.268+0000 7f1c6a157640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c64101a80 0x7f1c6419c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:20.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.268+0000 7f1c6a157640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c64106bd0 0x7f1c6419c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:20.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.268+0000 7f1c6a157640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c64109530 0x7f1c6419d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:20.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.268+0000 7f1c6a157640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6410f180 con 0x7f1c64106bd0 2024-09-21T06:48:20.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.268+0000 7f1c6a157640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6410f000 con 0x7f1c64101a80 2024-09-21T06:48:20.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.268+0000 7f1c6a157640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6410f300 con 0x7f1c64109530 2024-09-21T06:48:20.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.268+0000 7f1c62ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c64106bd0 0x7f1c6419c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:20.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.268+0000 7f1c62ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c64106bd0 0x7f1c6419c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54664/0 (socket says 172.21.15.67:54664) 2024-09-21T06:48:20.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.268+0000 7f1c62ffd640 1 -- 172.21.15.67:0/3625965691 learned_addr learned my addr 172.21.15.67:0/3625965691 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:48:20.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.268+0000 7f1c637fe640 1 --2- 172.21.15.67:0/3625965691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c64101a80 0x7f1c6419c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:20.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.268+0000 7f1c63fff640 1 --2- 172.21.15.67:0/3625965691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c64109530 0x7f1c6419d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:20.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.268+0000 7f1c62ffd640 1 -- 172.21.15.67:0/3625965691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c64109530 msgr2=0x7f1c6419d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:20.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.268+0000 7f1c62ffd640 1 --2- 172.21.15.67:0/3625965691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c64109530 0x7f1c6419d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:20.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.268+0000 7f1c62ffd640 1 -- 172.21.15.67:0/3625965691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c64101a80 msgr2=0x7f1c6419c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:20.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.268+0000 7f1c62ffd640 1 --2- 172.21.15.67:0/3625965691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c64101a80 0x7f1c6419c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:20.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.268+0000 7f1c62ffd640 1 -- 172.21.15.67:0/3625965691 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c6419daa0 con 0x7f1c64106bd0 2024-09-21T06:48:20.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.268+0000 7f1c63fff640 1 --2- 172.21.15.67:0/3625965691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c64109530 0x7f1c6419d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:48:20.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.269+0000 7f1c62ffd640 1 --2- 172.21.15.67:0/3625965691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c64106bd0 0x7f1c6419c6d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800d950 tx=0x7f1c5800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:20.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.269+0000 7f1c60ff9640 1 -- 172.21.15.67:0/3625965691 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c58017070 con 0x7f1c64106bd0 2024-09-21T06:48:20.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.269+0000 7f1c60ff9640 1 -- 172.21.15.67:0/3625965691 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1c58009c30 con 0x7f1c64106bd0 2024-09-21T06:48:20.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.269+0000 7f1c6a157640 1 -- 172.21.15.67:0/3625965691 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c6419dd90 con 0x7f1c64106bd0 2024-09-21T06:48:20.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.269+0000 7f1c60ff9640 1 -- 172.21.15.67:0/3625965691 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5800f620 con 0x7f1c64106bd0 2024-09-21T06:48:20.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.269+0000 7f1c6a157640 1 -- 172.21.15.67:0/3625965691 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c641a4480 con 0x7f1c64106bd0 2024-09-21T06:48:20.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.270+0000 7f1c6a157640 1 -- 172.21.15.67:0/3625965691 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c641a4650 con 0x7f1c64106bd0 2024-09-21T06:48:20.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.271+0000 7f1c60ff9640 1 -- 172.21.15.67:0/3625965691 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f1c58010460 con 0x7f1c64106bd0 2024-09-21T06:48:20.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.272+0000 7f1c60ff9640 1 --2- 172.21.15.67:0/3625965691 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f1c4c0b6510 0x7f1c4c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:20.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.273+0000 7f1c637fe640 1 --2- 172.21.15.67:0/3625965691 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f1c4c0b6510 0x7f1c4c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:20.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.273+0000 7f1c60ff9640 1 -- 172.21.15.67:0/3625965691 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f1c580e73a0 con 0x7f1c64106bd0 2024-09-21T06:48:20.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.273+0000 7f1c637fe640 1 --2- 172.21.15.67:0/3625965691 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f1c4c0b6510 0x7f1c4c0b8a20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1c54000c00 tx=0x7f1c54005720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:20.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.275+0000 7f1c60ff9640 1 -- 172.21.15.67:0/3625965691 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1c580ad860 con 0x7f1c64106bd0 2024-09-21T06:48:20.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.463+0000 7f1c6a157640 1 -- 172.21.15.67:0/3625965691 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1c641a4820 con 0x7f1c64106bd0 2024-09-21T06:48:20.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.464+0000 7f1c60ff9640 1 -- 172.21.15.67:0/3625965691 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) ==== 104+0+11 (secure 0 0 0) 0x7f1c58009de0 con 0x7f1c64106bd0 2024-09-21T06:48:20.466 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-09-21T06:48:20.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.466+0000 7f1c6a157640 1 -- 172.21.15.67:0/3625965691 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f1c4c0b6510 msgr2=0x7f1c4c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:20.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.466+0000 7f1c6a157640 1 --2- 172.21.15.67:0/3625965691 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f1c4c0b6510 0x7f1c4c0b8a20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1c54000c00 tx=0x7f1c54005720 comp rx=0 tx=0).stop 2024-09-21T06:48:20.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.466+0000 7f1c6a157640 1 -- 172.21.15.67:0/3625965691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c64106bd0 msgr2=0x7f1c6419c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:20.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.466+0000 7f1c6a157640 1 --2- 172.21.15.67:0/3625965691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c64106bd0 0x7f1c6419c6d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800d950 tx=0x7f1c5800de10 comp rx=0 tx=0).stop 2024-09-21T06:48:20.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.466+0000 7f1c6a157640 1 -- 172.21.15.67:0/3625965691 shutdown_connections 2024-09-21T06:48:20.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.466+0000 7f1c6a157640 1 --2- 172.21.15.67:0/3625965691 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f1c4c0b6510 0x7f1c4c0b8a20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:20.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.466+0000 7f1c6a157640 1 --2- 172.21.15.67:0/3625965691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c64109530 0x7f1c6419d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:20.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.466+0000 7f1c6a157640 1 --2- 172.21.15.67:0/3625965691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c64106bd0 0x7f1c6419c6d0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:20.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.466+0000 7f1c6a157640 1 --2- 172.21.15.67:0/3625965691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c64101a80 0x7f1c6419c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:20.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.466+0000 7f1c6a157640 1 -- 172.21.15.67:0/3625965691 >> 172.21.15.67:0/3625965691 conn(0x7f1c640fd660 msgr2=0x7f1c64103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:20.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.466+0000 7f1c6a157640 1 -- 172.21.15.67:0/3625965691 shutdown_connections 2024-09-21T06:48:20.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.466+0000 7f1c6a157640 1 -- 172.21.15.67:0/3625965691 wait complete. 2024-09-21T06:48:20.480 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T06:48:20.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.594+0000 7f2458227640 1 -- 172.21.15.67:0/1484725904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2450103fc0 msgr2=0x7f245010a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:20.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.594+0000 7f2458227640 1 --2- 172.21.15.67:0/1484725904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2450103fc0 0x7f245010a960 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2448009c90 tx=0x7f244802a860 comp rx=0 tx=0).stop 2024-09-21T06:48:20.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.594+0000 7f2458227640 1 -- 172.21.15.67:0/1484725904 shutdown_connections 2024-09-21T06:48:20.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.594+0000 7f2458227640 1 --2- 172.21.15.67:0/1484725904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2450103620 0x7f2450103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:20.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.594+0000 7f2458227640 1 --2- 172.21.15.67:0/1484725904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2450103fc0 0x7f245010a960 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:20.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.594+0000 7f2458227640 1 --2- 172.21.15.67:0/1484725904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24501025a0 0x7f2450102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:20.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.594+0000 7f2458227640 1 -- 172.21.15.67:0/1484725904 >> 172.21.15.67:0/1484725904 conn(0x7f24500fdee0 msgr2=0x7f2450100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:20.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.594+0000 7f2458227640 1 -- 172.21.15.67:0/1484725904 shutdown_connections 2024-09-21T06:48:20.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.594+0000 7f2458227640 1 -- 172.21.15.67:0/1484725904 wait complete. 2024-09-21T06:48:20.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.595+0000 7f2458227640 1 Processor -- start 2024-09-21T06:48:20.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.595+0000 7f2458227640 1 -- start start 2024-09-21T06:48:20.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.595+0000 7f2458227640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24501025a0 0x7f245006c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:20.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.595+0000 7f2458227640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2450103620 0x7f245019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:20.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.595+0000 7f2458227640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2450103fc0 0x7f24501a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:20.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.595+0000 7f2458227640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f245010e0b0 con 0x7f2450103620 2024-09-21T06:48:20.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.595+0000 7f2458227640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f245010df30 con 0x7f24501025a0 2024-09-21T06:48:20.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.595+0000 7f2458227640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f245010e230 con 0x7f2450103fc0 2024-09-21T06:48:20.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.595+0000 7f24557c2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2450103620 0x7f245019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:20.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.595+0000 7f24557c2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2450103620 0x7f245019ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54686/0 (socket says 172.21.15.67:54686) 2024-09-21T06:48:20.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.595+0000 7f24557c2640 1 -- 172.21.15.67:0/1566856140 learned_addr learned my addr 172.21.15.67:0/1566856140 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:48:20.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.595+0000 7f2455fc3640 1 --2- 172.21.15.67:0/1566856140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24501025a0 0x7f245006c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:20.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.595+0000 7f24557c2640 1 -- 172.21.15.67:0/1566856140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2450103fc0 msgr2=0x7f24501a1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:20.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.595+0000 7f24557c2640 1 --2- 172.21.15.67:0/1566856140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2450103fc0 0x7f24501a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:20.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.595+0000 7f24557c2640 1 -- 172.21.15.67:0/1566856140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24501025a0 msgr2=0x7f245006c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:20.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.595+0000 7f24557c2640 1 --2- 172.21.15.67:0/1566856140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24501025a0 0x7f245006c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:20.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.595+0000 7f24557c2640 1 -- 172.21.15.67:0/1566856140 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24501a1920 con 0x7f2450103620 2024-09-21T06:48:20.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.595+0000 7f2455fc3640 1 --2- 172.21.15.67:0/1566856140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24501025a0 0x7f245006c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:48:20.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.596+0000 7f24557c2640 1 --2- 172.21.15.67:0/1566856140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2450103620 0x7f245019ebb0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f244c00d950 tx=0x7f244c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:20.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.596+0000 7f2442ffd640 1 -- 172.21.15.67:0/1566856140 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c017070 con 0x7f2450103620 2024-09-21T06:48:20.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.596+0000 7f2442ffd640 1 -- 172.21.15.67:0/1566856140 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f244c009c30 con 0x7f2450103620 2024-09-21T06:48:20.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.596+0000 7f2442ffd640 1 -- 172.21.15.67:0/1566856140 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c00f620 con 0x7f2450103620 2024-09-21T06:48:20.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.596+0000 7f2458227640 1 -- 172.21.15.67:0/1566856140 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24501a1c10 con 0x7f2450103620 2024-09-21T06:48:20.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.596+0000 7f2458227640 1 -- 172.21.15.67:0/1566856140 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24501a2160 con 0x7f2450103620 2024-09-21T06:48:20.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.597+0000 7f2458227640 1 -- 172.21.15.67:0/1566856140 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2450109770 con 0x7f2450103620 2024-09-21T06:48:20.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.598+0000 7f2442ffd640 1 -- 172.21.15.67:0/1566856140 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f244c010460 con 0x7f2450103620 2024-09-21T06:48:20.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.599+0000 7f2442ffd640 1 --2- 172.21.15.67:0/1566856140 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f243c0b6510 0x7f243c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:20.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.599+0000 7f2455fc3640 1 --2- 172.21.15.67:0/1566856140 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f243c0b6510 0x7f243c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:20.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.599+0000 7f2442ffd640 1 -- 172.21.15.67:0/1566856140 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f244c0e73c0 con 0x7f2450103620 2024-09-21T06:48:20.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.600+0000 7f2455fc3640 1 --2- 172.21.15.67:0/1566856140 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f243c0b6510 0x7f243c0b8a20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f24440058c0 tx=0x7f2444005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:20.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.604+0000 7f2442ffd640 1 -- 172.21.15.67:0/1566856140 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f244c0ad880 con 0x7f2450103620 2024-09-21T06:48:20.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:20 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2284860709' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:48:20.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:20 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3980505772' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:48:20.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:20 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3625965691' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T06:48:20.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.794+0000 7f2458227640 1 -- 172.21.15.67:0/1566856140 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2450063ff0 con 0x7f2450103620 2024-09-21T06:48:20.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.794+0000 7f2442ffd640 1 -- 172.21.15.67:0/1566856140 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) ==== 100+0+11 (secure 0 0 0) 0x7f244c009de0 con 0x7f2450103620 2024-09-21T06:48:20.796 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-09-21T06:48:20.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.796+0000 7f2458227640 1 -- 172.21.15.67:0/1566856140 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f243c0b6510 msgr2=0x7f243c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:20.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.796+0000 7f2458227640 1 --2- 172.21.15.67:0/1566856140 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f243c0b6510 0x7f243c0b8a20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f24440058c0 tx=0x7f2444005d30 comp rx=0 tx=0).stop 2024-09-21T06:48:20.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.796+0000 7f2458227640 1 -- 172.21.15.67:0/1566856140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2450103620 msgr2=0x7f245019ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:20.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.796+0000 7f2458227640 1 --2- 172.21.15.67:0/1566856140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2450103620 0x7f245019ebb0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f244c00d950 tx=0x7f244c00de10 comp rx=0 tx=0).stop 2024-09-21T06:48:20.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.796+0000 7f2458227640 1 -- 172.21.15.67:0/1566856140 shutdown_connections 2024-09-21T06:48:20.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.796+0000 7f2458227640 1 --2- 172.21.15.67:0/1566856140 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f243c0b6510 0x7f243c0b8a20 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:20.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.796+0000 7f2458227640 1 --2- 172.21.15.67:0/1566856140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2450103fc0 0x7f24501a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:20.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.796+0000 7f2458227640 1 --2- 172.21.15.67:0/1566856140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2450103620 0x7f245019ebb0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:20.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.796+0000 7f2458227640 1 --2- 172.21.15.67:0/1566856140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24501025a0 0x7f245006c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:20.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.796+0000 7f2458227640 1 -- 172.21.15.67:0/1566856140 >> 172.21.15.67:0/1566856140 conn(0x7f24500fdee0 msgr2=0x7f2450106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:20.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.796+0000 7f2458227640 1 -- 172.21.15.67:0/1566856140 shutdown_connections 2024-09-21T06:48:20.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.796+0000 7f2458227640 1 -- 172.21.15.67:0/1566856140 wait complete. 2024-09-21T06:48:20.811 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-21T06:48:20.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.925+0000 7fb0b81c6640 1 -- 172.21.15.67:0/2857593376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0b0103600 msgr2=0x7fb0b0103a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:20.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.925+0000 7fb0b81c6640 1 --2- 172.21.15.67:0/2857593376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0b0103600 0x7fb0b0103a10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4009900 tx=0x7fb0a402a410 comp rx=0 tx=0).stop 2024-09-21T06:48:20.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.925+0000 7fb0b81c6640 1 -- 172.21.15.67:0/2857593376 shutdown_connections 2024-09-21T06:48:20.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.925+0000 7fb0b81c6640 1 --2- 172.21.15.67:0/2857593376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0b0103fa0 0x7fb0b010a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:20.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.925+0000 7fb0b81c6640 1 --2- 172.21.15.67:0/2857593376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0b0103600 0x7fb0b0103a10 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:20.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.925+0000 7fb0b81c6640 1 --2- 172.21.15.67:0/2857593376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b0102580 0x7fb0b01029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:20.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.925+0000 7fb0b81c6640 1 -- 172.21.15.67:0/2857593376 >> 172.21.15.67:0/2857593376 conn(0x7fb0b00fdee0 msgr2=0x7fb0b0100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:20.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.925+0000 7fb0b81c6640 1 -- 172.21.15.67:0/2857593376 shutdown_connections 2024-09-21T06:48:20.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.925+0000 7fb0b81c6640 1 -- 172.21.15.67:0/2857593376 wait complete. 2024-09-21T06:48:20.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.926+0000 7fb0b81c6640 1 Processor -- start 2024-09-21T06:48:20.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.926+0000 7fb0b81c6640 1 -- start start 2024-09-21T06:48:20.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.926+0000 7fb0b81c6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b0102580 0x7fb0b0198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:20.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.926+0000 7fb0b81c6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0b0103600 0x7fb0b01993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:20.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.926+0000 7fb0b81c6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0b0103fa0 0x7fb0b0199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:20.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.926+0000 7fb0b81c6640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b010e090 con 0x7fb0b0103600 2024-09-21T06:48:20.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.926+0000 7fb0b81c6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b010df10 con 0x7fb0b0102580 2024-09-21T06:48:20.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.926+0000 7fb0b81c6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b010e210 con 0x7fb0b0103fa0 2024-09-21T06:48:20.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.926+0000 7fb0b5761640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0b0103600 0x7fb0b01993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:20.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.926+0000 7fb0b5761640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0b0103600 0x7fb0b01993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54708/0 (socket says 172.21.15.67:54708) 2024-09-21T06:48:20.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.926+0000 7fb0b5761640 1 -- 172.21.15.67:0/661546867 learned_addr learned my addr 172.21.15.67:0/661546867 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:48:20.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.926+0000 7fb0b6763640 1 --2- 172.21.15.67:0/661546867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0b0103fa0 0x7fb0b0199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:20.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.926+0000 7fb0b5f62640 1 --2- 172.21.15.67:0/661546867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b0102580 0x7fb0b0198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:20.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.926+0000 7fb0b5761640 1 -- 172.21.15.67:0/661546867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0b0103fa0 msgr2=0x7fb0b0199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:20.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.926+0000 7fb0b5761640 1 --2- 172.21.15.67:0/661546867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0b0103fa0 0x7fb0b0199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:20.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.926+0000 7fb0b5761640 1 -- 172.21.15.67:0/661546867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b0102580 msgr2=0x7fb0b0198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:20.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.926+0000 7fb0b5761640 1 --2- 172.21.15.67:0/661546867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b0102580 0x7fb0b0198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:20.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.926+0000 7fb0b5761640 1 -- 172.21.15.67:0/661546867 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b019a790 con 0x7fb0b0103600 2024-09-21T06:48:20.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.926+0000 7fb0b6763640 1 --2- 172.21.15.67:0/661546867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0b0103fa0 0x7fb0b0199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:48:20.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.926+0000 7fb0b5761640 1 --2- 172.21.15.67:0/661546867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0b0103600 0x7fb0b01993c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4009230 tx=0x7fb0a4004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:20.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.927+0000 7fb0a2ffd640 1 -- 172.21.15.67:0/661546867 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a403f030 con 0x7fb0b0103600 2024-09-21T06:48:20.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.927+0000 7fb0a2ffd640 1 -- 172.21.15.67:0/661546867 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0a4003c90 con 0x7fb0b0103600 2024-09-21T06:48:20.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.927+0000 7fb0b81c6640 1 -- 172.21.15.67:0/661546867 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b019aa20 con 0x7fb0b0103600 2024-09-21T06:48:20.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.927+0000 7fb0a2ffd640 1 -- 172.21.15.67:0/661546867 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4035070 con 0x7fb0b0103600 2024-09-21T06:48:20.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.927+0000 7fb0b81c6640 1 -- 172.21.15.67:0/661546867 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b01a1170 con 0x7fb0b0103600 2024-09-21T06:48:20.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.928+0000 7fb0b81c6640 1 -- 172.21.15.67:0/661546867 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0b01029f0 con 0x7fb0b0103600 2024-09-21T06:48:20.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.929+0000 7fb0a2ffd640 1 -- 172.21.15.67:0/661546867 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fb0a4042030 con 0x7fb0b0103600 2024-09-21T06:48:20.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.930+0000 7fb0a2ffd640 1 --2- 172.21.15.67:0/661546867 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb09c0b6590 0x7fb09c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:20.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.930+0000 7fb0b5f62640 1 --2- 172.21.15.67:0/661546867 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb09c0b6590 0x7fb09c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:20.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.930+0000 7fb0a2ffd640 1 -- 172.21.15.67:0/661546867 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fb0a4107e80 con 0x7fb0b0103600 2024-09-21T06:48:20.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.931+0000 7fb0b5f62640 1 --2- 172.21.15.67:0/661546867 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb09c0b6590 0x7fb09c0b8aa0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0058c0 tx=0x7fb0ac00a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:20.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:20.934+0000 7fb0a2ffd640 1 -- 172.21.15.67:0/661546867 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb0a40ce340 con 0x7fb0b0103600 2024-09-21T06:48:20.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:20 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2284860709' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:48:20.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:20 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3980505772' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:48:20.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:20 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3625965691' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T06:48:21.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:20 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2284860709' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:48:21.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:20 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3980505772' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:48:21.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:20 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3625965691' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T06:48:21.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.121+0000 7fb0b81c6640 1 -- 172.21.15.67:0/661546867 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb0b002e9f0 con 0x7fb0b0103600 2024-09-21T06:48:21.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.121+0000 7fb0a2ffd640 1 -- 172.21.15.67:0/661546867 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v85) ==== 103+0+11 (secure 0 0 0) 0x7fb0b002e9f0 con 0x7fb0b0103600 2024-09-21T06:48:21.124 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-09-21T06:48:21.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.124+0000 7fb0b81c6640 1 -- 172.21.15.67:0/661546867 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb09c0b6590 msgr2=0x7fb09c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:21.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.124+0000 7fb0b81c6640 1 --2- 172.21.15.67:0/661546867 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb09c0b6590 0x7fb09c0b8aa0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0058c0 tx=0x7fb0ac00a4c0 comp rx=0 tx=0).stop 2024-09-21T06:48:21.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.124+0000 7fb0b81c6640 1 -- 172.21.15.67:0/661546867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0b0103600 msgr2=0x7fb0b01993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:21.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.124+0000 7fb0b81c6640 1 --2- 172.21.15.67:0/661546867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0b0103600 0x7fb0b01993c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4009230 tx=0x7fb0a4004360 comp rx=0 tx=0).stop 2024-09-21T06:48:21.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.124+0000 7fb0b81c6640 1 -- 172.21.15.67:0/661546867 shutdown_connections 2024-09-21T06:48:21.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.124+0000 7fb0b81c6640 1 --2- 172.21.15.67:0/661546867 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb09c0b6590 0x7fb09c0b8aa0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:21.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.124+0000 7fb0b81c6640 1 --2- 172.21.15.67:0/661546867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0b0103fa0 0x7fb0b0199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:21.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.124+0000 7fb0b81c6640 1 --2- 172.21.15.67:0/661546867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0b0103600 0x7fb0b01993c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:21.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.124+0000 7fb0b81c6640 1 --2- 172.21.15.67:0/661546867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b0102580 0x7fb0b0198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:21.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.124+0000 7fb0b81c6640 1 -- 172.21.15.67:0/661546867 >> 172.21.15.67:0/661546867 conn(0x7fb0b00fdee0 msgr2=0x7fb0b0106900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:21.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.124+0000 7fb0b81c6640 1 -- 172.21.15.67:0/661546867 shutdown_connections 2024-09-21T06:48:21.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.124+0000 7fb0b81c6640 1 -- 172.21.15.67:0/661546867 wait complete. 2024-09-21T06:48:21.138 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-21T06:48:21.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.249+0000 7f7f8c669640 1 -- 172.21.15.67:0/2682844811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f841025a0 msgr2=0x7f7f84102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:21.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.249+0000 7f7f8c669640 1 --2- 172.21.15.67:0/2682844811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f841025a0 0x7f7f84102a10 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7f80009900 tx=0x7f7f8002a410 comp rx=0 tx=0).stop 2024-09-21T06:48:21.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.249+0000 7f7f8c669640 1 -- 172.21.15.67:0/2682844811 shutdown_connections 2024-09-21T06:48:21.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.249+0000 7f7f8c669640 1 --2- 172.21.15.67:0/2682844811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f84103620 0x7f7f84103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:21.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.249+0000 7f7f8c669640 1 --2- 172.21.15.67:0/2682844811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f84103fc0 0x7f7f8410a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:21.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.249+0000 7f7f8c669640 1 --2- 172.21.15.67:0/2682844811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f841025a0 0x7f7f84102a10 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:21.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.249+0000 7f7f8c669640 1 -- 172.21.15.67:0/2682844811 >> 172.21.15.67:0/2682844811 conn(0x7f7f840fdee0 msgr2=0x7f7f84100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:21.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.249+0000 7f7f8c669640 1 -- 172.21.15.67:0/2682844811 shutdown_connections 2024-09-21T06:48:21.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.249+0000 7f7f8c669640 1 -- 172.21.15.67:0/2682844811 wait complete. 2024-09-21T06:48:21.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.250+0000 7f7f8c669640 1 Processor -- start 2024-09-21T06:48:21.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.250+0000 7f7f8c669640 1 -- start start 2024-09-21T06:48:21.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.250+0000 7f7f8c669640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f841025a0 0x7f7f8406c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:21.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.250+0000 7f7f8c669640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f84103620 0x7f7f8419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:21.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.250+0000 7f7f8c669640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f84103fc0 0x7f7f841a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:21.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.250+0000 7f7f8c669640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8410e0a0 con 0x7f7f84103fc0 2024-09-21T06:48:21.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.250+0000 7f7f8c669640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8410df20 con 0x7f7f84103620 2024-09-21T06:48:21.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.250+0000 7f7f8c669640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8410e220 con 0x7f7f841025a0 2024-09-21T06:48:21.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.250+0000 7f7f8ac06640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f84103fc0 0x7f7f841a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:21.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.250+0000 7f7f8ac06640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f84103fc0 0x7f7f841a10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54724/0 (socket says 172.21.15.67:54724) 2024-09-21T06:48:21.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.250+0000 7f7f8ac06640 1 -- 172.21.15.67:0/652228116 learned_addr learned my addr 172.21.15.67:0/652228116 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:48:21.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.250+0000 7f7f89c04640 1 --2- 172.21.15.67:0/652228116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f84103620 0x7f7f8419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:21.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.250+0000 7f7f8a405640 1 --2- 172.21.15.67:0/652228116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f841025a0 0x7f7f8406c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:21.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.250+0000 7f7f8ac06640 1 -- 172.21.15.67:0/652228116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f841025a0 msgr2=0x7f7f8406c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:21.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.250+0000 7f7f8ac06640 1 --2- 172.21.15.67:0/652228116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f841025a0 0x7f7f8406c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:21.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.250+0000 7f7f8ac06640 1 -- 172.21.15.67:0/652228116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f84103620 msgr2=0x7f7f8419eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:21.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.250+0000 7f7f8ac06640 1 --2- 172.21.15.67:0/652228116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f84103620 0x7f7f8419eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:21.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.250+0000 7f7f8ac06640 1 -- 172.21.15.67:0/652228116 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f841a1880 con 0x7f7f84103fc0 2024-09-21T06:48:21.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.250+0000 7f7f8a405640 1 --2- 172.21.15.67:0/652228116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f841025a0 0x7f7f8406c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:48:21.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.250+0000 7f7f8ac06640 1 --2- 172.21.15.67:0/652228116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f84103fc0 0x7f7f841a10f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c00eb40 tx=0x7f7f7c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:21.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.251+0000 7f7f7b7fe640 1 -- 172.21.15.67:0/652228116 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c010430 con 0x7f7f84103fc0 2024-09-21T06:48:21.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.251+0000 7f7f7b7fe640 1 -- 172.21.15.67:0/652228116 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f7c010a60 con 0x7f7f84103fc0 2024-09-21T06:48:21.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.251+0000 7f7f7b7fe640 1 -- 172.21.15.67:0/652228116 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c018ac0 con 0x7f7f84103fc0 2024-09-21T06:48:21.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.251+0000 7f7f8c669640 1 -- 172.21.15.67:0/652228116 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f841a1b10 con 0x7f7f84103fc0 2024-09-21T06:48:21.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.251+0000 7f7f8c669640 1 -- 172.21.15.67:0/652228116 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f841a20b0 con 0x7f7f84103fc0 2024-09-21T06:48:21.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.252+0000 7f7f8c669640 1 -- 172.21.15.67:0/652228116 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f84109770 con 0x7f7f84103fc0 2024-09-21T06:48:21.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.253+0000 7f7f7b7fe640 1 -- 172.21.15.67:0/652228116 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f7f7c010c10 con 0x7f7f84103fc0 2024-09-21T06:48:21.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.254+0000 7f7f7b7fe640 1 --2- 172.21.15.67:0/652228116 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f7f700b6590 0x7f7f700b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:21.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.254+0000 7f7f8a405640 1 --2- 172.21.15.67:0/652228116 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f7f700b6590 0x7f7f700b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:21.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.254+0000 7f7f7b7fe640 1 -- 172.21.15.67:0/652228116 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f7f7c0e8020 con 0x7f7f84103fc0 2024-09-21T06:48:21.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.255+0000 7f7f8a405640 1 --2- 172.21.15.67:0/652228116 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f7f700b6590 0x7f7f700b8aa0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7f80005f70 tx=0x7f7f80000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:21.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.257+0000 7f7f7b7fe640 1 -- 172.21.15.67:0/652228116 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7f7c0ae4e0 con 0x7f7f84103fc0 2024-09-21T06:48:21.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.496+0000 7f7f8c669640 1 -- 172.21.15.67:0/652228116 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f7f8402e810 con 0x7f7f84103fc0 2024-09-21T06:48:21.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.505+0000 7f7f7b7fe640 1 -- 172.21.15.67:0/652228116 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+833522 (secure 0 0 0) 0x7f7f7c0b28c0 con 0x7f7f84103fc0 2024-09-21T06:48:21.519 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:48:21.519 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-21T06:48:21.520 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 18, 2024-09-21T06:48:21.520 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.520 INFO:teuthology.orchestra.run.smithi067.stdout: "active_gid": 14152, 2024-09-21T06:48:21.520 INFO:teuthology.orchestra.run.smithi067.stdout: "active_name": "x", 2024-09-21T06:48:21.520 INFO:teuthology.orchestra.run.smithi067.stdout: "active_addrs": { 2024-09-21T06:48:21.520 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-09-21T06:48:21.520 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.520 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-09-21T06:48:21.520 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6800", 2024-09-21T06:48:21.521 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 1134893458 2024-09-21T06:48:21.521 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.521 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.521 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-09-21T06:48:21.521 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6801", 2024-09-21T06:48:21.521 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 1134893458 2024-09-21T06:48:21.521 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.521 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-21T06:48:21.521 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.522 INFO:teuthology.orchestra.run.smithi067.stdout: "active_addr": "172.21.15.67:6801/1134893458", 2024-09-21T06:48:21.522 INFO:teuthology.orchestra.run.smithi067.stdout: "active_change": "2024-09-21T06:27:03.740607+0000", 2024-09-21T06:48:21.522 INFO:teuthology.orchestra.run.smithi067.stdout: "active_mgr_features": 4540719139924082687, 2024-09-21T06:48:21.522 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-09-21T06:48:21.522 INFO:teuthology.orchestra.run.smithi067.stdout: "standbys": [ 2024-09-21T06:48:21.522 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.523 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 24121, 2024-09-21T06:48:21.523 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "z", 2024-09-21T06:48:21.523 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_features": 4540719139924082687, 2024-09-21T06:48:21.523 INFO:teuthology.orchestra.run.smithi067.stdout: "available_modules": [ 2024-09-21T06:48:21.523 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.523 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "alerts", 2024-09-21T06:48:21.523 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.523 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.523 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.523 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-09-21T06:48:21.524 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-09-21T06:48:21.524 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:21.524 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.524 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.524 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-09-21T06:48:21.524 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.524 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.524 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.524 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T06:48:21.524 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.524 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.524 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.524 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.524 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.525 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.525 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.525 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.525 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.525 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.525 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.525 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.525 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.525 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.525 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.525 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.525 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.525 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.526 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.526 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.526 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.526 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.526 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.526 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.526 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.526 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.526 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.526 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.526 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.526 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.526 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.527 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.527 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.527 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.527 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.527 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.527 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.527 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.527 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.527 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.527 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.527 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.527 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.527 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.527 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.528 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.528 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.528 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.528 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.528 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.528 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.528 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.528 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.528 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.528 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.528 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.528 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.528 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.528 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.528 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.529 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.529 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.529 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.529 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.529 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.529 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.529 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.529 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.529 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.529 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.529 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.529 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.530 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.530 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.530 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_destination": { 2024-09-21T06:48:21.530 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_destination", 2024-09-21T06:48:21.530 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.530 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.530 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.530 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.530 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.530 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.530 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.530 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T06:48:21.530 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.531 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.531 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.531 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.531 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_from_name": { 2024-09-21T06:48:21.531 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_from_name", 2024-09-21T06:48:21.531 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.531 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.531 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.531 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "Ceph", 2024-09-21T06:48:21.531 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.531 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.531 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.531 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Email From: name", 2024-09-21T06:48:21.531 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.532 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.532 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.532 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.532 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_host": { 2024-09-21T06:48:21.532 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_host", 2024-09-21T06:48:21.532 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.532 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.532 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.532 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.532 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.532 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.532 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.532 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP server", 2024-09-21T06:48:21.533 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.533 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.533 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.533 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.533 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_password": { 2024-09-21T06:48:21.533 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_password", 2024-09-21T06:48:21.533 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.533 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.533 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.534 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.534 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.534 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.534 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.534 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Password to authenticate with", 2024-09-21T06:48:21.534 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.534 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.534 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.534 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.534 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_port": { 2024-09-21T06:48:21.534 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_port", 2024-09-21T06:48:21.534 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.534 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.534 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.535 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "465", 2024-09-21T06:48:21.535 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.535 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.535 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.535 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP port", 2024-09-21T06:48:21.535 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.535 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.535 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.535 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.535 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_sender": { 2024-09-21T06:48:21.535 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_sender", 2024-09-21T06:48:21.535 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.535 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.535 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.536 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.536 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.536 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.536 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.536 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP envelope sender", 2024-09-21T06:48:21.536 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.536 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.536 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.536 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.536 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_ssl": { 2024-09-21T06:48:21.536 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_ssl", 2024-09-21T06:48:21.536 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.537 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.537 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.537 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.537 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.537 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.537 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.537 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T06:48:21.537 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.537 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.537 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.537 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.537 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_user": { 2024-09-21T06:48:21.537 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_user", 2024-09-21T06:48:21.537 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.538 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.538 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.538 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.538 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.538 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.538 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.538 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "User to authenticate as", 2024-09-21T06:48:21.538 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.538 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.538 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.538 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.538 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.538 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.538 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.539 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.539 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.539 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.539 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.539 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.539 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.539 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.539 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.539 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.539 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.539 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.539 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.539 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.539 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.539 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "balancer", 2024-09-21T06:48:21.540 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.540 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.540 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.540 INFO:teuthology.orchestra.run.smithi067.stdout: "active": { 2024-09-21T06:48:21.540 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "active", 2024-09-21T06:48:21.540 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.540 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.540 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.540 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.540 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.540 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.540 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.540 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T06:48:21.540 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.541 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.541 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.541 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.541 INFO:teuthology.orchestra.run.smithi067.stdout: "begin_time": { 2024-09-21T06:48:21.541 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "begin_time", 2024-09-21T06:48:21.541 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.541 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.541 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.541 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0000", 2024-09-21T06:48:21.541 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.541 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.541 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.541 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T06:48:21.541 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T06:48:21.541 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.542 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.542 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.542 INFO:teuthology.orchestra.run.smithi067.stdout: "begin_weekday": { 2024-09-21T06:48:21.542 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "begin_weekday", 2024-09-21T06:48:21.542 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-09-21T06:48:21.542 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.542 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.542 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.542 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0", 2024-09-21T06:48:21.542 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "6", 2024-09-21T06:48:21.542 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.542 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T06:48:21.542 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T06:48:21.542 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.543 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.543 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.543 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_max_iterations": { 2024-09-21T06:48:21.543 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_max_iterations", 2024-09-21T06:48:21.543 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-09-21T06:48:21.543 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.543 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.543 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "25", 2024-09-21T06:48:21.543 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-09-21T06:48:21.544 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "250", 2024-09-21T06:48:21.544 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.544 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T06:48:21.544 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.544 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.544 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.544 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.544 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_metrics": { 2024-09-21T06:48:21.544 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_metrics", 2024-09-21T06:48:21.544 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.544 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.544 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.544 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T06:48:21.544 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.545 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.545 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.545 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T06:48:21.545 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T06:48:21.545 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.545 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.545 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.545 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_step": { 2024-09-21T06:48:21.545 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_step", 2024-09-21T06:48:21.545 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-21T06:48:21.545 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.545 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.545 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0.5", 2024-09-21T06:48:21.545 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0.001", 2024-09-21T06:48:21.546 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "0.999", 2024-09-21T06:48:21.546 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.546 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "aggressiveness of optimization", 2024-09-21T06:48:21.546 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T06:48:21.546 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.546 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.546 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.546 INFO:teuthology.orchestra.run.smithi067.stdout: "end_time": { 2024-09-21T06:48:21.546 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "end_time", 2024-09-21T06:48:21.546 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.546 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.546 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.546 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "2359", 2024-09-21T06:48:21.546 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.546 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.547 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.547 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T06:48:21.547 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T06:48:21.547 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.547 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.547 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.547 INFO:teuthology.orchestra.run.smithi067.stdout: "end_weekday": { 2024-09-21T06:48:21.547 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "end_weekday", 2024-09-21T06:48:21.547 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-09-21T06:48:21.547 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.547 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.547 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.547 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0", 2024-09-21T06:48:21.547 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "6", 2024-09-21T06:48:21.548 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.548 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T06:48:21.548 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T06:48:21.548 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.548 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.548 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.548 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.548 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.548 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.548 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.548 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.548 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.548 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.548 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.548 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.549 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.549 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.549 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.549 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.549 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.549 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.549 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.549 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.549 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.549 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.549 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.549 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.550 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.550 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.550 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.550 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.550 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.550 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.550 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.550 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.550 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.550 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.550 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.550 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.550 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.550 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.551 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.551 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.551 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.551 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.551 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.551 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.551 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.551 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.551 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.551 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.551 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.551 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.551 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.551 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.551 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.552 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.552 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.552 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.552 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.552 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.552 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.552 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.552 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.552 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.552 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.552 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.552 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.552 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.552 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.553 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.553 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.553 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.553 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.553 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.553 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.553 INFO:teuthology.orchestra.run.smithi067.stdout: "min_score": { 2024-09-21T06:48:21.553 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "min_score", 2024-09-21T06:48:21.553 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-21T06:48:21.553 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.553 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.554 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.554 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.554 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.554 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.554 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T06:48:21.554 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.554 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.554 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.554 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.554 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": { 2024-09-21T06:48:21.554 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mode", 2024-09-21T06:48:21.554 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.555 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.555 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.555 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "upmap", 2024-09-21T06:48:21.555 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.555 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.555 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.555 INFO:teuthology.orchestra.run.smithi067.stdout: "crush-compat", 2024-09-21T06:48:21.555 INFO:teuthology.orchestra.run.smithi067.stdout: "none", 2024-09-21T06:48:21.555 INFO:teuthology.orchestra.run.smithi067.stdout: "read", 2024-09-21T06:48:21.555 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap", 2024-09-21T06:48:21.555 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap-read" 2024-09-21T06:48:21.555 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.555 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Balancer mode", 2024-09-21T06:48:21.555 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.556 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.556 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.556 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.556 INFO:teuthology.orchestra.run.smithi067.stdout: "pool_ids": { 2024-09-21T06:48:21.556 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pool_ids", 2024-09-21T06:48:21.556 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.556 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.556 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.556 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.556 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.556 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.556 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.556 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T06:48:21.556 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.557 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.557 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.557 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.557 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-09-21T06:48:21.557 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-09-21T06:48:21.557 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:21.557 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.557 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.557 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-09-21T06:48:21.557 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.557 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.557 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.557 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T06:48:21.557 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.557 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.558 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.558 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.558 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.558 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.558 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.558 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.558 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.558 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.558 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.558 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.558 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.558 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.558 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.559 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.559 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.559 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.559 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap_max_deviation": { 2024-09-21T06:48:21.559 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "upmap_max_deviation", 2024-09-21T06:48:21.559 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.559 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.559 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.559 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-09-21T06:48:21.559 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-09-21T06:48:21.559 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.559 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.559 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T06:48:21.559 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T06:48:21.559 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.560 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.560 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.560 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap_max_optimizations": { 2024-09-21T06:48:21.560 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "upmap_max_optimizations", 2024-09-21T06:48:21.560 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-09-21T06:48:21.560 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.560 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.560 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-21T06:48:21.560 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.560 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.560 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.560 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T06:48:21.560 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.560 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.561 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.561 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.561 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.561 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.561 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.561 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephadm", 2024-09-21T06:48:21.561 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.561 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.561 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.561 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_down_multiplier": { 2024-09-21T06:48:21.561 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_down_multiplier", 2024-09-21T06:48:21.561 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-21T06:48:21.561 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.561 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.562 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3.0", 2024-09-21T06:48:21.562 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.562 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.562 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.562 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T06:48:21.562 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.562 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.562 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.562 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.562 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_refresh_rate": { 2024-09-21T06:48:21.562 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_refresh_rate", 2024-09-21T06:48:21.562 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:21.562 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.563 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.563 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "20", 2024-09-21T06:48:21.563 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.563 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.563 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.563 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T06:48:21.563 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.563 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.563 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.563 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.563 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_starting_port": { 2024-09-21T06:48:21.563 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_starting_port", 2024-09-21T06:48:21.563 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.564 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.564 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.564 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "4721", 2024-09-21T06:48:21.564 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.564 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.564 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.564 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T06:48:21.564 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.564 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.564 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.564 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.564 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_ptrace": { 2024-09-21T06:48:21.564 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_ptrace", 2024-09-21T06:48:21.565 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.565 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.565 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.565 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.565 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.565 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.565 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.565 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T06:48:21.565 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T06:48:21.565 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.565 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.565 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.565 INFO:teuthology.orchestra.run.smithi067.stdout: "autotune_interval": { 2024-09-21T06:48:21.565 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "autotune_interval", 2024-09-21T06:48:21.566 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:21.566 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.566 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.566 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-09-21T06:48:21.566 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.566 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.566 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.566 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T06:48:21.566 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.566 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.566 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.566 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.566 INFO:teuthology.orchestra.run.smithi067.stdout: "autotune_memory_target_ratio": { 2024-09-21T06:48:21.566 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T06:48:21.567 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-21T06:48:21.567 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.567 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.567 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0.7", 2024-09-21T06:48:21.567 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.567 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.567 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.567 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T06:48:21.567 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.567 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.567 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.567 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.567 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm_log_destination": { 2024-09-21T06:48:21.567 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephadm_log_destination", 2024-09-21T06:48:21.568 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.568 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.568 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.568 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.568 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.568 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.568 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.568 INFO:teuthology.orchestra.run.smithi067.stdout: "file", 2024-09-21T06:48:21.568 INFO:teuthology.orchestra.run.smithi067.stdout: "file,syslog", 2024-09-21T06:48:21.568 INFO:teuthology.orchestra.run.smithi067.stdout: "syslog" 2024-09-21T06:48:21.568 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.568 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T06:48:21.568 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.568 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.568 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.569 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.569 INFO:teuthology.orchestra.run.smithi067.stdout: "cgroups_split": { 2024-09-21T06:48:21.569 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cgroups_split", 2024-09-21T06:48:21.569 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.569 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.569 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.569 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.569 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.569 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.569 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.569 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T06:48:21.569 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.569 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.569 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.569 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.570 INFO:teuthology.orchestra.run.smithi067.stdout: "config_checks_enabled": { 2024-09-21T06:48:21.570 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_checks_enabled", 2024-09-21T06:48:21.570 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.570 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.570 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.570 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.570 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.570 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.570 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.570 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T06:48:21.570 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.570 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.570 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.570 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.571 INFO:teuthology.orchestra.run.smithi067.stdout: "config_dashboard": { 2024-09-21T06:48:21.571 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_dashboard", 2024-09-21T06:48:21.571 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.571 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.571 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.571 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.571 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.571 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.571 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.571 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T06:48:21.571 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.571 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.571 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.571 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.572 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_alertmanager": { 2024-09-21T06:48:21.572 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_alertmanager", 2024-09-21T06:48:21.572 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.572 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.572 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.572 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T06:48:21.572 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.572 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.572 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.572 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-09-21T06:48:21.572 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.572 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.572 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.572 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.573 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_base": { 2024-09-21T06:48:21.573 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_base", 2024-09-21T06:48:21.573 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.573 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.573 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.573 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T06:48:21.573 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.573 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.573 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.573 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Container image name, without the tag", 2024-09-21T06:48:21.573 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.573 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.573 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.573 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.574 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_elasticsearch": { 2024-09-21T06:48:21.574 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_elasticsearch", 2024-09-21T06:48:21.574 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.574 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.574 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.574 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T06:48:21.574 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.574 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.574 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.574 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "elasticsearch container image", 2024-09-21T06:48:21.574 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.574 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.574 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.575 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.575 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_grafana": { 2024-09-21T06:48:21.575 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_grafana", 2024-09-21T06:48:21.575 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.575 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.575 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.575 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T06:48:21.575 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.575 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.575 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.575 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-09-21T06:48:21.575 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.575 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.575 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.575 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.576 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_haproxy": { 2024-09-21T06:48:21.576 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_haproxy", 2024-09-21T06:48:21.576 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.576 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.576 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.576 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T06:48:21.576 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.576 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.576 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.576 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "HAproxy container image", 2024-09-21T06:48:21.576 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.576 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.576 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.576 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.577 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_agent": { 2024-09-21T06:48:21.577 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_agent", 2024-09-21T06:48:21.577 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.577 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.577 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.577 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T06:48:21.577 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.577 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.577 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.577 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger agent container image", 2024-09-21T06:48:21.577 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.577 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.577 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.577 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.578 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_collector": { 2024-09-21T06:48:21.578 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_collector", 2024-09-21T06:48:21.578 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.578 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.578 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.578 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T06:48:21.578 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.578 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.578 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.578 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger collector container image", 2024-09-21T06:48:21.578 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.578 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.578 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.578 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.579 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_query": { 2024-09-21T06:48:21.579 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_query", 2024-09-21T06:48:21.579 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.579 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.579 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.579 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T06:48:21.579 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.579 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.579 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.579 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger query container image", 2024-09-21T06:48:21.579 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.579 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.579 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.579 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.579 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_keepalived": { 2024-09-21T06:48:21.580 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_keepalived", 2024-09-21T06:48:21.580 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.580 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.580 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.580 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T06:48:21.580 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.580 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.580 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.580 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Keepalived container image", 2024-09-21T06:48:21.580 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.580 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.580 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.580 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.580 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_loki": { 2024-09-21T06:48:21.581 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_loki", 2024-09-21T06:48:21.581 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.581 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.581 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.581 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T06:48:21.581 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.581 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.581 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.581 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Loki container image", 2024-09-21T06:48:21.581 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.581 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.581 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.581 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.581 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_nginx": { 2024-09-21T06:48:21.581 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_nginx", 2024-09-21T06:48:21.582 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.582 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.582 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.582 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T06:48:21.582 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.582 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.582 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.582 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Nginx container image", 2024-09-21T06:48:21.582 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.582 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.582 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.582 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.582 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_node_exporter": { 2024-09-21T06:48:21.582 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_node_exporter", 2024-09-21T06:48:21.583 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.583 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.583 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.583 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T06:48:21.583 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.583 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.583 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.583 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-09-21T06:48:21.583 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.583 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.583 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.583 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.583 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_nvmeof": { 2024-09-21T06:48:21.583 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_nvmeof", 2024-09-21T06:48:21.583 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.584 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.584 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.584 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T06:48:21.584 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.584 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.584 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.584 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Nvme-of container image", 2024-09-21T06:48:21.584 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.584 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.584 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.584 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.584 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_oauth2_proxy": { 2024-09-21T06:48:21.584 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T06:48:21.584 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.585 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.585 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.585 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T06:48:21.585 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.585 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.585 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.585 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "oauth2-proxy container image", 2024-09-21T06:48:21.585 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.585 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.585 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.585 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.585 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_prometheus": { 2024-09-21T06:48:21.585 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_prometheus", 2024-09-21T06:48:21.585 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.586 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.586 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.586 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T06:48:21.586 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.586 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.586 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.586 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-09-21T06:48:21.586 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.586 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.586 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.586 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.586 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_promtail": { 2024-09-21T06:48:21.587 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_promtail", 2024-09-21T06:48:21.587 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.587 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.587 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.587 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T06:48:21.587 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.587 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.587 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.587 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Promtail container image", 2024-09-21T06:48:21.587 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.587 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.587 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.587 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.587 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_samba": { 2024-09-21T06:48:21.588 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_samba", 2024-09-21T06:48:21.588 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.588 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.588 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.588 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T06:48:21.588 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.588 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.588 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.588 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Samba/SMB container image", 2024-09-21T06:48:21.588 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.588 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.588 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.588 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.588 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_samba_metrics": { 2024-09-21T06:48:21.588 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_samba_metrics", 2024-09-21T06:48:21.589 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.589 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.589 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.589 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T06:48:21.589 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.589 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.589 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.589 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T06:48:21.589 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.589 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.589 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.589 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.589 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_snmp_gateway": { 2024-09-21T06:48:21.589 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_snmp_gateway", 2024-09-21T06:48:21.590 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.590 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.590 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.590 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T06:48:21.590 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.590 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.590 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.590 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SNMP Gateway container image", 2024-09-21T06:48:21.590 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.590 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.590 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.590 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.590 INFO:teuthology.orchestra.run.smithi067.stdout: "container_init": { 2024-09-21T06:48:21.591 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_init", 2024-09-21T06:48:21.591 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.591 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.591 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.591 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.591 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.593 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.593 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.594 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T06:48:21.594 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.594 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.594 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.594 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.594 INFO:teuthology.orchestra.run.smithi067.stdout: "daemon_cache_timeout": { 2024-09-21T06:48:21.594 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "daemon_cache_timeout", 2024-09-21T06:48:21.594 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:21.594 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.594 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.594 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-09-21T06:48:21.594 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.594 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.595 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.595 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T06:48:21.595 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.595 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.595 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.595 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.595 INFO:teuthology.orchestra.run.smithi067.stdout: "default_cephadm_command_timeout": { 2024-09-21T06:48:21.595 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T06:48:21.595 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.595 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.595 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.595 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "900", 2024-09-21T06:48:21.595 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.595 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.595 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.596 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T06:48:21.596 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.596 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.596 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.596 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.596 INFO:teuthology.orchestra.run.smithi067.stdout: "default_registry": { 2024-09-21T06:48:21.596 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "default_registry", 2024-09-21T06:48:21.596 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.596 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.596 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.596 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io", 2024-09-21T06:48:21.596 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.596 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.596 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.596 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T06:48:21.597 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.597 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.597 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.597 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.597 INFO:teuthology.orchestra.run.smithi067.stdout: "device_cache_timeout": { 2024-09-21T06:48:21.597 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_cache_timeout", 2024-09-21T06:48:21.597 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:21.597 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.597 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.597 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1800", 2024-09-21T06:48:21.597 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.597 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.597 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.597 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache device inventory", 2024-09-21T06:48:21.598 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.598 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.598 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.598 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.598 INFO:teuthology.orchestra.run.smithi067.stdout: "device_enhanced_scan": { 2024-09-21T06:48:21.598 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_enhanced_scan", 2024-09-21T06:48:21.598 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.598 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.598 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.598 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.598 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.599 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.599 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.599 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T06:48:21.599 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.599 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.599 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.599 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.599 INFO:teuthology.orchestra.run.smithi067.stdout: "facts_cache_timeout": { 2024-09-21T06:48:21.599 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "facts_cache_timeout", 2024-09-21T06:48:21.599 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:21.599 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.599 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.599 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-09-21T06:48:21.599 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.600 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.600 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.600 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache host facts data", 2024-09-21T06:48:21.600 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.600 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.600 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.600 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.600 INFO:teuthology.orchestra.run.smithi067.stdout: "grafana_dashboards_path": { 2024-09-21T06:48:21.600 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "grafana_dashboards_path", 2024-09-21T06:48:21.600 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.600 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.600 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.600 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T06:48:21.600 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.601 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.601 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.601 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T06:48:21.601 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.601 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.601 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.601 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.601 INFO:teuthology.orchestra.run.smithi067.stdout: "host_check_interval": { 2024-09-21T06:48:21.601 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "host_check_interval", 2024-09-21T06:48:21.601 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:21.601 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.601 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.602 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-09-21T06:48:21.602 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.602 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.602 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.602 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to perform a host check", 2024-09-21T06:48:21.602 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.602 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.602 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.602 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.602 INFO:teuthology.orchestra.run.smithi067.stdout: "hw_monitoring": { 2024-09-21T06:48:21.602 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "hw_monitoring", 2024-09-21T06:48:21.602 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.602 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.602 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.603 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.603 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.603 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.603 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.603 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T06:48:21.603 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.603 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.603 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.603 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.603 INFO:teuthology.orchestra.run.smithi067.stdout: "inventory_list_all": { 2024-09-21T06:48:21.603 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "inventory_list_all", 2024-09-21T06:48:21.603 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.603 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.604 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.604 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.604 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.604 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.604 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.604 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T06:48:21.604 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.604 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.604 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.604 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.604 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.604 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.604 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.604 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.604 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.605 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.605 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.605 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.605 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.605 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.605 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.605 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.605 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.605 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.605 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.605 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.605 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.605 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.605 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.605 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.605 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.606 INFO:teuthology.orchestra.run.smithi067.stdout: "log_refresh_metadata": { 2024-09-21T06:48:21.606 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_refresh_metadata", 2024-09-21T06:48:21.606 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.606 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.606 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.606 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.606 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.606 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.606 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.606 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T06:48:21.606 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.606 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.606 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.606 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.607 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.607 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.607 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.607 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.607 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.607 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.607 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.607 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.607 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.607 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T06:48:21.607 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.607 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.607 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.608 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.608 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.608 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.608 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.608 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.608 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.608 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.608 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.608 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.608 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.608 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.608 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.608 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.608 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.608 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.608 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.608 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.609 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.609 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.609 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.609 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.609 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.609 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.609 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.609 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.609 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.609 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.609 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.609 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.609 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.609 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.609 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.609 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.610 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.610 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.610 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.610 INFO:teuthology.orchestra.run.smithi067.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T06:48:21.610 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T06:48:21.610 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.610 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.610 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.610 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.610 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.610 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.610 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.610 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T06:48:21.610 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.610 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.610 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.611 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.611 INFO:teuthology.orchestra.run.smithi067.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T06:48:21.611 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T06:48:21.611 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.611 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.611 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.611 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "*", 2024-09-21T06:48:21.611 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.611 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.611 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.611 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T06:48:21.611 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.611 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.611 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.611 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.611 INFO:teuthology.orchestra.run.smithi067.stdout: "max_count_per_host": { 2024-09-21T06:48:21.612 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_count_per_host", 2024-09-21T06:48:21.612 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.612 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.612 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.612 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-21T06:48:21.612 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.612 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.612 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.612 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "max number of daemons per service per host", 2024-09-21T06:48:21.612 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.612 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.612 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.612 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.612 INFO:teuthology.orchestra.run.smithi067.stdout: "max_osd_draining_count": { 2024-09-21T06:48:21.612 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_osd_draining_count", 2024-09-21T06:48:21.613 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.613 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.613 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.613 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-21T06:48:21.613 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.613 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.613 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.613 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T06:48:21.613 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.613 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.613 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.613 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.613 INFO:teuthology.orchestra.run.smithi067.stdout: "migration_current": { 2024-09-21T06:48:21.614 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "migration_current", 2024-09-21T06:48:21.614 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.614 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.614 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.614 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.614 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.614 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.614 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.614 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "internal - do not modify", 2024-09-21T06:48:21.614 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.614 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.614 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.614 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.614 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": { 2024-09-21T06:48:21.614 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mode", 2024-09-21T06:48:21.615 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.615 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.615 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.615 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "root", 2024-09-21T06:48:21.615 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.615 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.615 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.615 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm-package", 2024-09-21T06:48:21.615 INFO:teuthology.orchestra.run.smithi067.stdout: "root" 2024-09-21T06:48:21.615 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.615 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T06:48:21.615 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.615 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.616 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.616 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.616 INFO:teuthology.orchestra.run.smithi067.stdout: "oob_default_addr": { 2024-09-21T06:48:21.616 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "oob_default_addr", 2024-09-21T06:48:21.616 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.616 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.616 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.616 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "169.254.1.1", 2024-09-21T06:48:21.616 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.616 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.616 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.616 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T06:48:21.616 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.616 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.617 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.617 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.617 INFO:teuthology.orchestra.run.smithi067.stdout: "prometheus_alerts_path": { 2024-09-21T06:48:21.617 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prometheus_alerts_path", 2024-09-21T06:48:21.617 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.617 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.617 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.617 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T06:48:21.617 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.617 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.617 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.617 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T06:48:21.617 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.618 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.618 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.618 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.618 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_insecure": { 2024-09-21T06:48:21.618 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_insecure", 2024-09-21T06:48:21.618 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.618 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.618 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.618 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.618 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.618 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.618 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.618 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T06:48:21.619 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.619 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.619 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.619 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.619 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_password": { 2024-09-21T06:48:21.619 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_password", 2024-09-21T06:48:21.619 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.619 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.619 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.619 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.619 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.619 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.619 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.619 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T06:48:21.620 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.620 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.620 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.620 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.620 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_url": { 2024-09-21T06:48:21.620 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_url", 2024-09-21T06:48:21.620 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.620 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.620 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.620 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.620 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.620 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.620 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.620 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T06:48:21.620 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.620 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.621 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.621 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.621 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_username": { 2024-09-21T06:48:21.621 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_username", 2024-09-21T06:48:21.621 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.621 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.621 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.621 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.621 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.621 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.621 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.621 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T06:48:21.621 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.621 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.621 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.621 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.622 INFO:teuthology.orchestra.run.smithi067.stdout: "secure_monitoring_stack": { 2024-09-21T06:48:21.622 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "secure_monitoring_stack", 2024-09-21T06:48:21.622 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.622 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.622 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.622 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.622 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.622 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.622 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.622 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T06:48:21.622 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.622 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.622 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.622 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.622 INFO:teuthology.orchestra.run.smithi067.stdout: "service_discovery_port": { 2024-09-21T06:48:21.623 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "service_discovery_port", 2024-09-21T06:48:21.623 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.623 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.623 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.623 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8765", 2024-09-21T06:48:21.623 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.623 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.623 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.623 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "cephadm service discovery port", 2024-09-21T06:48:21.623 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.623 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.623 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.623 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.623 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.623 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.623 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.623 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.624 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.624 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.624 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.624 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.624 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.624 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.624 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.624 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.624 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.624 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.624 INFO:teuthology.orchestra.run.smithi067.stdout: "ssh_config_file": { 2024-09-21T06:48:21.624 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssh_config_file", 2024-09-21T06:48:21.624 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.624 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.624 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.625 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.625 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.625 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.625 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.625 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T06:48:21.625 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.625 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.625 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.625 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.625 INFO:teuthology.orchestra.run.smithi067.stdout: "ssh_keepalive_count_max": { 2024-09-21T06:48:21.625 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T06:48:21.625 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.625 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.625 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.626 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3", 2024-09-21T06:48:21.626 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.626 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.626 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.626 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T06:48:21.626 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.626 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.626 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.626 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.626 INFO:teuthology.orchestra.run.smithi067.stdout: "ssh_keepalive_interval": { 2024-09-21T06:48:21.626 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssh_keepalive_interval", 2024-09-21T06:48:21.626 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.626 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.626 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.627 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7", 2024-09-21T06:48:21.627 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.627 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.627 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.627 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T06:48:21.627 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.627 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.627 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.627 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.627 INFO:teuthology.orchestra.run.smithi067.stdout: "use_agent": { 2024-09-21T06:48:21.627 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "use_agent", 2024-09-21T06:48:21.627 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.627 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.627 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.627 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.627 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.628 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.628 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.628 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T06:48:21.628 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.628 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.628 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.628 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.628 INFO:teuthology.orchestra.run.smithi067.stdout: "use_repo_digest": { 2024-09-21T06:48:21.628 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "use_repo_digest", 2024-09-21T06:48:21.628 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.628 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.628 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.628 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.628 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T06:48:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_failed_host_check": { 2024-09-21T06:48:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_failed_host_check", 2024-09-21T06:48:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.630 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.630 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T06:48:21.630 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.630 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.630 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.630 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.630 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_stray_daemons": { 2024-09-21T06:48:21.630 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_stray_daemons", 2024-09-21T06:48:21.630 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.630 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.630 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.630 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.630 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.630 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.630 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.631 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T06:48:21.631 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.631 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.631 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.631 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.631 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_stray_hosts": { 2024-09-21T06:48:21.631 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_stray_hosts", 2024-09-21T06:48:21.631 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.631 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.631 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.631 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.631 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.631 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.631 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.631 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T06:48:21.632 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.632 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.632 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.632 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.632 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.632 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.632 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.632 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crash", 2024-09-21T06:48:21.632 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.632 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.632 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.632 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.632 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.632 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.632 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.633 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.633 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.633 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.633 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.633 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.633 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.633 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.633 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.633 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.633 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.633 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.633 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.633 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.633 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.633 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.633 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.634 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.634 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.634 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.634 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.634 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.634 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.634 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.634 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.634 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.634 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.634 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.634 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.634 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.634 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.634 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.635 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.635 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.635 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.635 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.635 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.635 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.635 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.635 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.635 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.635 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.635 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.635 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.635 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.635 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.635 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.636 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.636 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.636 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.636 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.636 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.636 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.636 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.636 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.636 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.636 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.636 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.636 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.636 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.636 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.636 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.637 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.637 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.637 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.637 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.637 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.637 INFO:teuthology.orchestra.run.smithi067.stdout: "retain_interval": { 2024-09-21T06:48:21.637 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "retain_interval", 2024-09-21T06:48:21.637 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:21.637 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.637 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.637 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "31536000", 2024-09-21T06:48:21.637 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.637 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.638 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.638 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T06:48:21.638 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.638 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.638 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.638 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.638 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.638 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.638 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.638 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.638 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.638 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.638 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.638 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.639 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.639 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.639 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.639 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.639 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.639 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.639 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_recent_interval": { 2024-09-21T06:48:21.639 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_recent_interval", 2024-09-21T06:48:21.639 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:21.639 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.639 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.639 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1209600", 2024-09-21T06:48:21.639 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.639 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.640 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.640 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T06:48:21.640 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.640 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.640 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.640 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.640 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.640 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.640 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.640 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "dashboard", 2024-09-21T06:48:21.640 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.640 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.640 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.640 INFO:teuthology.orchestra.run.smithi067.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T06:48:21.641 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T06:48:21.641 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.641 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.641 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.641 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-21T06:48:21.641 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.641 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.641 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.641 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.641 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.641 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.641 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.641 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.641 INFO:teuthology.orchestra.run.smithi067.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T06:48:21.642 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T06:48:21.642 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.642 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.642 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.642 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.642 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.642 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.642 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.642 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.642 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.642 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.642 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.642 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.642 INFO:teuthology.orchestra.run.smithi067.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T06:48:21.643 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T06:48:21.643 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.643 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.643 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.643 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.643 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.643 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.643 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.643 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.643 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.643 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.643 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.643 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.643 INFO:teuthology.orchestra.run.smithi067.stdout: "AUDIT_API_ENABLED": { 2024-09-21T06:48:21.644 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T06:48:21.644 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.644 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.644 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.644 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.644 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.644 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.644 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.644 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.644 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.644 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.644 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.644 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.644 INFO:teuthology.orchestra.run.smithi067.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T06:48:21.644 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T06:48:21.645 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.645 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.645 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.645 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.645 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.645 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.645 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.645 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.645 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.645 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.645 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.645 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.645 INFO:teuthology.orchestra.run.smithi067.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T06:48:21.645 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T06:48:21.645 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.645 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.646 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.646 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.646 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.646 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.646 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.646 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.646 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.646 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.646 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.646 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.646 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T06:48:21.646 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T06:48:21.646 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.646 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.647 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.647 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.647 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.647 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.647 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.647 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.647 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.647 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.647 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.647 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.647 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T06:48:21.647 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T06:48:21.647 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.647 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.648 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.648 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.648 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.648 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.648 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.648 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.648 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.648 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.648 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.648 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.648 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T06:48:21.648 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T06:48:21.648 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.648 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.648 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.648 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.649 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.649 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.649 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.649 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.649 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.649 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.649 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.649 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.649 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T06:48:21.649 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T06:48:21.649 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.649 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.649 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.649 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.649 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.650 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.650 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.650 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.650 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.650 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.650 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.650 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.650 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T06:48:21.650 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T06:48:21.650 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.650 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.650 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.651 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.651 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.651 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.651 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.651 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.651 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.651 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.651 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.651 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.651 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T06:48:21.651 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T06:48:21.651 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.651 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.652 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.652 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.652 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.652 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.652 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.652 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.652 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.652 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.652 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.652 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.652 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T06:48:21.652 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T06:48:21.652 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.652 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.652 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.652 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.653 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.653 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.653 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.653 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.653 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.653 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.653 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.653 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.653 INFO:teuthology.orchestra.run.smithi067.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T06:48:21.653 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T06:48:21.653 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.653 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.653 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.653 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.654 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.654 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.654 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.654 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.654 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.654 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.654 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.654 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.654 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T06:48:21.654 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T06:48:21.654 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.654 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.654 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.654 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-09-21T06:48:21.655 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.655 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.655 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.655 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.655 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.655 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.655 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.655 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.655 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T06:48:21.655 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T06:48:21.655 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.655 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.655 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.656 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.656 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.656 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.656 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.656 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.656 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.656 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.656 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.656 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.656 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_URL": { 2024-09-21T06:48:21.656 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_URL", 2024-09-21T06:48:21.656 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.656 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.656 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.656 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.657 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.657 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.657 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.657 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.657 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.657 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.657 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.657 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.657 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T06:48:21.657 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T06:48:21.657 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.657 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.657 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.657 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-09-21T06:48:21.657 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.658 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.658 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.658 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.658 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.658 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.658 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.658 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.658 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T06:48:21.658 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T06:48:21.658 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.658 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.658 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.658 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.658 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.659 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.659 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.659 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.659 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.659 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.659 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.659 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.659 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T06:48:21.659 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T06:48:21.659 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.659 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.659 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.659 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.659 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.659 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.660 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.660 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.660 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.660 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.660 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.660 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.660 INFO:teuthology.orchestra.run.smithi067.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T06:48:21.660 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T06:48:21.660 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.660 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.660 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.660 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.660 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.660 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.661 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.661 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.661 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.661 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.661 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.661 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.661 INFO:teuthology.orchestra.run.smithi067.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T06:48:21.661 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T06:48:21.661 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.661 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.661 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.661 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.661 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.661 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.661 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.662 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.662 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.662 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.662 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.662 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.662 INFO:teuthology.orchestra.run.smithi067.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T06:48:21.662 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T06:48:21.662 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.662 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.662 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.662 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "[]", 2024-09-21T06:48:21.662 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.662 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.662 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.663 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.663 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.663 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.663 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.663 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.663 INFO:teuthology.orchestra.run.smithi067.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T06:48:21.663 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T06:48:21.663 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.663 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.663 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.663 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "{}", 2024-09-21T06:48:21.663 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.664 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.664 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.664 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.664 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.664 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.664 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.664 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.664 INFO:teuthology.orchestra.run.smithi067.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T06:48:21.664 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T06:48:21.664 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.664 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.664 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.665 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.665 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.665 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.665 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.665 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.665 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.665 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.665 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.665 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.665 INFO:teuthology.orchestra.run.smithi067.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T06:48:21.665 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T06:48:21.665 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.665 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.665 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.666 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.666 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.666 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.666 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.666 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.666 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.666 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.666 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.666 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.666 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T06:48:21.666 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T06:48:21.666 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.666 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.667 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.667 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.667 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.667 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.667 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.667 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.667 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.667 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.667 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.667 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.667 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T06:48:21.667 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T06:48:21.667 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.668 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.668 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.668 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.668 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.668 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.668 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.668 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.668 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.668 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.668 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.668 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.668 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T06:48:21.668 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T06:48:21.668 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.669 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.669 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.669 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.669 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.669 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.669 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.669 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.669 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.669 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.669 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.669 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.669 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T06:48:21.669 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T06:48:21.669 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.670 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.670 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.670 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.670 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.670 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.670 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.670 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.670 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.670 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.670 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.670 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.670 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T06:48:21.670 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T06:48:21.670 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.671 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.671 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.671 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.671 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.671 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.671 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.671 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.671 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.671 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.671 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.671 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.671 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T06:48:21.671 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T06:48:21.671 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.671 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.672 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.672 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.672 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.672 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.672 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.672 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.672 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.672 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.672 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.672 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.672 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T06:48:21.672 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T06:48:21.672 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.672 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.672 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.673 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.673 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.673 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.673 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.673 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.673 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.673 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.673 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.673 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.673 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T06:48:21.673 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T06:48:21.673 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.673 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.673 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.673 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.673 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.674 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.674 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.674 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.674 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.674 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.674 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.674 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.674 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T06:48:21.674 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T06:48:21.674 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.674 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.674 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.674 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T06:48:21.674 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.675 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.675 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.675 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.675 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.675 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.675 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.675 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.675 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T06:48:21.675 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T06:48:21.675 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.675 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.675 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.675 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-21T06:48:21.675 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.676 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.676 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.676 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.676 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.676 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.676 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.676 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.676 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T06:48:21.676 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T06:48:21.676 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.676 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.676 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.676 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8", 2024-09-21T06:48:21.677 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.677 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.677 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.677 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.677 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.677 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.677 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.677 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.677 INFO:teuthology.orchestra.run.smithi067.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T06:48:21.677 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T06:48:21.677 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.677 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.677 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.677 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "45", 2024-09-21T06:48:21.678 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.678 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.678 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.678 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.678 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.678 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.678 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.678 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.678 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T06:48:21.678 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T06:48:21.678 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.678 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.678 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.678 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.679 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.679 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.679 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.679 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.679 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.679 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.679 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.679 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.679 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T06:48:21.679 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T06:48:21.679 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.679 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.679 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.679 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-09-21T06:48:21.680 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.680 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.680 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.680 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.680 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.680 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.680 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.680 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.680 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T06:48:21.680 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T06:48:21.680 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.680 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.680 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.680 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.681 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.681 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.681 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.681 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.681 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.681 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.681 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.681 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.681 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T06:48:21.681 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T06:48:21.681 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.681 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.681 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.681 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.682 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.682 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.682 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.682 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.682 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.682 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.682 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.682 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.682 INFO:teuthology.orchestra.run.smithi067.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T06:48:21.682 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T06:48:21.682 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.682 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.682 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.682 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.682 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.683 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.683 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.683 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.683 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.683 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.683 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.683 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.683 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T06:48:21.683 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T06:48:21.683 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.683 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.683 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.683 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.683 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.684 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.684 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.684 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.684 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.684 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.684 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.684 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.684 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T06:48:21.684 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T06:48:21.684 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.684 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.684 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.684 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-21T06:48:21.684 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.684 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.685 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.685 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.685 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.685 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.685 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.685 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.685 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T06:48:21.685 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T06:48:21.685 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.685 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.685 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.685 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-09-21T06:48:21.685 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.685 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.685 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.686 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.686 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.686 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.686 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.686 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.686 INFO:teuthology.orchestra.run.smithi067.stdout: "cross_origin_url": { 2024-09-21T06:48:21.686 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cross_origin_url", 2024-09-21T06:48:21.686 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.686 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.686 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.686 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.686 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.686 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.687 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.687 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.687 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.687 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.687 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.687 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.687 INFO:teuthology.orchestra.run.smithi067.stdout: "crt_file": { 2024-09-21T06:48:21.687 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crt_file", 2024-09-21T06:48:21.687 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.687 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.687 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.687 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.687 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.687 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.687 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.688 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.688 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.688 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.688 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.688 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.688 INFO:teuthology.orchestra.run.smithi067.stdout: "debug": { 2024-09-21T06:48:21.688 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "debug", 2024-09-21T06:48:21.688 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.688 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.688 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.688 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.688 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.688 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.688 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.689 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable/disable debug options", 2024-09-21T06:48:21.689 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.689 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.689 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.689 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.689 INFO:teuthology.orchestra.run.smithi067.stdout: "jwt_token_ttl": { 2024-09-21T06:48:21.689 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "jwt_token_ttl", 2024-09-21T06:48:21.689 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.689 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.689 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.689 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "28800", 2024-09-21T06:48:21.689 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.690 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.690 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.690 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.690 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.690 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.690 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.690 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.690 INFO:teuthology.orchestra.run.smithi067.stdout: "key_file": { 2024-09-21T06:48:21.690 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "key_file", 2024-09-21T06:48:21.690 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.690 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.690 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.690 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.690 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.691 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.691 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.691 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.691 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.691 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.691 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.691 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.691 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.691 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.691 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.691 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.691 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.691 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.691 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.692 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.692 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.692 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.692 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.692 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.692 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.692 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.692 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.692 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.692 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.692 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.692 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.692 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.692 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.693 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.693 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.693 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.693 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.693 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.693 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.693 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.693 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.693 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.693 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.693 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.693 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.693 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.694 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.694 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.694 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.694 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.694 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.694 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.694 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.694 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.694 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.694 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.694 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.694 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.694 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.694 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.695 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.695 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.695 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.695 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.695 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.695 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.695 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.695 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.695 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.695 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.695 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.695 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.695 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.695 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.695 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.695 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.696 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.696 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.696 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.696 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.696 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.696 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.696 INFO:teuthology.orchestra.run.smithi067.stdout: "motd": { 2024-09-21T06:48:21.696 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "motd", 2024-09-21T06:48:21.696 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.696 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.696 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.696 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.696 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.696 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.697 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.697 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "The message of the day", 2024-09-21T06:48:21.697 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.697 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.697 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.697 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.697 INFO:teuthology.orchestra.run.smithi067.stdout: "redirect_resolve_ip_addr": { 2024-09-21T06:48:21.697 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T06:48:21.697 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.697 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.697 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.697 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.697 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.698 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.698 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.698 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.698 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.698 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.698 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.698 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.698 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-09-21T06:48:21.698 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-09-21T06:48:21.698 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.698 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.698 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.698 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "::", 2024-09-21T06:48:21.698 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.698 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.699 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.699 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.699 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.699 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.699 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.699 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.699 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-09-21T06:48:21.699 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-09-21T06:48:21.699 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.699 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.699 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.699 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8080", 2024-09-21T06:48:21.699 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.699 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.700 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.700 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.700 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.700 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.700 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.700 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.700 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.700 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.700 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.700 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.700 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.700 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.700 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.700 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.700 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.701 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.701 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.701 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.701 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.701 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.701 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl": { 2024-09-21T06:48:21.701 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl", 2024-09-21T06:48:21.701 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.701 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.701 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.701 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.701 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.701 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.701 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.702 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.702 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.702 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.702 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.702 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.702 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl_server_port": { 2024-09-21T06:48:21.702 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl_server_port", 2024-09-21T06:48:21.702 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.702 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.702 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.702 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8443", 2024-09-21T06:48:21.702 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.702 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.703 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.703 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.703 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.703 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.703 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.703 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.703 INFO:teuthology.orchestra.run.smithi067.stdout: "sso_oauth2": { 2024-09-21T06:48:21.703 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sso_oauth2", 2024-09-21T06:48:21.703 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.703 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.703 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.703 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.703 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.703 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.704 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.704 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.704 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.704 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.704 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.704 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.704 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_behaviour": { 2024-09-21T06:48:21.704 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_behaviour", 2024-09-21T06:48:21.704 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.704 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.704 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.704 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "redirect", 2024-09-21T06:48:21.704 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.704 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.704 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.704 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.705 INFO:teuthology.orchestra.run.smithi067.stdout: "redirect" 2024-09-21T06:48:21.705 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.705 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.705 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.705 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.705 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.705 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.705 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_error_status_code": { 2024-09-21T06:48:21.705 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_error_status_code", 2024-09-21T06:48:21.705 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.705 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.705 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.705 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-09-21T06:48:21.706 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "400", 2024-09-21T06:48:21.706 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "599", 2024-09-21T06:48:21.706 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.706 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.706 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.706 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.706 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.706 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.706 INFO:teuthology.orchestra.run.smithi067.stdout: "url_prefix": { 2024-09-21T06:48:21.706 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "url_prefix", 2024-09-21T06:48:21.706 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.706 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.706 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.706 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.707 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.707 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.707 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.707 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.707 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.707 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.707 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.707 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.707 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.707 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.707 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.707 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "devicehealth", 2024-09-21T06:48:21.707 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.707 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.708 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.708 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_monitoring": { 2024-09-21T06:48:21.708 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enable_monitoring", 2024-09-21T06:48:21.708 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.708 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.708 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.708 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.708 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.708 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.708 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.708 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "monitor device health metrics", 2024-09-21T06:48:21.708 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.708 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.708 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.709 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.709 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.709 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.709 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.709 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.709 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.709 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.709 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.709 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.709 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.709 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.709 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.709 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.709 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.710 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.710 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.710 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.710 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.710 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.710 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.710 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.710 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.710 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.710 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.710 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.710 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.710 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.710 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.711 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.711 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.711 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.711 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.711 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.711 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.711 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.711 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.711 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.711 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.711 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.711 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.711 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.711 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.711 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.712 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.712 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.712 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.712 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.712 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.712 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.712 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.712 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.712 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.712 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.712 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.712 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.712 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.712 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.713 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.713 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.713 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.713 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.713 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.713 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.713 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.713 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.713 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.713 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.713 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.713 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.713 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.713 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.713 INFO:teuthology.orchestra.run.smithi067.stdout: "mark_out_threshold": { 2024-09-21T06:48:21.713 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mark_out_threshold", 2024-09-21T06:48:21.714 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:21.714 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.714 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.714 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "2419200", 2024-09-21T06:48:21.714 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.714 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.714 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.714 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T06:48:21.714 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.714 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.714 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.714 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.714 INFO:teuthology.orchestra.run.smithi067.stdout: "pool_name": { 2024-09-21T06:48:21.714 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pool_name", 2024-09-21T06:48:21.715 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.715 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.715 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.715 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "device_health_metrics", 2024-09-21T06:48:21.715 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.715 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.715 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.715 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T06:48:21.715 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.715 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.715 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.715 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.715 INFO:teuthology.orchestra.run.smithi067.stdout: "retention_period": { 2024-09-21T06:48:21.715 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "retention_period", 2024-09-21T06:48:21.715 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:21.716 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.716 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.716 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15552000", 2024-09-21T06:48:21.716 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.716 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.716 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.716 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long to retain device health metrics", 2024-09-21T06:48:21.716 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.716 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.716 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.716 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.716 INFO:teuthology.orchestra.run.smithi067.stdout: "scrape_frequency": { 2024-09-21T06:48:21.716 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "scrape_frequency", 2024-09-21T06:48:21.717 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:21.717 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.717 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.717 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "86400", 2024-09-21T06:48:21.717 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.717 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.717 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.717 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T06:48:21.717 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.717 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.717 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.717 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.717 INFO:teuthology.orchestra.run.smithi067.stdout: "self_heal": { 2024-09-21T06:48:21.717 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "self_heal", 2024-09-21T06:48:21.718 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.718 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.718 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.718 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.718 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.718 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.718 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.718 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T06:48:21.718 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.718 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.718 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.718 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.718 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-09-21T06:48:21.718 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-09-21T06:48:21.719 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:21.719 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.719 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.719 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-09-21T06:48:21.719 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.719 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.719 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.719 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T06:48:21.719 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.719 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.719 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.719 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.719 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.719 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.719 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.719 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.720 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.720 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.720 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.720 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.720 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.720 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.720 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.720 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.720 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.720 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.720 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_threshold": { 2024-09-21T06:48:21.720 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_threshold", 2024-09-21T06:48:21.720 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:21.720 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.720 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.720 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7257600", 2024-09-21T06:48:21.721 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.721 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.721 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.721 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T06:48:21.721 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.721 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.721 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.721 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.721 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.721 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.721 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.721 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "diskprediction_local", 2024-09-21T06:48:21.721 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.721 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.722 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.722 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.722 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.722 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.722 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.722 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.722 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.722 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.722 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.722 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.722 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.722 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.722 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.722 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.722 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.722 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.723 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.723 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.723 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.723 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.723 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.723 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.723 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.723 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.723 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.723 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.723 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.723 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.723 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.723 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.723 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.723 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.724 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.724 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.724 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.724 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.724 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.724 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.724 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.724 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.724 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.724 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.724 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.724 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.724 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.724 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.724 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.725 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.725 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.725 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.725 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.725 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.725 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.725 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.725 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.725 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.725 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.725 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.725 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.725 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.725 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.725 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.726 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.726 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.726 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.726 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.726 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.726 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.726 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.726 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.726 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.726 INFO:teuthology.orchestra.run.smithi067.stdout: "predict_interval": { 2024-09-21T06:48:21.726 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "predict_interval", 2024-09-21T06:48:21.726 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.726 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.726 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.726 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "86400", 2024-09-21T06:48:21.727 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.727 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.727 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.727 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.727 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.727 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.727 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.727 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.727 INFO:teuthology.orchestra.run.smithi067.stdout: "predictor_model": { 2024-09-21T06:48:21.727 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "predictor_model", 2024-09-21T06:48:21.727 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.727 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.727 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.727 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "prophetstor", 2024-09-21T06:48:21.727 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.727 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.728 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.728 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.728 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.728 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.728 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.728 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.728 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-09-21T06:48:21.728 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-09-21T06:48:21.728 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.728 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.728 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.728 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-09-21T06:48:21.728 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.728 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.729 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.729 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.729 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.729 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.729 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.729 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.729 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.729 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.729 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.729 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.729 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.729 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.729 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.729 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.729 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.730 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.730 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.730 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.730 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.730 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.730 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.730 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.730 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.730 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "influx", 2024-09-21T06:48:21.730 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": false, 2024-09-21T06:48:21.730 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "influxdb python module not found", 2024-09-21T06:48:21.730 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.730 INFO:teuthology.orchestra.run.smithi067.stdout: "batch_size": { 2024-09-21T06:48:21.730 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "batch_size", 2024-09-21T06:48:21.731 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.731 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.731 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.731 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5000", 2024-09-21T06:48:21.731 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.731 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.731 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.731 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T06:48:21.731 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.731 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.731 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.731 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.731 INFO:teuthology.orchestra.run.smithi067.stdout: "database": { 2024-09-21T06:48:21.731 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "database", 2024-09-21T06:48:21.731 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.731 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.732 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.732 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "ceph", 2024-09-21T06:48:21.732 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.732 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.732 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.732 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T06:48:21.732 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.732 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.732 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.732 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.732 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": { 2024-09-21T06:48:21.732 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "hostname", 2024-09-21T06:48:21.732 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.732 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.733 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.733 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.733 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.733 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.733 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.733 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB server hostname", 2024-09-21T06:48:21.733 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.733 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.733 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.733 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.733 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-09-21T06:48:21.733 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-09-21T06:48:21.733 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:21.733 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.733 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.733 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "30", 2024-09-21T06:48:21.734 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "5", 2024-09-21T06:48:21.734 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.734 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.734 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T06:48:21.734 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.734 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.734 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.734 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.734 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.734 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.734 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.734 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.734 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.734 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.735 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.735 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.735 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.735 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.735 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.735 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.735 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.735 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.735 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.735 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.735 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.735 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.735 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.735 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.735 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.735 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.736 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.736 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.736 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.736 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.736 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.736 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.736 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.736 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.736 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.736 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.736 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.736 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.736 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.736 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.736 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.737 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.737 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.737 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.737 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.737 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.737 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.737 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.737 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.737 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.737 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.737 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.737 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.737 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.737 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.737 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.738 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.738 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.738 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.738 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.738 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.738 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.738 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.738 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.738 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.738 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.738 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.738 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.738 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.738 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.739 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.739 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.739 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.739 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.739 INFO:teuthology.orchestra.run.smithi067.stdout: "password": { 2024-09-21T06:48:21.739 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "password", 2024-09-21T06:48:21.739 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.739 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.739 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.739 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.739 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.739 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.739 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.739 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "password of InfluxDB server user", 2024-09-21T06:48:21.739 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.739 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.740 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.740 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.740 INFO:teuthology.orchestra.run.smithi067.stdout: "port": { 2024-09-21T06:48:21.740 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "port", 2024-09-21T06:48:21.740 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.740 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.740 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.740 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8086", 2024-09-21T06:48:21.740 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.740 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.740 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.740 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB server port", 2024-09-21T06:48:21.740 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.740 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.741 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.741 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.741 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.741 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.741 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.741 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.741 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.741 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.741 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.741 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.741 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.741 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.742 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.742 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.742 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.742 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.742 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl": { 2024-09-21T06:48:21.742 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl", 2024-09-21T06:48:21.742 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.742 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.742 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.742 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "false", 2024-09-21T06:48:21.742 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.742 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.742 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.742 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T06:48:21.743 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.743 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.743 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.743 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.743 INFO:teuthology.orchestra.run.smithi067.stdout: "threads": { 2024-09-21T06:48:21.743 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "threads", 2024-09-21T06:48:21.743 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.743 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.743 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.743 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-09-21T06:48:21.743 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-09-21T06:48:21.743 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "32", 2024-09-21T06:48:21.743 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.743 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T06:48:21.744 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.744 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.744 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.744 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.744 INFO:teuthology.orchestra.run.smithi067.stdout: "username": { 2024-09-21T06:48:21.744 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "username", 2024-09-21T06:48:21.744 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.744 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.744 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.744 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.744 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.744 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.744 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.745 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "username of InfluxDB server user", 2024-09-21T06:48:21.745 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.745 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.745 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.745 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.745 INFO:teuthology.orchestra.run.smithi067.stdout: "verify_ssl": { 2024-09-21T06:48:21.745 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "verify_ssl", 2024-09-21T06:48:21.745 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.745 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.745 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.745 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "true", 2024-09-21T06:48:21.745 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.745 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.745 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.746 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T06:48:21.746 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.746 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.746 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.746 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.746 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.746 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.746 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.746 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "insights", 2024-09-21T06:48:21.746 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.746 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.746 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.746 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.746 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.747 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.747 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.747 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.747 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.747 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.747 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.747 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.747 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.747 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.747 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.747 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.747 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.747 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.747 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.748 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.748 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.748 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.748 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.748 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.748 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.748 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.748 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.748 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.748 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.748 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.748 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.748 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.748 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.748 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.749 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.749 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.749 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.749 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.749 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.749 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.749 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.749 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.749 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.749 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.749 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.749 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.749 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.749 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.749 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.750 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.750 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.750 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.750 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.750 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.750 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.750 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.750 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.750 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.750 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.750 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.750 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.750 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.750 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.750 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.750 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.751 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.751 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.751 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.751 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.751 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.751 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.751 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.751 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.751 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.751 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.751 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.751 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.751 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.751 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.751 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.752 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.752 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.752 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.752 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.752 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.752 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.752 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.752 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.752 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.752 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.752 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "iostat", 2024-09-21T06:48:21.752 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.752 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.752 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.752 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.753 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.753 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.753 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.753 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.753 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.753 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.753 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.753 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.753 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.753 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.753 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.753 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.753 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.753 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.753 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.753 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.754 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.754 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.754 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.754 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.754 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.754 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.754 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.754 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.754 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.754 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.754 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.754 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.754 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.754 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.755 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.755 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.755 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.755 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.755 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.755 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.755 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.755 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.755 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.755 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.755 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.755 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.755 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.755 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.755 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.756 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.756 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.756 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.756 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.756 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.756 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.756 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.756 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.756 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.756 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.756 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.756 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.756 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.756 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.757 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.757 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.757 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.757 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.757 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.757 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.757 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.757 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.757 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.757 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.757 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.757 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.757 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.757 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.758 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.758 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.758 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.758 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.758 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.758 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.758 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.758 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.758 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.758 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.758 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.758 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.758 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.758 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "k8sevents", 2024-09-21T06:48:21.759 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.759 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.759 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.759 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_event_retention_days": { 2024-09-21T06:48:21.759 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ceph_event_retention_days", 2024-09-21T06:48:21.759 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.759 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.759 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.759 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7", 2024-09-21T06:48:21.759 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.759 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.759 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.759 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T06:48:21.759 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.760 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.760 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.760 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.760 INFO:teuthology.orchestra.run.smithi067.stdout: "config_check_secs": { 2024-09-21T06:48:21.760 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_check_secs", 2024-09-21T06:48:21.760 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.760 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.760 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.760 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-21T06:48:21.760 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "10", 2024-09-21T06:48:21.760 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.760 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.760 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T06:48:21.760 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.761 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.761 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.761 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.761 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.761 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.761 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.761 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.761 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.761 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.761 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.761 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.761 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.761 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.761 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.761 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.762 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.762 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.762 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.762 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.762 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.762 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.762 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.762 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.762 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.762 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.762 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.762 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.762 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.762 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.762 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.763 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.763 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.763 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.763 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.763 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.763 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.763 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.763 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.763 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.763 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.763 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.763 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.763 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.763 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.763 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.764 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.764 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.764 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.764 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.764 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.764 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.764 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.764 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.764 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.764 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.764 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.764 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.764 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.764 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.765 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.765 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.765 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.765 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.765 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.765 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.765 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.765 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.765 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.765 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.765 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.765 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.765 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.765 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.766 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.766 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.766 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.766 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.766 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.766 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.766 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.766 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.766 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.766 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.766 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.766 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.766 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.766 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.767 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.767 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.767 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.767 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "localpool", 2024-09-21T06:48:21.767 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.767 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.767 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.767 INFO:teuthology.orchestra.run.smithi067.stdout: "failure_domain": { 2024-09-21T06:48:21.767 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "failure_domain", 2024-09-21T06:48:21.767 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.767 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.767 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.767 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "host", 2024-09-21T06:48:21.767 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.768 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.768 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.768 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "failure domain for any created local pool", 2024-09-21T06:48:21.768 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T06:48:21.768 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.768 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.768 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.768 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.768 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.768 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.768 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.768 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.769 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.769 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.769 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.769 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.769 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.769 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.769 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.769 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.769 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.769 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.769 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.769 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.769 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.769 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.769 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.770 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.770 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.770 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.770 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.770 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.770 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.770 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.770 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.770 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.770 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.770 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.770 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.770 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.770 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.771 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.771 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.771 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.771 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.771 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.771 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.771 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.771 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.771 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.771 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.771 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.771 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.771 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.771 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.772 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.772 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.772 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.772 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.772 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.772 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.772 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.772 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.772 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.772 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.772 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.772 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.772 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.772 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.772 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.772 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.773 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.773 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.773 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.773 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.773 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.773 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.773 INFO:teuthology.orchestra.run.smithi067.stdout: "min_size": { 2024-09-21T06:48:21.773 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "min_size", 2024-09-21T06:48:21.773 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.773 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.773 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.773 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.773 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.773 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.773 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.773 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default min_size for any created local pool", 2024-09-21T06:48:21.774 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T06:48:21.774 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.774 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.774 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.774 INFO:teuthology.orchestra.run.smithi067.stdout: "num_rep": { 2024-09-21T06:48:21.774 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "num_rep", 2024-09-21T06:48:21.774 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.774 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.774 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.774 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3", 2024-09-21T06:48:21.774 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.774 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.774 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.775 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default replica count for any created local pool", 2024-09-21T06:48:21.775 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.775 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.775 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.775 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.775 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_num": { 2024-09-21T06:48:21.775 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pg_num", 2024-09-21T06:48:21.775 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.775 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.775 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.775 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "128", 2024-09-21T06:48:21.775 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.775 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.775 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.776 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T06:48:21.776 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.776 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.776 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.776 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.776 INFO:teuthology.orchestra.run.smithi067.stdout: "prefix": { 2024-09-21T06:48:21.776 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prefix", 2024-09-21T06:48:21.776 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.776 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.776 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.776 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.776 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.776 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.776 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.777 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "name prefix for any created local pool", 2024-09-21T06:48:21.777 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.777 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.777 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.777 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.777 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.777 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.777 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.777 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.777 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.777 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.777 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.777 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.778 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.778 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.778 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.778 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.778 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.778 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.778 INFO:teuthology.orchestra.run.smithi067.stdout: "subtree": { 2024-09-21T06:48:21.778 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subtree", 2024-09-21T06:48:21.778 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.778 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.778 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.778 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "rack", 2024-09-21T06:48:21.778 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.778 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.779 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.779 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T06:48:21.779 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T06:48:21.779 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.779 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.779 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.779 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.779 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.779 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.779 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mds_autoscaler", 2024-09-21T06:48:21.779 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.779 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.779 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.779 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.779 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.780 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.780 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.780 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.780 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.780 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.780 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.780 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.780 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.780 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.780 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.780 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.780 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.780 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.780 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.780 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.781 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.781 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.781 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.781 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.781 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.781 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.781 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.781 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.781 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.781 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.781 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.781 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.781 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.781 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.782 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.782 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.782 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.782 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.782 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.782 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.782 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.782 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.782 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.782 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.782 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.782 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.782 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.782 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.783 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.783 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.783 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.783 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.783 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.783 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.783 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.783 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.783 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.783 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.783 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.783 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.783 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.783 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.783 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.784 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.784 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.784 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.784 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.784 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.784 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.784 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.784 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.784 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.784 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.784 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.784 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.784 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.784 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.784 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.785 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.785 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.785 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.785 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.785 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.785 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.785 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.785 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.785 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.785 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.785 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.785 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.785 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mirroring", 2024-09-21T06:48:21.786 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.786 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.786 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.786 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.786 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.786 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.786 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.786 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.786 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.786 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.786 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.786 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.786 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.786 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.787 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.787 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.787 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.787 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.787 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.787 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.787 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.787 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.787 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.787 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.787 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.787 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.787 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.787 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.788 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.788 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.788 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.788 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.788 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.788 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.788 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.788 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.788 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.788 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.788 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.788 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.788 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.788 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.789 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.789 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.789 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.789 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.789 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.789 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.789 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.789 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.789 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.789 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.789 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.789 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.789 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.789 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.789 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.789 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.790 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.790 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.790 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.790 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.790 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.790 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.790 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.790 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.790 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.790 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.790 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.790 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.790 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.790 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.791 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.791 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.791 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.791 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.791 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.791 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.791 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.791 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.791 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.791 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.791 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.791 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.791 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.791 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.791 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.791 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.792 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.792 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.792 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "nfs", 2024-09-21T06:48:21.792 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.792 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.792 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.792 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.792 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.792 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.792 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.792 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.792 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.792 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.792 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.792 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.792 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.793 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.793 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.793 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.793 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.793 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.793 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.793 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.793 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.793 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.793 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.793 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.793 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.793 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.793 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.793 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.793 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.793 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.794 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.794 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.794 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.794 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.794 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.794 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.794 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.794 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.794 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.794 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.794 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.794 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.794 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.794 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.795 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.795 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.795 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.795 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.795 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.795 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.795 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.795 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.795 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.795 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.795 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.795 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.795 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.795 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.796 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.796 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.796 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.796 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.796 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.796 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.796 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.796 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.796 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.796 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.796 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.796 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.796 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.797 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.797 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.797 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.797 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.797 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.797 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.797 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.797 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.797 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.797 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.797 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.797 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.797 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.797 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.798 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.798 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.798 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.798 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.798 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.798 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "orchestrator", 2024-09-21T06:48:21.798 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.798 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.798 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.798 INFO:teuthology.orchestra.run.smithi067.stdout: "fail_fs": { 2024-09-21T06:48:21.798 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "fail_fs", 2024-09-21T06:48:21.798 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.798 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.799 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.799 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.799 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.801 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.802 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.802 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T06:48:21.802 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.802 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.802 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.802 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.802 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.802 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.802 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.802 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.802 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.802 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.802 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.802 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.803 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.803 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.803 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.803 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.803 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.803 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.803 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.803 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.803 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.803 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.803 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.803 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.803 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.803 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.804 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.804 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.804 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.804 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.804 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.804 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.804 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.804 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.804 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.804 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.804 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.804 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.804 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.804 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.804 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.804 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.805 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.805 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.805 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.805 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.805 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.805 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.805 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.805 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.805 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.805 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.805 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.805 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.805 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.805 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.805 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.806 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.806 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.806 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.806 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.806 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.806 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.806 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.806 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.806 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.806 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.806 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.806 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.806 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.806 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.807 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.807 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.807 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.807 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator": { 2024-09-21T06:48:21.807 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "orchestrator", 2024-09-21T06:48:21.807 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.807 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.807 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.807 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.807 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.807 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.807 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.807 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm", 2024-09-21T06:48:21.807 INFO:teuthology.orchestra.run.smithi067.stdout: "rook", 2024-09-21T06:48:21.808 INFO:teuthology.orchestra.run.smithi067.stdout: "test_orchestrator" 2024-09-21T06:48:21.808 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.808 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Orchestrator backend", 2024-09-21T06:48:21.808 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.808 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.808 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.808 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.808 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.808 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.808 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.808 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.808 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.808 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.808 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.809 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.809 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.809 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.809 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.809 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.809 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.809 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.809 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.809 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.809 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.809 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "osd_perf_query", 2024-09-21T06:48:21.809 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.809 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.809 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.810 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.810 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.810 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.810 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.810 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.810 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.810 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.810 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.810 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.810 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.810 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.810 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.810 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.810 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.810 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.811 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.811 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.811 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.811 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.811 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.811 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.811 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.811 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.811 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.811 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.811 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.811 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.811 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.814 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.814 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.814 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.814 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.814 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.814 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.814 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.814 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.814 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.814 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.814 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.814 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.814 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.814 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.815 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.815 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.815 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.815 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.815 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.815 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.815 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.815 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.815 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.815 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.815 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.815 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.815 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.815 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.815 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.816 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.816 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "osd_support", 2024-09-21T06:48:21.816 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.816 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.816 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.816 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.816 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.816 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.816 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.816 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.816 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.816 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.816 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.817 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.817 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.817 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.817 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.817 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.817 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.817 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.817 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.817 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.817 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.817 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.817 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.817 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.817 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.818 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.818 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.818 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.818 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.818 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.818 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.818 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.818 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.818 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.818 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.818 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.818 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.818 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.818 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.819 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.819 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.819 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.819 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.819 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.819 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.819 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.819 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.819 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.819 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.819 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.819 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.819 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.819 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.819 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.820 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.820 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.820 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.820 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.820 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.820 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.820 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.820 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.820 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.820 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.820 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.820 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.820 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.820 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.821 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.821 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.821 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.821 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.821 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.821 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.821 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.821 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.821 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.821 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.821 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.821 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.821 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.821 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.821 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.822 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.822 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.822 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.822 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.822 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.822 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.822 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.822 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pg_autoscaler", 2024-09-21T06:48:21.822 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.822 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.822 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.822 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.822 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.822 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.822 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.823 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.823 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.823 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.823 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.823 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.823 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.823 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.823 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.823 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.823 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.823 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.823 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.823 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.823 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.824 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.824 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.824 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.824 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.824 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.824 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.824 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.824 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.824 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.824 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.824 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.824 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.824 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.824 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.825 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.825 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.825 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.825 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.825 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.825 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.825 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.825 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.825 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.825 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.825 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.825 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.825 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.826 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.826 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.826 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.826 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.826 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.826 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.826 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.826 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.826 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.826 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.826 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.826 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.826 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.827 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.827 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.827 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.827 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.827 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.827 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.827 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.827 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.827 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.827 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.827 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.827 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-09-21T06:48:21.827 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-09-21T06:48:21.827 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:21.827 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.828 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.828 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-09-21T06:48:21.828 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.828 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.828 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.828 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.828 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.828 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.828 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.828 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.828 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.828 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.828 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.828 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.829 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.829 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.829 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.829 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.829 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.829 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.829 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.829 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.829 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.829 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.829 INFO:teuthology.orchestra.run.smithi067.stdout: "threshold": { 2024-09-21T06:48:21.829 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "threshold", 2024-09-21T06:48:21.829 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-21T06:48:21.830 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.830 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.830 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3.0", 2024-09-21T06:48:21.830 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1.0", 2024-09-21T06:48:21.830 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.830 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.830 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "scaling threshold", 2024-09-21T06:48:21.830 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T06:48:21.830 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.830 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.830 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.830 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.830 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.830 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.831 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "progress", 2024-09-21T06:48:21.831 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.831 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.831 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.831 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_pg_recovery_event": { 2024-09-21T06:48:21.831 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_pg_recovery_event", 2024-09-21T06:48:21.831 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.831 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.831 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.831 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.831 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.831 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.831 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.832 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T06:48:21.832 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.832 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.832 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.832 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.832 INFO:teuthology.orchestra.run.smithi067.stdout: "enabled": { 2024-09-21T06:48:21.832 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enabled", 2024-09-21T06:48:21.832 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.832 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.832 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.832 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.832 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.832 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.832 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.833 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.833 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.833 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.833 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.833 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.833 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.833 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.833 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.833 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.833 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.833 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.833 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.833 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.833 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.834 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.834 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.834 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.834 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.834 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.834 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.834 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.834 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.834 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.834 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.834 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.834 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.834 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.834 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.835 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.835 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.835 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.835 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.835 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.835 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.835 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.835 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.835 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.835 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.835 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.835 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.835 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.835 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.836 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.836 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.836 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.836 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.836 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.836 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.836 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.836 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.836 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.836 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.836 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.836 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.836 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.836 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.837 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.837 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.837 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.837 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.837 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.837 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.837 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.837 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.837 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.837 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.837 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.837 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.837 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.837 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.837 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.838 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.838 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.838 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.838 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.838 INFO:teuthology.orchestra.run.smithi067.stdout: "max_completed_events": { 2024-09-21T06:48:21.838 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_completed_events", 2024-09-21T06:48:21.838 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.838 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.838 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.838 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "50", 2024-09-21T06:48:21.838 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.839 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.839 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.839 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "number of past completed events to remember", 2024-09-21T06:48:21.839 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.839 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.839 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.839 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.839 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-09-21T06:48:21.839 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-09-21T06:48:21.839 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:21.839 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.839 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.839 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-09-21T06:48:21.840 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.840 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.840 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.840 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long the module is going to sleep", 2024-09-21T06:48:21.840 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.840 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.840 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.840 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.840 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.840 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.840 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.840 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.840 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.841 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.841 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.841 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.841 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.841 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.841 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.841 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.841 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.841 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.841 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.841 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.841 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.841 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prometheus", 2024-09-21T06:48:21.842 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.842 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.842 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.842 INFO:teuthology.orchestra.run.smithi067.stdout: "cache": { 2024-09-21T06:48:21.842 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cache", 2024-09-21T06:48:21.842 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.842 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.842 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.842 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.842 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.842 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.842 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.842 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.842 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.842 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.843 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.843 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.843 INFO:teuthology.orchestra.run.smithi067.stdout: "exclude_perf_counters": { 2024-09-21T06:48:21.843 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "exclude_perf_counters", 2024-09-21T06:48:21.843 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.843 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.843 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.843 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.843 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.843 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.843 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.843 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T06:48:21.843 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T06:48:21.844 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.844 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.844 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.844 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.844 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.844 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.844 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.844 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.844 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.844 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.844 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.844 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.844 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.844 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.844 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.845 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.845 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.845 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.845 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.845 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.845 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.845 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.845 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.845 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.845 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.845 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.845 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.845 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.845 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.845 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.846 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.846 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.846 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.846 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.846 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.846 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.846 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.846 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.846 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.846 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.846 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.846 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.846 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.846 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.846 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.847 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.847 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.847 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.847 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.847 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.847 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.847 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.847 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.847 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.847 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.847 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.847 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.847 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.847 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.848 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.848 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.848 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.848 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.848 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.848 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.848 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.848 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.848 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.848 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.848 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.848 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.848 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.848 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.849 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_stats_pools": { 2024-09-21T06:48:21.849 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_stats_pools", 2024-09-21T06:48:21.849 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.849 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.849 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.849 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.849 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.849 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.849 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.849 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.849 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.849 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.849 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.849 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.849 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T06:48:21.850 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T06:48:21.850 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.850 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.850 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.850 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "300", 2024-09-21T06:48:21.850 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.850 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.850 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.850 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.850 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.850 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.850 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.850 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.850 INFO:teuthology.orchestra.run.smithi067.stdout: "scrape_interval": { 2024-09-21T06:48:21.851 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "scrape_interval", 2024-09-21T06:48:21.851 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-21T06:48:21.851 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.851 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.851 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15.0", 2024-09-21T06:48:21.851 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.851 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.851 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.851 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.851 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.851 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.851 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.851 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.851 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-09-21T06:48:21.852 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-09-21T06:48:21.852 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.852 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.852 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.852 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "::", 2024-09-21T06:48:21.852 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.852 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.852 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.852 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T06:48:21.852 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.852 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.852 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.852 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.853 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-09-21T06:48:21.853 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-09-21T06:48:21.853 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.853 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.853 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.853 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "9283", 2024-09-21T06:48:21.853 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.853 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.853 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.853 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T06:48:21.853 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.853 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.853 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.853 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.854 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.854 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.854 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.854 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.854 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.854 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.854 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.854 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.854 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.854 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.854 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.854 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.854 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.855 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.855 INFO:teuthology.orchestra.run.smithi067.stdout: "stale_cache_strategy": { 2024-09-21T06:48:21.855 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "stale_cache_strategy", 2024-09-21T06:48:21.855 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.855 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.855 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.855 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "log", 2024-09-21T06:48:21.855 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.855 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.855 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.855 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.855 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.855 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.855 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.856 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.856 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_behaviour": { 2024-09-21T06:48:21.856 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_behaviour", 2024-09-21T06:48:21.856 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.856 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.856 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.856 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "default", 2024-09-21T06:48:21.856 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.856 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.856 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.856 INFO:teuthology.orchestra.run.smithi067.stdout: "default", 2024-09-21T06:48:21.856 INFO:teuthology.orchestra.run.smithi067.stdout: "error" 2024-09-21T06:48:21.856 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.856 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.856 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.857 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.857 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.857 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.857 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_error_status_code": { 2024-09-21T06:48:21.857 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_error_status_code", 2024-09-21T06:48:21.857 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.857 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.857 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.857 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-09-21T06:48:21.857 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "400", 2024-09-21T06:48:21.857 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "599", 2024-09-21T06:48:21.857 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.857 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.857 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.858 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.858 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.858 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.858 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.858 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.858 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.858 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_support", 2024-09-21T06:48:21.858 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.858 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.858 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.858 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.858 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.858 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.858 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.858 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.859 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.859 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.859 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.859 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.859 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.859 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.859 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.859 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.859 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.859 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.859 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.859 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.859 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.859 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.860 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.860 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.860 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.860 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.860 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.860 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.860 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.860 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.860 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.860 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.860 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.860 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.860 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.860 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.860 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.861 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.861 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.861 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.861 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.861 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.861 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.861 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.861 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.861 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.861 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.861 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.861 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.861 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.861 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.861 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.862 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.862 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.862 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.862 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.862 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.862 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.862 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.862 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.862 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.862 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.862 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.862 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.862 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.863 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.863 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.863 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.863 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.863 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.863 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.863 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.863 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.863 INFO:teuthology.orchestra.run.smithi067.stdout: "max_concurrent_snap_create": { 2024-09-21T06:48:21.863 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_concurrent_snap_create", 2024-09-21T06:48:21.863 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.863 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.863 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.863 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-21T06:48:21.863 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.864 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.864 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.864 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.864 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.864 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.864 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.864 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.864 INFO:teuthology.orchestra.run.smithi067.stdout: "mirror_snapshot_schedule": { 2024-09-21T06:48:21.864 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T06:48:21.864 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.864 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.864 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.864 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.864 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.865 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.865 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.865 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.865 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.865 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.865 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.865 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.865 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.865 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.865 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.865 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.865 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.865 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.865 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.866 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.866 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.866 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.866 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.866 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.866 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.866 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.866 INFO:teuthology.orchestra.run.smithi067.stdout: "trash_purge_schedule": { 2024-09-21T06:48:21.866 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "trash_purge_schedule", 2024-09-21T06:48:21.866 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.866 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.866 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.866 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.866 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.867 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.867 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.867 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.867 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.867 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.867 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.867 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.867 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.867 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.867 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.867 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "restful", 2024-09-21T06:48:21.867 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.867 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.867 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.867 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_auth": { 2024-09-21T06:48:21.868 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enable_auth", 2024-09-21T06:48:21.868 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.868 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.868 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.868 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.868 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.868 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.868 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.868 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.868 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.868 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.868 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.868 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.869 INFO:teuthology.orchestra.run.smithi067.stdout: "key_file": { 2024-09-21T06:48:21.869 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "key_file", 2024-09-21T06:48:21.869 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.869 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.869 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.869 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.869 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.869 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.869 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.869 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.869 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.869 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.869 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.869 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.869 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.870 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.870 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.870 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.870 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.870 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.870 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.870 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.870 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.870 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.870 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.870 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.870 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.870 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.870 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.871 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.871 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.871 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.871 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.871 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.871 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.871 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.871 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.871 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.871 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.871 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.871 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.871 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.871 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.871 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.871 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.872 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.872 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.872 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.872 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.872 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.872 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.872 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.872 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.872 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.872 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.872 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.872 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.872 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.872 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.872 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.872 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.873 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.873 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.873 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.873 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.873 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.873 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.873 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.873 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.873 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.873 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.873 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.873 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.873 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.873 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.873 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.873 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.874 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.874 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.874 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.874 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.874 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.874 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.874 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.874 INFO:teuthology.orchestra.run.smithi067.stdout: "max_requests": { 2024-09-21T06:48:21.874 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_requests", 2024-09-21T06:48:21.874 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.874 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.874 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.874 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-09-21T06:48:21.874 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.874 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.875 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.875 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T06:48:21.875 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.875 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.875 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.875 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.875 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-09-21T06:48:21.875 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-09-21T06:48:21.875 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.875 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.875 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.875 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.875 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.875 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.876 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.876 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.876 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.876 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.876 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.876 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.876 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-09-21T06:48:21.876 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-09-21T06:48:21.876 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.876 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.876 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.876 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.876 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.877 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.877 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.877 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.877 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.877 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.877 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.877 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.877 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.877 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.877 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.877 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.877 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.877 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.877 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.878 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.878 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.878 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.878 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.878 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.878 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.878 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.878 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.878 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.878 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.878 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rgw", 2024-09-21T06:48:21.878 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.878 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.879 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.879 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.879 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.879 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.879 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.879 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.879 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.879 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.879 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.879 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.879 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.879 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.879 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.880 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.880 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.880 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.880 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.880 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.880 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.880 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.880 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.880 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.880 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.880 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.880 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.880 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.880 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.881 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.881 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.881 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.881 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.881 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.881 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.881 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.881 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.881 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.881 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.881 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.881 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.881 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.881 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.881 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.882 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.882 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.882 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.882 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.882 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.882 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.882 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.882 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.882 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.882 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.882 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.882 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.882 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.882 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.883 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.883 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.883 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.883 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.883 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.883 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.883 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.883 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.883 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.883 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.883 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.883 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.883 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.883 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.883 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.884 INFO:teuthology.orchestra.run.smithi067.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T06:48:21.884 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T06:48:21.884 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.884 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.884 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.884 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-09-21T06:48:21.884 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.884 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.884 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.884 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T06:48:21.884 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.884 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.884 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.885 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.885 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.885 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.885 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.885 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.885 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.885 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.885 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.885 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.885 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.885 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.885 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.885 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.885 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.886 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.886 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.886 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.886 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.886 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rook", 2024-09-21T06:48:21.886 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.886 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.886 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.886 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.886 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.886 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.886 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.886 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.887 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.887 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.887 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.887 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.887 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.887 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.887 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.887 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.887 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.887 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.887 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.887 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.887 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.887 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.887 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.888 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.888 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.888 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.888 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.888 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.888 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.888 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.888 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.888 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.888 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.888 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.888 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.888 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.888 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.889 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.889 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.889 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.889 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.889 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.889 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.889 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.889 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.889 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.889 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.889 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.889 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.889 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.889 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.890 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.890 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.890 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.890 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.890 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.890 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.890 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.890 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.890 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.890 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.890 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.890 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.890 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.890 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.890 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.891 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.891 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.891 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.891 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.891 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.891 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.891 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.891 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.891 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.891 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.891 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.891 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.891 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.891 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.891 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.892 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.892 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.892 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.892 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.892 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.892 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.892 INFO:teuthology.orchestra.run.smithi067.stdout: "storage_class": { 2024-09-21T06:48:21.892 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "storage_class", 2024-09-21T06:48:21.892 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.892 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.892 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.892 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "local", 2024-09-21T06:48:21.892 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.893 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.893 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.893 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T06:48:21.893 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.893 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.893 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.893 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.893 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.893 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.893 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.893 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "selftest", 2024-09-21T06:48:21.893 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.893 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.893 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.893 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.893 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.894 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.894 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.894 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.894 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.894 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.894 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.894 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.894 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.894 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.894 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.894 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.894 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.894 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.895 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.895 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.895 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.895 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.895 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.895 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.895 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.895 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.895 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.895 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.895 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.895 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.895 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.895 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.896 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.896 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.896 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.896 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.896 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.896 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.896 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.896 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.896 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.896 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.896 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.896 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.896 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.897 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.897 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.897 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.897 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.897 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.897 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.897 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.897 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.897 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.897 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.897 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.897 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.897 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.897 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.897 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.898 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.898 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.898 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.898 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.898 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.898 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.898 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.898 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.898 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.898 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.898 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.898 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.898 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.898 INFO:teuthology.orchestra.run.smithi067.stdout: "roption1": { 2024-09-21T06:48:21.898 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "roption1", 2024-09-21T06:48:21.898 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.899 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.899 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.899 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.899 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.899 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.899 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.899 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.899 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.899 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.899 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.899 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.899 INFO:teuthology.orchestra.run.smithi067.stdout: "roption2": { 2024-09-21T06:48:21.899 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "roption2", 2024-09-21T06:48:21.899 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.900 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.900 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.900 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "xyz", 2024-09-21T06:48:21.900 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.900 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.900 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.900 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.900 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.900 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.900 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.900 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.900 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption1": { 2024-09-21T06:48:21.900 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption1", 2024-09-21T06:48:21.900 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.900 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.901 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.901 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.901 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.901 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.901 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.901 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.901 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.901 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.901 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.901 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.901 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption2": { 2024-09-21T06:48:21.901 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption2", 2024-09-21T06:48:21.901 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.901 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.901 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.902 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.902 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.902 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.902 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.902 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.902 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.902 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.902 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.902 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.902 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption3": { 2024-09-21T06:48:21.902 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption3", 2024-09-21T06:48:21.902 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-21T06:48:21.902 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.902 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.902 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.903 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.903 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.903 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.903 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.903 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.903 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.903 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.903 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.903 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption4": { 2024-09-21T06:48:21.903 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption4", 2024-09-21T06:48:21.903 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.903 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.903 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.903 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.903 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.903 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.904 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.904 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.904 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.904 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.904 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.904 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.904 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption5": { 2024-09-21T06:48:21.904 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption5", 2024-09-21T06:48:21.904 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.904 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.904 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.904 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.904 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.904 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.904 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.905 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.905 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.905 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.905 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.905 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.905 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption6": { 2024-09-21T06:48:21.905 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption6", 2024-09-21T06:48:21.905 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.905 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.905 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.905 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.905 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.905 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.905 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.905 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.906 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.906 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.906 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.906 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.906 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption7": { 2024-09-21T06:48:21.906 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption7", 2024-09-21T06:48:21.906 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.906 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.906 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.906 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.906 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-09-21T06:48:21.906 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "42", 2024-09-21T06:48:21.906 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.906 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.907 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.907 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.907 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.907 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.907 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.907 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.907 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.907 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.907 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.907 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.907 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.907 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.907 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.907 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.907 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.907 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.908 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.908 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.908 INFO:teuthology.orchestra.run.smithi067.stdout: "testkey": { 2024-09-21T06:48:21.908 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testkey", 2024-09-21T06:48:21.908 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.908 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.908 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.908 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.908 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.908 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.908 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.908 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.908 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.908 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.909 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.909 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.909 INFO:teuthology.orchestra.run.smithi067.stdout: "testlkey": { 2024-09-21T06:48:21.909 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testlkey", 2024-09-21T06:48:21.909 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.909 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.909 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.909 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.909 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.909 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.909 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.909 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.909 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.910 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.910 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.910 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.910 INFO:teuthology.orchestra.run.smithi067.stdout: "testnewline": { 2024-09-21T06:48:21.910 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testnewline", 2024-09-21T06:48:21.910 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.910 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.910 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.910 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.910 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.910 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.910 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.910 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.910 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.911 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.911 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.911 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.911 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.911 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.911 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.911 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smb", 2024-09-21T06:48:21.911 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.911 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.911 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.911 INFO:teuthology.orchestra.run.smithi067.stdout: "internal_store_backend": { 2024-09-21T06:48:21.911 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "internal_store_backend", 2024-09-21T06:48:21.911 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.912 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.912 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.912 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.912 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.912 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.912 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.912 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T06:48:21.912 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.912 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.912 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.912 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.912 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.912 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.912 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.912 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.912 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.913 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.913 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.913 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.913 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.913 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.913 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.913 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.913 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.913 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.913 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.913 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.913 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.913 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.913 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.913 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.913 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.914 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.914 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.914 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.914 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.914 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.914 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.914 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.914 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.914 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.914 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.914 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.914 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.914 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.915 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.915 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.915 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.915 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.915 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.915 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.915 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.915 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.915 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.915 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.915 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.915 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.915 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.915 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.915 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.916 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.916 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.916 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.916 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.916 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.916 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.916 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.916 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.916 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.916 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.916 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.916 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.916 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.917 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.917 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.917 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.917 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.917 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.917 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.917 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.917 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.917 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.917 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.917 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.917 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.917 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.917 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.918 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.918 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.918 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.918 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.918 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.918 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.918 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.918 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.918 INFO:teuthology.orchestra.run.smithi067.stdout: "update_orchestration": { 2024-09-21T06:48:21.918 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "update_orchestration", 2024-09-21T06:48:21.918 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.918 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.918 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.918 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.919 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.919 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.919 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.919 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T06:48:21.919 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.919 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.919 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.919 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.919 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.919 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.919 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.919 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snap_schedule", 2024-09-21T06:48:21.919 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.920 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.920 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.920 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_m_granularity": { 2024-09-21T06:48:21.920 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_m_granularity", 2024-09-21T06:48:21.920 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.920 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.920 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.920 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.920 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.920 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.920 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.920 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T06:48:21.920 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.920 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.920 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.921 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.921 INFO:teuthology.orchestra.run.smithi067.stdout: "dump_on_update": { 2024-09-21T06:48:21.921 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "dump_on_update", 2024-09-21T06:48:21.921 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.921 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.921 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.921 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.921 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.921 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.921 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.921 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "dump database to debug log on update", 2024-09-21T06:48:21.921 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.921 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.922 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.922 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.922 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.922 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.922 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.922 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.922 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.922 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.922 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.922 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.922 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.922 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.922 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.922 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.923 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.923 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.923 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.923 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.923 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.923 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.923 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.923 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.923 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.923 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.923 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.923 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.923 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.923 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.923 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.924 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.924 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.924 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.924 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.924 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.924 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.924 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.924 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.924 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.924 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.924 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.924 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.924 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.924 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.924 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.925 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.925 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.925 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.925 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.925 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.925 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.925 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.925 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.925 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.925 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.925 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.925 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.925 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.925 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.926 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.926 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.926 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.926 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.926 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.926 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.926 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.926 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.926 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.926 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.926 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.926 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.926 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.926 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.927 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.927 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.927 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.927 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.927 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.927 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.927 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.927 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.927 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.927 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.927 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.927 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.927 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.927 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.928 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.928 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.928 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.928 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "stats", 2024-09-21T06:48:21.928 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.928 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.928 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.928 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.928 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.928 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.928 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.928 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.928 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.928 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.928 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.929 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.929 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.929 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.929 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.929 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.929 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.929 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.929 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.929 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.929 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.929 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.929 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.929 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.929 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.930 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.930 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.930 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.930 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.930 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.930 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.930 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.930 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.930 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.930 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.930 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.930 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.930 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.930 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.931 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.931 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.931 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.931 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.931 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.931 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.931 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.931 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.931 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.931 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.931 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.931 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.931 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.931 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.931 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.931 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.932 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.932 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.932 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.932 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.932 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.932 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.932 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.932 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.932 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.932 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.932 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.932 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.932 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.932 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.932 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.933 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.933 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.933 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.933 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.933 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.933 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.933 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.933 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.933 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.933 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.933 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.933 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.933 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.933 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.933 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.933 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.934 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.934 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.934 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.934 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.934 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "status", 2024-09-21T06:48:21.934 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.934 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.934 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.934 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.934 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.934 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.934 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.934 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.934 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.935 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.935 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.935 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.935 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.935 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.935 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.935 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.935 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.935 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.935 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.935 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.935 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.935 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.935 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.936 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.936 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.936 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.936 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.936 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.936 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.936 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.936 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.936 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.936 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.936 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.936 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.936 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.936 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.937 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.937 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.937 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.937 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.937 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.937 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.937 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.937 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.937 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.937 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.937 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.937 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.937 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.937 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.938 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.938 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.938 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.938 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.938 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.938 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.938 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.938 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.938 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.938 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.938 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.938 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.938 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.938 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.938 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.939 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.939 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.939 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.939 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.939 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.939 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.939 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.939 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.939 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.939 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.939 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.939 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.939 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.939 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.939 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.940 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.940 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.940 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.940 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.940 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.940 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.940 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.940 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.940 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.940 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "telegraf", 2024-09-21T06:48:21.940 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.940 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.940 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.940 INFO:teuthology.orchestra.run.smithi067.stdout: "address": { 2024-09-21T06:48:21.941 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "address", 2024-09-21T06:48:21.941 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.941 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.941 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.941 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T06:48:21.941 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.941 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.941 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.941 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.941 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.941 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.941 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.941 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.941 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-09-21T06:48:21.942 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-09-21T06:48:21.942 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:21.942 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.942 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.942 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15", 2024-09-21T06:48:21.942 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.942 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.942 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.942 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.942 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.942 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.942 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.942 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.942 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.942 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.943 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.943 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.943 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.943 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.943 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.943 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.943 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.943 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.943 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.943 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.943 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.943 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.943 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.943 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.943 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.944 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.944 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.944 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.944 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.944 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.944 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.944 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.944 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.944 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.944 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.944 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.944 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.944 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.944 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.945 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.945 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.945 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.945 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.945 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.945 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.945 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.945 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.945 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.945 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.945 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.945 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.945 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.946 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.946 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.946 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.946 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.946 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.946 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.946 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.946 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.946 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.946 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.946 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.946 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.946 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.946 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.947 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.947 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.947 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.947 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.947 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.947 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.947 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.947 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.947 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.947 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.947 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.947 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.947 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.947 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.948 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.948 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.948 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.948 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.948 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.948 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.948 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.948 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.948 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.948 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.948 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.948 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.948 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.948 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.949 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.949 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "telemetry", 2024-09-21T06:48:21.949 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.949 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.949 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.949 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_basic": { 2024-09-21T06:48:21.949 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_basic", 2024-09-21T06:48:21.949 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.949 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.949 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.949 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.949 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.949 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.949 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.949 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T06:48:21.950 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.950 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.950 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.950 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.950 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_crash": { 2024-09-21T06:48:21.950 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_crash", 2024-09-21T06:48:21.950 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.950 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.950 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.950 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.950 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.950 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.950 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.950 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T06:48:21.950 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.951 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.951 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.951 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.951 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_device": { 2024-09-21T06:48:21.951 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_device", 2024-09-21T06:48:21.951 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.951 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.951 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.951 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.951 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.951 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.951 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.951 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T06:48:21.951 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.951 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_ident": { 2024-09-21T06:48:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_ident", 2024-09-21T06:48:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T06:48:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.953 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_perf": { 2024-09-21T06:48:21.953 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_perf", 2024-09-21T06:48:21.953 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.953 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.953 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.953 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.953 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.953 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.953 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.953 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T06:48:21.953 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.953 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.953 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.953 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.954 INFO:teuthology.orchestra.run.smithi067.stdout: "contact": { 2024-09-21T06:48:21.954 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "contact", 2024-09-21T06:48:21.954 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.954 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.954 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.954 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.954 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.954 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.954 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.954 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.954 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.954 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.954 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.954 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.955 INFO:teuthology.orchestra.run.smithi067.stdout: "description": { 2024-09-21T06:48:21.955 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "description", 2024-09-21T06:48:21.955 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.955 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.955 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.955 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.955 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.955 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.955 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.955 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.955 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.955 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.955 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.955 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.956 INFO:teuthology.orchestra.run.smithi067.stdout: "device_url": { 2024-09-21T06:48:21.956 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_url", 2024-09-21T06:48:21.956 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.956 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.956 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.956 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T06:48:21.956 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.956 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.956 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.956 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.956 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.956 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.956 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.956 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.957 INFO:teuthology.orchestra.run.smithi067.stdout: "enabled": { 2024-09-21T06:48:21.957 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enabled", 2024-09-21T06:48:21.957 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.957 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.957 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.957 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.957 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.957 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.957 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.957 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.957 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.957 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.957 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.957 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.957 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-09-21T06:48:21.958 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-09-21T06:48:21.958 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.958 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.958 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.958 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "24", 2024-09-21T06:48:21.958 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "8", 2024-09-21T06:48:21.958 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.958 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.958 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.958 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.958 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.958 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.958 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.958 INFO:teuthology.orchestra.run.smithi067.stdout: "last_opt_revision": { 2024-09-21T06:48:21.959 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "last_opt_revision", 2024-09-21T06:48:21.959 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.959 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.959 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.959 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1", 2024-09-21T06:48:21.959 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.959 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.959 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.959 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.959 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.959 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.959 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.959 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.959 INFO:teuthology.orchestra.run.smithi067.stdout: "leaderboard": { 2024-09-21T06:48:21.959 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "leaderboard", 2024-09-21T06:48:21.960 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.960 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.960 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.960 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.960 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.960 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.960 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.960 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.960 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.960 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.960 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.960 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.960 INFO:teuthology.orchestra.run.smithi067.stdout: "leaderboard_description": { 2024-09-21T06:48:21.960 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "leaderboard_description", 2024-09-21T06:48:21.960 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.961 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.961 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.961 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.961 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.961 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.961 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.961 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.961 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.961 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.961 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.961 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.961 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.961 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.961 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.961 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.961 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.962 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.962 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.962 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.962 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.962 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.962 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.962 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.962 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.962 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.962 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.962 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.962 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.962 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.962 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.963 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.963 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.963 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.963 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.963 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.963 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.963 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.963 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.963 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.963 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.963 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.963 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.963 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.963 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.964 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.964 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.964 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.964 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.964 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.964 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.964 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.964 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.964 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.964 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.964 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.964 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.964 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.964 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.964 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.965 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.965 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.965 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.965 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.965 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.965 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.965 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.965 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.965 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.965 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.965 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.965 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.965 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.965 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.966 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.966 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.966 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.966 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.966 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.966 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.966 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.966 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.966 INFO:teuthology.orchestra.run.smithi067.stdout: "organization": { 2024-09-21T06:48:21.966 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "organization", 2024-09-21T06:48:21.966 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.966 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.966 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.966 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.967 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.967 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.967 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.967 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.967 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.967 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.967 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.967 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.967 INFO:teuthology.orchestra.run.smithi067.stdout: "proxy": { 2024-09-21T06:48:21.967 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "proxy", 2024-09-21T06:48:21.967 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.967 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.967 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.967 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.967 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.968 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.968 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.968 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.968 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.968 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.968 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.968 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.968 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.968 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.968 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.968 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.968 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.968 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.968 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.968 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.968 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: "url": { 2024-09-21T06:48:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "url", 2024-09-21T06:48:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T06:48:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.970 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.970 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.970 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.970 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.970 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.970 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.970 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.970 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "test_orchestrator", 2024-09-21T06:48:21.970 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.970 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.970 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.970 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.970 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.970 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.971 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.971 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.971 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.971 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.971 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.971 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.971 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.971 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.971 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.971 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.971 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.971 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.971 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.971 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.971 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.972 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.972 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.972 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.972 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.972 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.972 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.972 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.972 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.972 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.972 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.972 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.972 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.972 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.972 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.972 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.973 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.973 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.973 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.973 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.973 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.973 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.973 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.973 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.973 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.973 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.973 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.973 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.973 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.973 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.973 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.973 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.974 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.974 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.974 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.974 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.974 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.974 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.974 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.974 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.974 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.974 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.974 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.974 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.974 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.974 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.974 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.975 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.975 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.975 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.975 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.975 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.975 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.975 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.975 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.975 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.975 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.975 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.975 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.975 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.976 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.976 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.976 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.976 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.976 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.976 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.976 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.976 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.976 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.976 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.976 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "volumes", 2024-09-21T06:48:21.976 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.976 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.976 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.976 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.977 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.977 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.977 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.977 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.977 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.977 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.977 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.977 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.977 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.977 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.977 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.977 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.977 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.977 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.978 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.978 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.978 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.978 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.978 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.978 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.978 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.978 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.978 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.978 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.978 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.978 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.978 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.978 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.978 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.979 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.979 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.979 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.979 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.979 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.979 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.979 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.979 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.979 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.979 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.979 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.979 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.979 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.979 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.979 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.980 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.980 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.980 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.980 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.980 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.980 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.980 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.980 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.980 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.980 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.980 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.980 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.980 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.980 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.981 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.981 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.981 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.981 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.981 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.981 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.981 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.981 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.981 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.981 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.981 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.981 INFO:teuthology.orchestra.run.smithi067.stdout: "max_concurrent_clones": { 2024-09-21T06:48:21.981 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_concurrent_clones", 2024-09-21T06:48:21.981 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.981 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.982 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.982 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "4", 2024-09-21T06:48:21.982 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.982 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.982 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.982 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T06:48:21.982 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.982 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.982 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.982 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.982 INFO:teuthology.orchestra.run.smithi067.stdout: "periodic_async_work": { 2024-09-21T06:48:21.982 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "periodic_async_work", 2024-09-21T06:48:21.982 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.982 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.983 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.983 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.983 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.983 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.983 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.983 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Periodically check for async work", 2024-09-21T06:48:21.983 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.983 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.983 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.983 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.983 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot_clone_delay": { 2024-09-21T06:48:21.983 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snapshot_clone_delay", 2024-09-21T06:48:21.983 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.983 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.983 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.984 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.984 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.984 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.984 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.984 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T06:48:21.984 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.984 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.984 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.984 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.984 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot_clone_no_wait": { 2024-09-21T06:48:21.984 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T06:48:21.984 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.984 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.984 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.985 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:21.985 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.985 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.985 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.985 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T06:48:21.985 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.985 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.985 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.985 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.985 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.985 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.985 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.985 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.985 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.985 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.986 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.986 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.986 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.986 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.986 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.986 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.986 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.986 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.986 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.986 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.986 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.986 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix", 2024-09-21T06:48:21.986 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:21.986 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:21.987 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:21.987 INFO:teuthology.orchestra.run.smithi067.stdout: "discovery_interval": { 2024-09-21T06:48:21.987 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "discovery_interval", 2024-09-21T06:48:21.987 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-09-21T06:48:21.987 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.987 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.987 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "100", 2024-09-21T06:48:21.987 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.987 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.987 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.987 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.987 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.987 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.987 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.988 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.988 INFO:teuthology.orchestra.run.smithi067.stdout: "identifier": { 2024-09-21T06:48:21.988 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "identifier", 2024-09-21T06:48:21.988 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.988 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.988 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.988 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.988 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.988 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.988 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.988 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.988 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.988 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.988 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.989 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.989 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-09-21T06:48:21.989 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-09-21T06:48:21.989 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:21.989 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.989 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.989 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-09-21T06:48:21.989 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.989 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.989 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.989 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.989 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.989 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.990 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.990 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.990 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:21.990 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:21.990 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.990 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.990 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.990 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.990 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.990 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.990 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.990 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.990 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.990 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.991 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.991 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.991 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.991 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.991 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.991 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.991 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.991 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.991 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.991 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:21.991 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:21.991 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.991 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.992 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.992 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.992 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.992 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.992 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.992 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.992 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.992 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.992 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.992 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.992 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:21.992 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:21.992 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.993 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.993 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.993 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:21.993 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.993 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.993 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:21.993 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:21.993 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:21.993 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:21.993 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:21.993 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:21.993 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:21.993 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:21.993 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.993 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.994 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.994 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.994 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.994 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:21.994 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:21.994 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:21.994 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.994 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.994 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:21.994 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.994 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.994 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.994 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.995 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.995 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.995 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.995 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.995 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:21.995 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:21.995 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.995 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:21.995 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:21.995 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:21.995 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.995 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.995 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.995 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.996 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.996 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.996 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.996 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.996 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_host": { 2024-09-21T06:48:21.996 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_host", 2024-09-21T06:48:21.996 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.996 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.996 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.996 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:21.996 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.996 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.996 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.996 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.997 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.997 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.997 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.997 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.997 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_port": { 2024-09-21T06:48:21.997 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_port", 2024-09-21T06:48:21.997 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:21.997 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.997 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.997 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10051", 2024-09-21T06:48:21.997 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.997 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.997 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.997 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.997 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.998 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.998 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.998 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.998 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_sender": { 2024-09-21T06:48:21.998 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_sender", 2024-09-21T06:48:21.998 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:21.998 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:21.998 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:21.998 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T06:48:21.998 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:21.998 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:21.998 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:21.998 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:21.999 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:21.999 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:21.999 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:21.999 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.999 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.999 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:21.999 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-21T06:48:21.999 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:21.999 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:21.999 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 24149, 2024-09-21T06:48:21.999 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "y", 2024-09-21T06:48:21.999 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_features": 4540719139924082687, 2024-09-21T06:48:21.999 INFO:teuthology.orchestra.run.smithi067.stdout: "available_modules": [ 2024-09-21T06:48:21.999 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.000 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "alerts", 2024-09-21T06:48:22.000 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.000 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.000 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.000 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-09-21T06:48:22.000 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-09-21T06:48:22.000 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.000 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.000 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.000 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-09-21T06:48:22.000 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.000 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.000 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.000 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T06:48:22.000 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.001 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.001 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.001 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.001 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.001 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.001 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.001 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.001 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.001 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.001 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.001 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.001 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.001 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.001 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.002 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.002 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.002 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.002 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.002 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.002 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.002 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.002 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.002 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.002 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.002 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.002 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.002 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.002 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.003 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.003 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.003 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.003 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.003 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.003 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.003 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.003 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.003 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.003 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.003 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.003 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.003 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.003 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.003 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.004 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.004 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.004 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.004 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.004 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.004 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.004 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.004 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.004 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.004 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.004 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.004 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.004 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.004 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.005 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.005 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.005 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.005 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.005 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.005 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.005 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.005 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.005 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.005 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.005 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.005 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.005 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.006 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.006 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.006 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.006 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_destination": { 2024-09-21T06:48:22.006 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_destination", 2024-09-21T06:48:22.006 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.006 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.006 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.006 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.006 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.006 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.006 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.006 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T06:48:22.006 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.007 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.007 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.007 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.007 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_from_name": { 2024-09-21T06:48:22.007 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_from_name", 2024-09-21T06:48:22.007 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.007 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.007 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.007 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "Ceph", 2024-09-21T06:48:22.007 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.007 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.007 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.007 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Email From: name", 2024-09-21T06:48:22.007 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.007 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.008 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.008 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.008 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_host": { 2024-09-21T06:48:22.008 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_host", 2024-09-21T06:48:22.008 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.008 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.008 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.008 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.008 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.008 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.008 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.008 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP server", 2024-09-21T06:48:22.008 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.008 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.009 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.009 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.009 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_password": { 2024-09-21T06:48:22.009 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_password", 2024-09-21T06:48:22.009 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.009 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.009 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.009 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.009 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.009 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.009 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.009 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Password to authenticate with", 2024-09-21T06:48:22.009 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.009 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.010 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.010 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.010 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_port": { 2024-09-21T06:48:22.010 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_port", 2024-09-21T06:48:22.010 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.010 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.010 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.010 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "465", 2024-09-21T06:48:22.010 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.010 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.010 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.010 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP port", 2024-09-21T06:48:22.010 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.010 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.010 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.011 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.011 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_sender": { 2024-09-21T06:48:22.011 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_sender", 2024-09-21T06:48:22.011 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.011 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.011 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.011 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.011 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.011 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.011 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.011 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP envelope sender", 2024-09-21T06:48:22.011 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.011 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.011 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.012 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.012 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_ssl": { 2024-09-21T06:48:22.012 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_ssl", 2024-09-21T06:48:22.012 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.012 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.012 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.012 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.012 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.012 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.012 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.012 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T06:48:22.012 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.012 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.012 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.013 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.013 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_user": { 2024-09-21T06:48:22.013 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_user", 2024-09-21T06:48:22.013 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.013 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.013 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.013 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.013 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.013 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.013 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.013 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "User to authenticate as", 2024-09-21T06:48:22.013 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.013 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.013 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.013 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.013 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.014 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.014 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.014 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.014 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.014 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.014 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.014 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.014 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.014 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.014 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.014 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.014 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.014 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.014 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.015 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.015 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.015 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "balancer", 2024-09-21T06:48:22.015 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.015 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.015 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.015 INFO:teuthology.orchestra.run.smithi067.stdout: "active": { 2024-09-21T06:48:22.015 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "active", 2024-09-21T06:48:22.015 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.015 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.015 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.015 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.015 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.015 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.015 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.016 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T06:48:22.016 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.016 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.016 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.016 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.016 INFO:teuthology.orchestra.run.smithi067.stdout: "begin_time": { 2024-09-21T06:48:22.016 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "begin_time", 2024-09-21T06:48:22.016 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.016 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.016 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.016 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0000", 2024-09-21T06:48:22.016 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.016 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.016 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.017 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T06:48:22.017 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T06:48:22.017 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.017 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.017 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.017 INFO:teuthology.orchestra.run.smithi067.stdout: "begin_weekday": { 2024-09-21T06:48:22.017 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "begin_weekday", 2024-09-21T06:48:22.017 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-09-21T06:48:22.017 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.017 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.017 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.017 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0", 2024-09-21T06:48:22.017 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "6", 2024-09-21T06:48:22.017 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.018 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T06:48:22.018 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T06:48:22.018 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.018 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.018 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.018 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_max_iterations": { 2024-09-21T06:48:22.018 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_max_iterations", 2024-09-21T06:48:22.018 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-09-21T06:48:22.018 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.018 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.018 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "25", 2024-09-21T06:48:22.018 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-09-21T06:48:22.018 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "250", 2024-09-21T06:48:22.018 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.018 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T06:48:22.019 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.019 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.019 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.019 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.019 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_metrics": { 2024-09-21T06:48:22.019 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_metrics", 2024-09-21T06:48:22.019 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.019 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.019 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.019 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T06:48:22.019 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.019 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.019 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.019 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T06:48:22.020 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T06:48:22.020 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.020 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.020 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.020 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_step": { 2024-09-21T06:48:22.020 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_step", 2024-09-21T06:48:22.020 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-21T06:48:22.020 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.020 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.020 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0.5", 2024-09-21T06:48:22.020 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0.001", 2024-09-21T06:48:22.020 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "0.999", 2024-09-21T06:48:22.020 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.020 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "aggressiveness of optimization", 2024-09-21T06:48:22.021 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T06:48:22.021 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.021 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.021 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.021 INFO:teuthology.orchestra.run.smithi067.stdout: "end_time": { 2024-09-21T06:48:22.021 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "end_time", 2024-09-21T06:48:22.021 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.021 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.021 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.021 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "2359", 2024-09-21T06:48:22.021 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.021 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.021 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.021 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T06:48:22.022 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T06:48:22.022 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.022 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.022 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.022 INFO:teuthology.orchestra.run.smithi067.stdout: "end_weekday": { 2024-09-21T06:48:22.022 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "end_weekday", 2024-09-21T06:48:22.022 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-09-21T06:48:22.022 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.022 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.022 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.022 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0", 2024-09-21T06:48:22.022 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "6", 2024-09-21T06:48:22.022 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.023 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T06:48:22.023 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T06:48:22.023 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.023 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.023 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.023 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.023 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.023 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.023 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.023 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.023 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.023 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.023 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.023 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.023 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.024 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.024 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.024 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.024 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.024 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.024 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.024 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.024 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.024 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.024 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.024 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.024 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.024 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.025 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.025 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.025 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.025 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.025 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.025 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.025 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.025 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.025 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.025 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.025 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.025 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.025 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.025 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.025 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.026 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.026 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.026 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.026 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.026 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.026 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.026 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.026 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.026 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.026 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.026 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.026 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.026 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.026 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.027 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.027 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.027 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.027 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.027 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.027 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.027 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.027 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.027 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.027 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.027 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.027 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.027 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.027 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.028 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.028 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.028 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.028 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.028 INFO:teuthology.orchestra.run.smithi067.stdout: "min_score": { 2024-09-21T06:48:22.028 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "min_score", 2024-09-21T06:48:22.028 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-21T06:48:22.028 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.028 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.028 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.028 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.028 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.028 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.028 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T06:48:22.028 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.029 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.029 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.029 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.029 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": { 2024-09-21T06:48:22.029 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mode", 2024-09-21T06:48:22.029 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.029 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.029 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.029 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "upmap", 2024-09-21T06:48:22.029 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.029 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.029 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.029 INFO:teuthology.orchestra.run.smithi067.stdout: "crush-compat", 2024-09-21T06:48:22.029 INFO:teuthology.orchestra.run.smithi067.stdout: "none", 2024-09-21T06:48:22.030 INFO:teuthology.orchestra.run.smithi067.stdout: "read", 2024-09-21T06:48:22.030 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap", 2024-09-21T06:48:22.030 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap-read" 2024-09-21T06:48:22.030 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.030 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Balancer mode", 2024-09-21T06:48:22.030 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.030 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.030 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.030 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.030 INFO:teuthology.orchestra.run.smithi067.stdout: "pool_ids": { 2024-09-21T06:48:22.030 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pool_ids", 2024-09-21T06:48:22.030 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.030 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.030 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.031 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.031 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.031 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.031 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.031 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T06:48:22.031 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.031 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.031 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.031 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.031 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-09-21T06:48:22.031 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-09-21T06:48:22.031 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.031 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.031 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.032 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-09-21T06:48:22.032 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.032 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.032 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.032 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T06:48:22.032 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.032 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.032 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.032 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.032 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.032 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.032 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.032 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.032 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.032 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.033 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.033 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.033 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.033 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.033 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.033 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.033 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.033 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.033 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap_max_deviation": { 2024-09-21T06:48:22.033 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "upmap_max_deviation", 2024-09-21T06:48:22.033 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.033 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.033 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.033 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-09-21T06:48:22.033 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-09-21T06:48:22.034 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.034 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.034 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T06:48:22.034 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T06:48:22.034 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.034 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.034 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.034 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap_max_optimizations": { 2024-09-21T06:48:22.034 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "upmap_max_optimizations", 2024-09-21T06:48:22.034 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-09-21T06:48:22.034 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.034 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.034 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-21T06:48:22.034 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.035 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.035 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.035 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T06:48:22.035 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.035 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.035 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.035 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.035 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.035 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.035 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.035 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephadm", 2024-09-21T06:48:22.035 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.035 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.035 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.035 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_down_multiplier": { 2024-09-21T06:48:22.036 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_down_multiplier", 2024-09-21T06:48:22.036 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-21T06:48:22.036 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.036 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.036 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3.0", 2024-09-21T06:48:22.036 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.036 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.036 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.036 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T06:48:22.036 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.036 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.036 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.036 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.036 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_refresh_rate": { 2024-09-21T06:48:22.037 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_refresh_rate", 2024-09-21T06:48:22.037 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.037 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.037 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.037 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "20", 2024-09-21T06:48:22.037 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.037 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.037 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.037 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T06:48:22.037 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.037 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.037 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.037 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.038 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_starting_port": { 2024-09-21T06:48:22.038 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_starting_port", 2024-09-21T06:48:22.038 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.038 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.038 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.038 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "4721", 2024-09-21T06:48:22.038 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.038 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.038 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.038 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T06:48:22.038 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.038 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.038 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.038 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.038 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_ptrace": { 2024-09-21T06:48:22.039 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_ptrace", 2024-09-21T06:48:22.039 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.039 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.039 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.039 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.039 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.039 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.039 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.039 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T06:48:22.039 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T06:48:22.039 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.039 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.039 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.039 INFO:teuthology.orchestra.run.smithi067.stdout: "autotune_interval": { 2024-09-21T06:48:22.040 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "autotune_interval", 2024-09-21T06:48:22.040 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.040 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.040 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.040 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-09-21T06:48:22.040 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.040 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.040 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.040 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T06:48:22.040 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.040 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.040 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.040 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.041 INFO:teuthology.orchestra.run.smithi067.stdout: "autotune_memory_target_ratio": { 2024-09-21T06:48:22.041 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T06:48:22.041 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-21T06:48:22.041 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.041 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.041 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0.7", 2024-09-21T06:48:22.041 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.041 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.041 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.041 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T06:48:22.041 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.041 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.041 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.041 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.042 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm_log_destination": { 2024-09-21T06:48:22.042 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephadm_log_destination", 2024-09-21T06:48:22.042 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.042 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.042 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.042 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.042 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.042 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.042 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.042 INFO:teuthology.orchestra.run.smithi067.stdout: "file", 2024-09-21T06:48:22.042 INFO:teuthology.orchestra.run.smithi067.stdout: "file,syslog", 2024-09-21T06:48:22.042 INFO:teuthology.orchestra.run.smithi067.stdout: "syslog" 2024-09-21T06:48:22.043 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.043 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T06:48:22.043 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.043 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.043 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.043 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.043 INFO:teuthology.orchestra.run.smithi067.stdout: "cgroups_split": { 2024-09-21T06:48:22.043 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cgroups_split", 2024-09-21T06:48:22.043 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.043 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.043 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.043 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.043 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.043 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.043 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.044 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T06:48:22.044 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.044 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.044 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.044 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.044 INFO:teuthology.orchestra.run.smithi067.stdout: "config_checks_enabled": { 2024-09-21T06:48:22.044 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_checks_enabled", 2024-09-21T06:48:22.044 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.044 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.044 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.044 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.044 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.044 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.044 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.044 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T06:48:22.045 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.045 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.045 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.045 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.045 INFO:teuthology.orchestra.run.smithi067.stdout: "config_dashboard": { 2024-09-21T06:48:22.045 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_dashboard", 2024-09-21T06:48:22.045 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.045 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.045 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.045 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.045 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.045 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.045 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.045 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T06:48:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_alertmanager": { 2024-09-21T06:48:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_alertmanager", 2024-09-21T06:48:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T06:48:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-09-21T06:48:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_base": { 2024-09-21T06:48:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_base", 2024-09-21T06:48:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T06:48:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Container image name, without the tag", 2024-09-21T06:48:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_elasticsearch": { 2024-09-21T06:48:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_elasticsearch", 2024-09-21T06:48:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T06:48:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "elasticsearch container image", 2024-09-21T06:48:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_grafana": { 2024-09-21T06:48:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_grafana", 2024-09-21T06:48:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T06:48:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-09-21T06:48:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_haproxy": { 2024-09-21T06:48:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_haproxy", 2024-09-21T06:48:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T06:48:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "HAproxy container image", 2024-09-21T06:48:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_agent": { 2024-09-21T06:48:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_agent", 2024-09-21T06:48:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T06:48:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger agent container image", 2024-09-21T06:48:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_collector": { 2024-09-21T06:48:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_collector", 2024-09-21T06:48:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T06:48:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger collector container image", 2024-09-21T06:48:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_query": { 2024-09-21T06:48:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_query", 2024-09-21T06:48:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T06:48:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger query container image", 2024-09-21T06:48:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_keepalived": { 2024-09-21T06:48:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_keepalived", 2024-09-21T06:48:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T06:48:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Keepalived container image", 2024-09-21T06:48:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_loki": { 2024-09-21T06:48:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_loki", 2024-09-21T06:48:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T06:48:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Loki container image", 2024-09-21T06:48:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_nginx": { 2024-09-21T06:48:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_nginx", 2024-09-21T06:48:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T06:48:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Nginx container image", 2024-09-21T06:48:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_node_exporter": { 2024-09-21T06:48:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_node_exporter", 2024-09-21T06:48:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T06:48:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-09-21T06:48:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.058 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.058 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.058 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.058 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_nvmeof": { 2024-09-21T06:48:22.058 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_nvmeof", 2024-09-21T06:48:22.058 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.058 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.058 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.058 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T06:48:22.058 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.058 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.058 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.058 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Nvme-of container image", 2024-09-21T06:48:22.058 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.059 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.059 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.059 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.059 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_oauth2_proxy": { 2024-09-21T06:48:22.059 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T06:48:22.059 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.059 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.059 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.059 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T06:48:22.059 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.059 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.059 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.059 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "oauth2-proxy container image", 2024-09-21T06:48:22.059 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.059 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.060 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.060 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.060 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_prometheus": { 2024-09-21T06:48:22.060 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_prometheus", 2024-09-21T06:48:22.060 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.060 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.060 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.060 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T06:48:22.060 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.060 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.060 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.060 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-09-21T06:48:22.060 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.060 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.060 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.061 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.061 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_promtail": { 2024-09-21T06:48:22.061 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_promtail", 2024-09-21T06:48:22.061 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.061 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.061 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.061 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T06:48:22.061 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.061 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.061 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.061 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Promtail container image", 2024-09-21T06:48:22.061 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.061 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.062 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.062 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.062 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_samba": { 2024-09-21T06:48:22.062 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_samba", 2024-09-21T06:48:22.062 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.062 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.062 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.062 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T06:48:22.062 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.062 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.062 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.062 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Samba/SMB container image", 2024-09-21T06:48:22.062 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.062 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.062 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.063 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.063 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_samba_metrics": { 2024-09-21T06:48:22.063 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_samba_metrics", 2024-09-21T06:48:22.063 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.063 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.063 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.063 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T06:48:22.063 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.063 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.063 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.063 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T06:48:22.063 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.063 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.063 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.064 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.064 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_snmp_gateway": { 2024-09-21T06:48:22.064 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_snmp_gateway", 2024-09-21T06:48:22.064 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.064 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.064 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.064 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T06:48:22.064 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.064 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.064 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.064 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SNMP Gateway container image", 2024-09-21T06:48:22.064 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.064 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.064 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.065 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.065 INFO:teuthology.orchestra.run.smithi067.stdout: "container_init": { 2024-09-21T06:48:22.065 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_init", 2024-09-21T06:48:22.065 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.065 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.065 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.065 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.065 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.065 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.065 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.065 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T06:48:22.065 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.065 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.065 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.066 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.066 INFO:teuthology.orchestra.run.smithi067.stdout: "daemon_cache_timeout": { 2024-09-21T06:48:22.066 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "daemon_cache_timeout", 2024-09-21T06:48:22.066 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.066 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.066 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.066 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-09-21T06:48:22.066 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.066 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.066 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.066 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T06:48:22.066 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.066 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.066 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.067 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.067 INFO:teuthology.orchestra.run.smithi067.stdout: "default_cephadm_command_timeout": { 2024-09-21T06:48:22.067 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T06:48:22.067 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.067 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.067 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.067 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "900", 2024-09-21T06:48:22.067 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.067 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.067 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.067 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T06:48:22.067 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.067 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.067 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.067 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.068 INFO:teuthology.orchestra.run.smithi067.stdout: "default_registry": { 2024-09-21T06:48:22.068 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "default_registry", 2024-09-21T06:48:22.068 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.068 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.068 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.068 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io", 2024-09-21T06:48:22.068 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.068 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.068 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.068 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T06:48:22.068 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.068 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.068 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.068 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.069 INFO:teuthology.orchestra.run.smithi067.stdout: "device_cache_timeout": { 2024-09-21T06:48:22.069 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_cache_timeout", 2024-09-21T06:48:22.069 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.069 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.069 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.069 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1800", 2024-09-21T06:48:22.069 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.069 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.069 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.069 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache device inventory", 2024-09-21T06:48:22.069 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.069 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.069 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.069 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.070 INFO:teuthology.orchestra.run.smithi067.stdout: "device_enhanced_scan": { 2024-09-21T06:48:22.070 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_enhanced_scan", 2024-09-21T06:48:22.070 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.070 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.070 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.070 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.070 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.070 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.070 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.070 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T06:48:22.070 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.070 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.070 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.070 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.070 INFO:teuthology.orchestra.run.smithi067.stdout: "facts_cache_timeout": { 2024-09-21T06:48:22.071 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "facts_cache_timeout", 2024-09-21T06:48:22.071 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.071 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.071 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.071 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-09-21T06:48:22.071 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.071 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.071 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.071 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache host facts data", 2024-09-21T06:48:22.071 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.071 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.071 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.071 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.071 INFO:teuthology.orchestra.run.smithi067.stdout: "grafana_dashboards_path": { 2024-09-21T06:48:22.072 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "grafana_dashboards_path", 2024-09-21T06:48:22.072 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.072 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.072 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.072 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T06:48:22.072 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.072 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.072 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.072 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T06:48:22.072 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.072 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.072 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.072 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.072 INFO:teuthology.orchestra.run.smithi067.stdout: "host_check_interval": { 2024-09-21T06:48:22.073 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "host_check_interval", 2024-09-21T06:48:22.073 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.073 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.073 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.073 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-09-21T06:48:22.073 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.073 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.073 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.073 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to perform a host check", 2024-09-21T06:48:22.073 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.073 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.073 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.073 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.073 INFO:teuthology.orchestra.run.smithi067.stdout: "hw_monitoring": { 2024-09-21T06:48:22.074 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "hw_monitoring", 2024-09-21T06:48:22.074 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.074 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.074 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.074 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.074 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.074 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.074 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.074 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T06:48:22.074 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.074 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.074 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.074 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.074 INFO:teuthology.orchestra.run.smithi067.stdout: "inventory_list_all": { 2024-09-21T06:48:22.075 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "inventory_list_all", 2024-09-21T06:48:22.075 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.075 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.075 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.075 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.075 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.075 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.075 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.075 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T06:48:22.075 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.075 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.075 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.075 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.075 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.076 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.076 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.076 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.076 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.076 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.076 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.076 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.076 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.076 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.076 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.076 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.076 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.076 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.076 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.077 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.077 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.077 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.077 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.077 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.077 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.077 INFO:teuthology.orchestra.run.smithi067.stdout: "log_refresh_metadata": { 2024-09-21T06:48:22.077 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_refresh_metadata", 2024-09-21T06:48:22.077 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.077 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.077 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.077 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.077 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.077 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.078 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.078 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T06:48:22.078 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.078 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.078 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.078 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.078 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.078 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.078 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.078 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.078 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.078 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.078 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.079 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.079 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.079 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T06:48:22.079 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.079 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.079 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.079 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.079 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.079 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.079 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.079 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.079 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.079 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.079 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.080 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.080 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.080 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.080 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.080 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.080 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.080 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.080 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.080 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.080 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.080 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.080 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.080 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.080 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.080 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T06:48:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T06:48:22.082 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.082 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.082 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.082 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.082 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.082 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.082 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.082 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T06:48:22.082 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.082 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.082 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.082 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.082 INFO:teuthology.orchestra.run.smithi067.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T06:48:22.082 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T06:48:22.083 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.083 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.083 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.083 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "*", 2024-09-21T06:48:22.083 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.083 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.083 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.083 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T06:48:22.083 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.083 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.083 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.083 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.083 INFO:teuthology.orchestra.run.smithi067.stdout: "max_count_per_host": { 2024-09-21T06:48:22.083 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_count_per_host", 2024-09-21T06:48:22.084 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.084 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.084 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.084 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-21T06:48:22.084 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.084 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.084 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.084 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "max number of daemons per service per host", 2024-09-21T06:48:22.084 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.084 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.084 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.084 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.084 INFO:teuthology.orchestra.run.smithi067.stdout: "max_osd_draining_count": { 2024-09-21T06:48:22.084 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_osd_draining_count", 2024-09-21T06:48:22.085 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.085 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.085 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.085 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-21T06:48:22.085 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.085 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.085 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.085 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T06:48:22.085 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.085 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.085 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.085 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.085 INFO:teuthology.orchestra.run.smithi067.stdout: "migration_current": { 2024-09-21T06:48:22.086 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "migration_current", 2024-09-21T06:48:22.086 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.086 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.086 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.086 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.086 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.086 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.086 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.086 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "internal - do not modify", 2024-09-21T06:48:22.086 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.086 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.086 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.086 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.086 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": { 2024-09-21T06:48:22.087 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mode", 2024-09-21T06:48:22.087 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.087 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.087 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.087 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "root", 2024-09-21T06:48:22.087 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.087 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.087 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.087 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm-package", 2024-09-21T06:48:22.087 INFO:teuthology.orchestra.run.smithi067.stdout: "root" 2024-09-21T06:48:22.087 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.087 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T06:48:22.087 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.087 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.088 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.088 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.088 INFO:teuthology.orchestra.run.smithi067.stdout: "oob_default_addr": { 2024-09-21T06:48:22.088 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "oob_default_addr", 2024-09-21T06:48:22.088 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.088 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.088 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.088 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "169.254.1.1", 2024-09-21T06:48:22.088 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.088 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.088 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.088 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T06:48:22.088 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.088 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.089 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.089 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.089 INFO:teuthology.orchestra.run.smithi067.stdout: "prometheus_alerts_path": { 2024-09-21T06:48:22.089 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prometheus_alerts_path", 2024-09-21T06:48:22.089 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.089 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.089 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.089 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T06:48:22.089 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.089 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.089 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.089 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T06:48:22.089 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.089 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.089 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.090 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.090 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_insecure": { 2024-09-21T06:48:22.090 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_insecure", 2024-09-21T06:48:22.090 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.090 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.090 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.090 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.090 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.090 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.090 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.090 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T06:48:22.090 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.090 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.090 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.091 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.091 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_password": { 2024-09-21T06:48:22.091 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_password", 2024-09-21T06:48:22.091 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.091 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.091 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.091 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.091 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.091 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.091 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.091 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T06:48:22.091 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.091 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.091 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.091 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.092 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_url": { 2024-09-21T06:48:22.092 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_url", 2024-09-21T06:48:22.092 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.092 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.092 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.092 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.092 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.092 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.092 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.092 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T06:48:22.092 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.092 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.092 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.092 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.092 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_username": { 2024-09-21T06:48:22.093 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_username", 2024-09-21T06:48:22.093 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.093 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.093 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.093 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.093 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.093 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.093 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.093 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T06:48:22.093 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.093 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.093 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.093 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.093 INFO:teuthology.orchestra.run.smithi067.stdout: "secure_monitoring_stack": { 2024-09-21T06:48:22.093 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "secure_monitoring_stack", 2024-09-21T06:48:22.094 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.094 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.094 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.094 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.094 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.094 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.094 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.094 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T06:48:22.094 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.094 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.094 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.094 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.094 INFO:teuthology.orchestra.run.smithi067.stdout: "service_discovery_port": { 2024-09-21T06:48:22.094 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "service_discovery_port", 2024-09-21T06:48:22.094 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.095 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.095 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.095 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8765", 2024-09-21T06:48:22.095 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.095 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.095 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.095 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "cephadm service discovery port", 2024-09-21T06:48:22.095 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.095 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.095 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.095 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.095 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.095 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.095 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.095 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.096 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.096 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.096 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.096 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.096 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.096 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.096 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.096 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.096 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.096 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.096 INFO:teuthology.orchestra.run.smithi067.stdout: "ssh_config_file": { 2024-09-21T06:48:22.096 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssh_config_file", 2024-09-21T06:48:22.096 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.096 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.096 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.097 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.097 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.097 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.097 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.097 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T06:48:22.097 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.097 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.097 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.097 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.097 INFO:teuthology.orchestra.run.smithi067.stdout: "ssh_keepalive_count_max": { 2024-09-21T06:48:22.097 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T06:48:22.097 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.097 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.097 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.098 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3", 2024-09-21T06:48:22.098 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.098 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.098 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.098 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T06:48:22.098 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.098 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.098 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.098 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.098 INFO:teuthology.orchestra.run.smithi067.stdout: "ssh_keepalive_interval": { 2024-09-21T06:48:22.098 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssh_keepalive_interval", 2024-09-21T06:48:22.098 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.098 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.098 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.099 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7", 2024-09-21T06:48:22.099 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.099 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.099 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.099 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T06:48:22.099 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.099 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.099 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.099 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.099 INFO:teuthology.orchestra.run.smithi067.stdout: "use_agent": { 2024-09-21T06:48:22.099 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "use_agent", 2024-09-21T06:48:22.099 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.099 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.099 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.099 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.100 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.100 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.100 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.100 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T06:48:22.100 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.100 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.100 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.100 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.100 INFO:teuthology.orchestra.run.smithi067.stdout: "use_repo_digest": { 2024-09-21T06:48:22.100 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "use_repo_digest", 2024-09-21T06:48:22.100 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.100 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.100 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.100 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.101 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.101 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.101 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.101 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T06:48:22.101 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.101 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.101 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.101 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.101 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_failed_host_check": { 2024-09-21T06:48:22.101 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_failed_host_check", 2024-09-21T06:48:22.101 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.101 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.101 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.101 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.101 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.102 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.102 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.102 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T06:48:22.102 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.102 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.102 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.102 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.102 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_stray_daemons": { 2024-09-21T06:48:22.102 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_stray_daemons", 2024-09-21T06:48:22.102 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.102 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.102 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.102 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.102 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.103 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.103 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.103 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T06:48:22.103 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.103 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.103 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.103 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.103 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_stray_hosts": { 2024-09-21T06:48:22.103 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_stray_hosts", 2024-09-21T06:48:22.103 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.103 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.103 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.103 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.103 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.103 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.104 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.104 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T06:48:22.104 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.104 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.104 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.104 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.104 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.104 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.104 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.104 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crash", 2024-09-21T06:48:22.104 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.104 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.104 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.104 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.104 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.105 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.105 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.105 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.105 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.105 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.105 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.105 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.105 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.105 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.105 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.105 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.105 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.105 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.105 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.105 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.106 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.106 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.106 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.106 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.106 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.106 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.106 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.106 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.106 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.106 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.106 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.106 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.106 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.106 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.107 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.107 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.107 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.107 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.107 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.107 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.107 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.107 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.107 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.107 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.107 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.107 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.107 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.107 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.108 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.108 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.108 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.108 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.108 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.108 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.108 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.108 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.108 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.108 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.108 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.108 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.108 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.108 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.108 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.109 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.109 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.109 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.109 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.109 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.109 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.109 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.109 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.109 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.109 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.109 INFO:teuthology.orchestra.run.smithi067.stdout: "retain_interval": { 2024-09-21T06:48:22.109 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "retain_interval", 2024-09-21T06:48:22.109 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.110 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.110 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.110 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "31536000", 2024-09-21T06:48:22.110 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.110 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.110 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.110 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T06:48:22.110 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.110 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.110 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.110 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.110 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.110 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.110 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.110 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.111 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.111 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.111 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.111 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.111 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.111 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.111 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.111 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.111 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.111 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.111 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_recent_interval": { 2024-09-21T06:48:22.111 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_recent_interval", 2024-09-21T06:48:22.111 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.111 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.112 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.112 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1209600", 2024-09-21T06:48:22.112 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.112 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.112 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.112 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T06:48:22.112 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.112 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.112 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.112 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.112 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.112 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.112 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.112 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "dashboard", 2024-09-21T06:48:22.112 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.113 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.113 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.113 INFO:teuthology.orchestra.run.smithi067.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T06:48:22.113 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T06:48:22.113 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.113 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.113 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.113 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-21T06:48:22.113 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.113 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.113 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.113 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.113 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.113 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.114 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.114 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.114 INFO:teuthology.orchestra.run.smithi067.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T06:48:22.114 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T06:48:22.114 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.114 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.114 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.114 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.114 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.114 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.114 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.114 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.114 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.114 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.115 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.115 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.115 INFO:teuthology.orchestra.run.smithi067.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T06:48:22.115 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T06:48:22.115 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.115 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.115 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.115 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.115 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.115 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.115 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.115 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.115 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.115 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.115 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.116 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.116 INFO:teuthology.orchestra.run.smithi067.stdout: "AUDIT_API_ENABLED": { 2024-09-21T06:48:22.116 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T06:48:22.116 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.116 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.116 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.116 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.116 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.116 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.116 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.116 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.116 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.116 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.116 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.116 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.117 INFO:teuthology.orchestra.run.smithi067.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T06:48:22.117 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T06:48:22.117 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.117 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.117 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.117 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.117 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.117 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.117 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.117 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.117 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.117 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.117 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.117 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.117 INFO:teuthology.orchestra.run.smithi067.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T06:48:22.118 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T06:48:22.118 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.118 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.118 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.118 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.118 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.118 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.118 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.118 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.118 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.118 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.118 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.118 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T06:48:22.118 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T06:48:22.119 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.119 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.119 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.119 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.119 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.119 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.119 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.119 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.119 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.119 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.119 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.119 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.119 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T06:48:22.119 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T06:48:22.120 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.120 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.120 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.120 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.120 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.120 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.120 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.120 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.120 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.120 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.120 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.120 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.120 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T06:48:22.120 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T06:48:22.120 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.121 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.121 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.121 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.121 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.121 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.121 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.121 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.121 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.121 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.121 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.121 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.121 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T06:48:22.121 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T06:48:22.121 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.121 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.122 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.122 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.122 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.122 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.122 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.122 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.122 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.122 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.122 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.122 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.122 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T06:48:22.122 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T06:48:22.122 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.123 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.123 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.123 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.123 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.123 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.123 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.123 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.123 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.123 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.123 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.123 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.123 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T06:48:22.123 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T06:48:22.123 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.123 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.124 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.124 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.124 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.124 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.124 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.124 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.124 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.124 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.124 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.124 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.124 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T06:48:22.124 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T06:48:22.124 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.124 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.124 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.125 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.125 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.125 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.125 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.125 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.125 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.125 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.125 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.125 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.125 INFO:teuthology.orchestra.run.smithi067.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T06:48:22.125 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T06:48:22.125 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.125 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.125 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.126 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.126 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.126 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.126 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.126 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.126 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.126 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.126 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.126 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.126 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T06:48:22.126 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T06:48:22.126 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.126 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.126 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.126 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-09-21T06:48:22.127 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.127 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.127 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.127 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.127 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.127 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.127 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.127 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.127 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T06:48:22.127 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T06:48:22.127 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.127 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.127 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.127 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.127 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.128 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.128 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.128 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.128 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.128 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.128 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.128 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.128 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_URL": { 2024-09-21T06:48:22.128 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_URL", 2024-09-21T06:48:22.128 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.128 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.128 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.128 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.128 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.129 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.129 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.129 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.129 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.129 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.129 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.129 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.129 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T06:48:22.129 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T06:48:22.129 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.129 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.129 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.129 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-09-21T06:48:22.129 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.129 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.130 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.130 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.130 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.130 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.130 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.130 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.130 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T06:48:22.130 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T06:48:22.130 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.130 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.130 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.130 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.130 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.130 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.130 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.131 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.131 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.131 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.131 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.131 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.131 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T06:48:22.131 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T06:48:22.131 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.131 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.131 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.131 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.131 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.131 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.131 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.131 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.132 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.132 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.132 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.132 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.132 INFO:teuthology.orchestra.run.smithi067.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T06:48:22.132 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T06:48:22.132 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.132 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.132 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.132 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.132 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.132 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.132 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.132 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.132 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.133 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.133 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.133 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.133 INFO:teuthology.orchestra.run.smithi067.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T06:48:22.133 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T06:48:22.133 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.133 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.133 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.133 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.133 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.133 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.133 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.133 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.133 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.133 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.133 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.134 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.134 INFO:teuthology.orchestra.run.smithi067.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T06:48:22.134 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T06:48:22.134 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.134 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.134 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.134 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "[]", 2024-09-21T06:48:22.134 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.134 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.134 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.134 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.134 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.134 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.134 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.134 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.135 INFO:teuthology.orchestra.run.smithi067.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T06:48:22.135 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T06:48:22.135 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.135 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.135 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.135 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "{}", 2024-09-21T06:48:22.135 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.135 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.135 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.135 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.135 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.135 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.135 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.135 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.135 INFO:teuthology.orchestra.run.smithi067.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T06:48:22.136 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T06:48:22.136 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.136 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.136 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.136 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.136 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.136 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.136 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.136 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.136 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.136 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.136 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.136 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.136 INFO:teuthology.orchestra.run.smithi067.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T06:48:22.137 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T06:48:22.137 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.137 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.137 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.137 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.137 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.137 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.137 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.137 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.137 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.137 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.137 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.137 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.137 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T06:48:22.138 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T06:48:22.138 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.138 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.138 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.138 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.138 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.138 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.138 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.138 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.138 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.138 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.138 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.138 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.138 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T06:48:22.138 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T06:48:22.139 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.139 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.139 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.139 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.139 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.139 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.139 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.139 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.139 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.139 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.139 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.139 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.139 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T06:48:22.139 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T06:48:22.140 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.140 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.140 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.140 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.140 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.140 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.140 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.140 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.140 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.140 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.140 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.140 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.140 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T06:48:22.140 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T06:48:22.141 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.141 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.141 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.141 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.141 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.141 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.141 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.141 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.141 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.141 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.141 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.141 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.141 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T06:48:22.141 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T06:48:22.142 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.142 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.142 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.142 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.142 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.142 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.142 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.142 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.142 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.142 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.142 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.142 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.142 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T06:48:22.142 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T06:48:22.143 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.143 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.143 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.143 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.143 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.143 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.143 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.143 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.143 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.143 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.143 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.143 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T06:48:22.143 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T06:48:22.144 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.144 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.144 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.144 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.144 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.144 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.144 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.144 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.144 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.144 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.144 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.144 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.144 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T06:48:22.144 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T06:48:22.144 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.145 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.145 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.145 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.145 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.145 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.145 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.145 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.145 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.145 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.145 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.145 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.145 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T06:48:22.145 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T06:48:22.145 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.145 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.146 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.146 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T06:48:22.146 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.146 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.146 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.146 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.146 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.146 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.146 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.146 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.146 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T06:48:22.146 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T06:48:22.146 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.146 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.146 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.147 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-21T06:48:22.147 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.147 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.147 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.147 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.147 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.147 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.147 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.147 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T06:48:22.147 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T06:48:22.147 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.147 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.147 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.148 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8", 2024-09-21T06:48:22.148 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.148 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.148 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.148 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.148 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.148 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.148 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.148 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.148 INFO:teuthology.orchestra.run.smithi067.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T06:48:22.148 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T06:48:22.148 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.148 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.148 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.148 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "45", 2024-09-21T06:48:22.149 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.149 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.149 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.149 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.149 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.149 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.149 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.149 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.149 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T06:48:22.149 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T06:48:22.149 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.149 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.149 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.149 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.149 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.149 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.149 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.150 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.150 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.150 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.150 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.150 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.150 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T06:48:22.150 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T06:48:22.150 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.150 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.150 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.150 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-09-21T06:48:22.150 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.150 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.150 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.151 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.151 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.151 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.151 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.151 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.151 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T06:48:22.151 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T06:48:22.151 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.151 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.151 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.151 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.151 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.151 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.152 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.152 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.152 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.152 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.152 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.152 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.152 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T06:48:22.152 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T06:48:22.152 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.152 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.152 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.152 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.152 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.152 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.152 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.152 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.153 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.153 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.153 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.153 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.153 INFO:teuthology.orchestra.run.smithi067.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T06:48:22.153 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T06:48:22.153 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.153 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.153 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.153 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.153 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.153 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.153 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.153 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.153 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.153 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.154 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.154 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.154 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T06:48:22.154 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T06:48:22.154 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.154 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.154 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.154 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.154 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.154 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.154 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.154 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.154 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.154 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.155 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.155 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.155 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T06:48:22.155 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T06:48:22.155 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.155 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.155 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.155 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-21T06:48:22.155 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.155 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.155 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.155 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.155 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.155 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.155 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.156 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.156 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T06:48:22.156 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T06:48:22.156 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.156 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.156 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.156 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-09-21T06:48:22.156 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.156 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.156 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.156 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.156 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.156 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.156 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.157 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.157 INFO:teuthology.orchestra.run.smithi067.stdout: "cross_origin_url": { 2024-09-21T06:48:22.157 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cross_origin_url", 2024-09-21T06:48:22.157 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.157 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.157 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.157 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.157 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.157 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.157 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.157 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.157 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.157 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.157 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.158 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.158 INFO:teuthology.orchestra.run.smithi067.stdout: "crt_file": { 2024-09-21T06:48:22.158 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crt_file", 2024-09-21T06:48:22.158 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.158 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.158 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.158 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.158 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.158 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.158 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.158 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.158 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.158 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.158 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.159 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.159 INFO:teuthology.orchestra.run.smithi067.stdout: "debug": { 2024-09-21T06:48:22.159 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "debug", 2024-09-21T06:48:22.159 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.159 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.159 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.159 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.159 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.159 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.159 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.159 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable/disable debug options", 2024-09-21T06:48:22.159 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.159 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.159 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.159 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.159 INFO:teuthology.orchestra.run.smithi067.stdout: "jwt_token_ttl": { 2024-09-21T06:48:22.160 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "jwt_token_ttl", 2024-09-21T06:48:22.160 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.160 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.160 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.160 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "28800", 2024-09-21T06:48:22.160 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.160 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.160 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.160 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.160 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.160 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.160 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.160 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.160 INFO:teuthology.orchestra.run.smithi067.stdout: "key_file": { 2024-09-21T06:48:22.160 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "key_file", 2024-09-21T06:48:22.161 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.161 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.161 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.161 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.161 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.161 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.161 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.161 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.161 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.161 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.161 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.161 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.161 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.161 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.161 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.161 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.162 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.162 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.162 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.162 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.162 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.162 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.162 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.162 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.162 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.162 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.162 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.162 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.162 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.162 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.163 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.163 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.163 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.163 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.163 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.163 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.163 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.163 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.163 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.163 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.163 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.163 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.163 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.163 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.163 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.164 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.164 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.164 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.164 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.164 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.164 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.164 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.164 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.164 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.164 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.164 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.164 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.164 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.164 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.164 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.164 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.165 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.165 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.165 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.165 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.165 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.165 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.165 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.165 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.165 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.165 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.165 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.165 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.165 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.165 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.166 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.166 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.166 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.166 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.166 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.166 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.166 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.166 INFO:teuthology.orchestra.run.smithi067.stdout: "motd": { 2024-09-21T06:48:22.166 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "motd", 2024-09-21T06:48:22.166 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.166 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.166 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.166 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.167 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.167 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.167 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.167 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "The message of the day", 2024-09-21T06:48:22.167 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.167 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.167 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.167 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.167 INFO:teuthology.orchestra.run.smithi067.stdout: "redirect_resolve_ip_addr": { 2024-09-21T06:48:22.167 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T06:48:22.167 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.167 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.167 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.167 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.167 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.168 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.168 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.168 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.168 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.168 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.168 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.168 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.168 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-09-21T06:48:22.168 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-09-21T06:48:22.168 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.168 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.168 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.168 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "::", 2024-09-21T06:48:22.168 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.169 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.169 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.169 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.169 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.169 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.169 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.169 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.169 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-09-21T06:48:22.169 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-09-21T06:48:22.169 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.169 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.169 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.169 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8080", 2024-09-21T06:48:22.169 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.169 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.170 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.170 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.170 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.170 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.170 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.170 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.170 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.170 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.170 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.170 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.170 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.170 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.170 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.170 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.171 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.171 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.171 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.171 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.171 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.171 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.171 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl": { 2024-09-21T06:48:22.171 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl", 2024-09-21T06:48:22.171 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.171 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.171 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.171 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.171 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.171 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.171 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.172 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.172 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.172 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.172 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.172 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.172 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl_server_port": { 2024-09-21T06:48:22.172 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl_server_port", 2024-09-21T06:48:22.172 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.172 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.172 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.172 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8443", 2024-09-21T06:48:22.172 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.172 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.172 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.173 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.173 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.173 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.173 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.173 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.173 INFO:teuthology.orchestra.run.smithi067.stdout: "sso_oauth2": { 2024-09-21T06:48:22.173 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sso_oauth2", 2024-09-21T06:48:22.173 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.173 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.173 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.173 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.174 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.174 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.174 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.174 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.174 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.174 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.174 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.174 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.174 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_behaviour": { 2024-09-21T06:48:22.174 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_behaviour", 2024-09-21T06:48:22.174 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.174 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.174 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.174 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "redirect", 2024-09-21T06:48:22.175 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.175 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.175 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.175 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.175 INFO:teuthology.orchestra.run.smithi067.stdout: "redirect" 2024-09-21T06:48:22.175 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.175 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.175 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.175 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.175 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.175 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.175 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_error_status_code": { 2024-09-21T06:48:22.176 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_error_status_code", 2024-09-21T06:48:22.176 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.176 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.176 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.176 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-09-21T06:48:22.176 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "400", 2024-09-21T06:48:22.176 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "599", 2024-09-21T06:48:22.176 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.176 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.176 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.176 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.176 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.176 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.176 INFO:teuthology.orchestra.run.smithi067.stdout: "url_prefix": { 2024-09-21T06:48:22.177 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "url_prefix", 2024-09-21T06:48:22.177 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.177 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.177 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.177 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.177 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.177 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.177 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.177 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.177 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.177 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.177 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.177 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.177 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.178 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.178 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.178 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "devicehealth", 2024-09-21T06:48:22.178 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.178 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.178 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.178 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_monitoring": { 2024-09-21T06:48:22.178 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enable_monitoring", 2024-09-21T06:48:22.178 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.178 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.178 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.178 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.178 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.178 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.179 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.179 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "monitor device health metrics", 2024-09-21T06:48:22.179 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.179 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.179 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.179 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.179 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.179 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.179 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.179 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.179 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.179 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.179 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.179 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.179 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.180 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.180 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.180 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.180 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.180 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.180 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.180 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.180 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.180 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.180 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.180 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.180 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.180 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.180 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.180 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.181 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.181 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.181 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.181 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.181 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.181 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.181 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.181 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.181 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.181 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.181 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.181 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.181 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.181 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.181 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.181 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.182 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.182 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.182 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.182 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.182 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.182 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.182 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.182 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.182 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.182 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.182 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.182 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.182 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.183 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.183 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.183 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.183 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.183 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.183 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.183 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.183 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.183 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.183 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.183 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.183 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.183 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.183 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.183 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.184 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.184 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.184 INFO:teuthology.orchestra.run.smithi067.stdout: "mark_out_threshold": { 2024-09-21T06:48:22.184 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mark_out_threshold", 2024-09-21T06:48:22.184 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.184 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.184 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.184 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "2419200", 2024-09-21T06:48:22.184 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.184 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.184 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.184 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T06:48:22.184 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.184 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.184 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.185 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.185 INFO:teuthology.orchestra.run.smithi067.stdout: "pool_name": { 2024-09-21T06:48:22.185 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pool_name", 2024-09-21T06:48:22.185 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.185 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.185 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.185 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "device_health_metrics", 2024-09-21T06:48:22.185 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.185 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.185 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.185 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T06:48:22.185 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.185 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.185 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.186 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.186 INFO:teuthology.orchestra.run.smithi067.stdout: "retention_period": { 2024-09-21T06:48:22.186 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "retention_period", 2024-09-21T06:48:22.186 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.186 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.186 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.186 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15552000", 2024-09-21T06:48:22.186 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.186 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.186 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.186 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long to retain device health metrics", 2024-09-21T06:48:22.186 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.186 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.187 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.187 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.187 INFO:teuthology.orchestra.run.smithi067.stdout: "scrape_frequency": { 2024-09-21T06:48:22.187 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "scrape_frequency", 2024-09-21T06:48:22.187 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.187 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.187 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.187 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "86400", 2024-09-21T06:48:22.187 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.187 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.187 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.187 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T06:48:22.187 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.187 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.187 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.188 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.188 INFO:teuthology.orchestra.run.smithi067.stdout: "self_heal": { 2024-09-21T06:48:22.188 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "self_heal", 2024-09-21T06:48:22.188 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.188 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.188 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.188 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.188 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.188 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.188 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.188 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T06:48:22.188 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.188 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.189 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.189 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.189 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-09-21T06:48:22.189 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-09-21T06:48:22.189 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.189 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.189 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.189 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-09-21T06:48:22.189 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.189 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.189 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.189 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T06:48:22.189 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.189 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.190 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.190 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.190 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.190 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.190 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.190 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.190 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.190 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.190 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.190 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.190 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.190 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.190 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.191 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.191 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.191 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.191 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_threshold": { 2024-09-21T06:48:22.191 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_threshold", 2024-09-21T06:48:22.191 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.191 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.191 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.191 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7257600", 2024-09-21T06:48:22.191 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.191 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.191 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.191 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T06:48:22.191 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.192 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.192 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.192 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.192 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.192 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.192 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.192 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "diskprediction_local", 2024-09-21T06:48:22.192 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.192 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.192 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.192 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.192 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.192 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.192 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.193 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.193 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.193 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.193 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.193 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.193 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.193 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.193 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.193 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.193 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.193 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.193 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.193 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.193 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.194 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.194 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.194 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.194 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.194 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.194 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.194 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.194 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.194 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.194 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.194 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.194 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.194 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.194 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.194 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.195 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.195 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.195 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.195 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.195 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.195 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.195 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.195 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.195 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.195 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.195 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.195 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.195 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.196 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.196 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.196 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.196 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.196 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.196 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.196 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.196 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.196 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.196 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.196 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.196 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.196 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.196 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.196 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.197 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.197 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.197 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.197 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.197 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.197 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.197 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.197 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.197 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.197 INFO:teuthology.orchestra.run.smithi067.stdout: "predict_interval": { 2024-09-21T06:48:22.197 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "predict_interval", 2024-09-21T06:48:22.197 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.197 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.197 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.197 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "86400", 2024-09-21T06:48:22.198 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.198 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.198 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.198 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.198 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.198 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.198 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.198 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.198 INFO:teuthology.orchestra.run.smithi067.stdout: "predictor_model": { 2024-09-21T06:48:22.198 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "predictor_model", 2024-09-21T06:48:22.198 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.198 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.198 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.198 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "prophetstor", 2024-09-21T06:48:22.198 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.199 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.199 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.199 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.199 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.199 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.199 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.199 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.199 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-09-21T06:48:22.199 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-09-21T06:48:22.199 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.199 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.199 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.199 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-09-21T06:48:22.199 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.199 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.200 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.200 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.200 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.200 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.200 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.200 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.200 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.200 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.200 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.200 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.200 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.200 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.200 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.200 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.201 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.201 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.201 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.201 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.201 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.201 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.201 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.201 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.201 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.201 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "influx", 2024-09-21T06:48:22.201 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": false, 2024-09-21T06:48:22.201 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "influxdb python module not found", 2024-09-21T06:48:22.201 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.201 INFO:teuthology.orchestra.run.smithi067.stdout: "batch_size": { 2024-09-21T06:48:22.201 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "batch_size", 2024-09-21T06:48:22.202 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.202 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.202 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.202 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5000", 2024-09-21T06:48:22.202 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.202 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.202 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.202 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T06:48:22.202 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.202 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.202 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.202 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.202 INFO:teuthology.orchestra.run.smithi067.stdout: "database": { 2024-09-21T06:48:22.202 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "database", 2024-09-21T06:48:22.202 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.203 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.203 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.203 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "ceph", 2024-09-21T06:48:22.203 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.203 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.203 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.203 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T06:48:22.203 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.203 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.203 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.203 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.203 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": { 2024-09-21T06:48:22.203 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "hostname", 2024-09-21T06:48:22.203 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.203 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.203 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.204 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.204 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.204 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.204 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.204 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB server hostname", 2024-09-21T06:48:22.204 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.204 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.204 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.204 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.204 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-09-21T06:48:22.204 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-09-21T06:48:22.204 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.204 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.204 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.204 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "30", 2024-09-21T06:48:22.204 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "5", 2024-09-21T06:48:22.205 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.205 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.205 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T06:48:22.205 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.205 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.205 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.205 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.205 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.205 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.205 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.205 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.205 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.205 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.205 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.205 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.205 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.206 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.206 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.206 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.206 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.206 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.206 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.206 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.206 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.206 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.206 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.206 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.206 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.206 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.210 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.210 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.210 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.210 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.210 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.210 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.210 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.210 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.210 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.210 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.210 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.210 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.211 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.211 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.211 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.211 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.211 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.211 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.211 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.211 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.211 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.211 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.211 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.211 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.211 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.211 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.212 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.212 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.212 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.212 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.212 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.212 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.212 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.212 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.212 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.212 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.212 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.212 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.212 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.212 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.212 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.213 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.213 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.213 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.213 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.213 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.213 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.213 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.213 INFO:teuthology.orchestra.run.smithi067.stdout: "password": { 2024-09-21T06:48:22.213 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "password", 2024-09-21T06:48:22.213 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.213 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.213 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.213 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.213 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.214 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.214 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.214 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "password of InfluxDB server user", 2024-09-21T06:48:22.214 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.214 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.214 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.214 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.214 INFO:teuthology.orchestra.run.smithi067.stdout: "port": { 2024-09-21T06:48:22.214 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "port", 2024-09-21T06:48:22.214 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.214 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.214 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.214 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8086", 2024-09-21T06:48:22.214 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.215 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.215 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.215 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB server port", 2024-09-21T06:48:22.215 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.215 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.215 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.215 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.215 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.215 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.215 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.215 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.215 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.215 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.215 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.215 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.215 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.216 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.216 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.216 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.216 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.216 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.216 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl": { 2024-09-21T06:48:22.216 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl", 2024-09-21T06:48:22.216 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.216 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.216 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.216 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "false", 2024-09-21T06:48:22.216 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.216 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.216 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.217 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T06:48:22.217 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.217 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.217 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.217 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.217 INFO:teuthology.orchestra.run.smithi067.stdout: "threads": { 2024-09-21T06:48:22.217 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "threads", 2024-09-21T06:48:22.217 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.217 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.217 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.217 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-09-21T06:48:22.217 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-09-21T06:48:22.217 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "32", 2024-09-21T06:48:22.217 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.217 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T06:48:22.218 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.218 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.218 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.218 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.218 INFO:teuthology.orchestra.run.smithi067.stdout: "username": { 2024-09-21T06:48:22.218 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "username", 2024-09-21T06:48:22.218 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.218 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.218 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.218 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.218 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.218 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.218 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.218 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "username of InfluxDB server user", 2024-09-21T06:48:22.218 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.219 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.219 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.219 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.219 INFO:teuthology.orchestra.run.smithi067.stdout: "verify_ssl": { 2024-09-21T06:48:22.219 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "verify_ssl", 2024-09-21T06:48:22.219 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.219 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.219 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.219 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "true", 2024-09-21T06:48:22.219 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.219 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.219 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.219 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T06:48:22.219 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.220 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.220 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.220 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.220 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.220 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.220 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.220 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "insights", 2024-09-21T06:48:22.220 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.220 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.220 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.220 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.220 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.220 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.220 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.221 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.221 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.221 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.221 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.221 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.221 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.221 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.221 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.221 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.221 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.221 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.221 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.221 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.221 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.222 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.222 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.222 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.222 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.222 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.222 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.222 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.222 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.222 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.222 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.222 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.222 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.222 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.222 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.223 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.223 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.223 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.223 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.223 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.223 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.223 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.223 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.223 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.223 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.223 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.223 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.223 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.223 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.224 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.224 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.224 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.224 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.224 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.224 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.224 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.224 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.224 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.224 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.224 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.224 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.224 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.224 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.224 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.225 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.225 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.225 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.225 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.225 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.225 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.225 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.225 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.225 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.225 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.225 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.225 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.225 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.225 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.225 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.226 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.226 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.226 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.226 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.226 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.226 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.226 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.226 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.226 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.226 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.226 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.226 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "iostat", 2024-09-21T06:48:22.226 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.226 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.227 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.227 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.227 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.227 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.227 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.227 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.227 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.227 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.227 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.227 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.227 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.227 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.227 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.227 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.227 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.228 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.228 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.228 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.228 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.228 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.228 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.228 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.228 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.228 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.228 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.228 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.228 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.228 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.228 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.229 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.229 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.229 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.229 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.229 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.229 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.229 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.229 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.229 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.229 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.229 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.229 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.229 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.230 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.230 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.230 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.230 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.230 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.230 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.230 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.230 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.230 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.230 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.230 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.230 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.230 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.230 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.231 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.231 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.231 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.231 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.231 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.231 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.231 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.231 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.231 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.231 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.231 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.231 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.231 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.231 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.231 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.232 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.232 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.232 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.232 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.232 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.232 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.232 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.232 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.232 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.232 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.232 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.232 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.232 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.232 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.232 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.233 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.233 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.233 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "k8sevents", 2024-09-21T06:48:22.233 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.233 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.233 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.233 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_event_retention_days": { 2024-09-21T06:48:22.233 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ceph_event_retention_days", 2024-09-21T06:48:22.233 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.233 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.233 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.233 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7", 2024-09-21T06:48:22.233 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.233 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.233 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.233 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T06:48:22.234 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.234 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.234 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.234 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.234 INFO:teuthology.orchestra.run.smithi067.stdout: "config_check_secs": { 2024-09-21T06:48:22.234 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_check_secs", 2024-09-21T06:48:22.234 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.234 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.234 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.234 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-21T06:48:22.234 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "10", 2024-09-21T06:48:22.234 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.234 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.234 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T06:48:22.235 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.235 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.235 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.235 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.235 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.235 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.235 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.235 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.235 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.235 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.235 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.235 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.235 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.235 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.235 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.236 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.236 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.236 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.236 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.236 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.236 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.236 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.236 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.236 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.236 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.236 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.236 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.236 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.236 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.237 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.237 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.237 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.237 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.237 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.237 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.237 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.237 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.237 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.237 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.237 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.237 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.237 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.237 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.237 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.237 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.238 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.238 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.238 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.238 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.238 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.238 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.238 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.238 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.238 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.238 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.238 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.238 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.238 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.238 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.239 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.239 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.239 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.239 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.239 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.239 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.239 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.239 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.239 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.239 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.239 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.239 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.239 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.239 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.239 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.240 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.240 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.240 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.240 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.240 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.240 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.240 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.240 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.240 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.240 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.240 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.240 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.240 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.240 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.240 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.241 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.241 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.241 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "localpool", 2024-09-21T06:48:22.241 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.241 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.241 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.241 INFO:teuthology.orchestra.run.smithi067.stdout: "failure_domain": { 2024-09-21T06:48:22.241 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "failure_domain", 2024-09-21T06:48:22.241 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.241 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.241 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.241 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "host", 2024-09-21T06:48:22.241 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.242 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.242 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.242 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "failure domain for any created local pool", 2024-09-21T06:48:22.242 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T06:48:22.242 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.242 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.242 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.242 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.242 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.242 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.242 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.242 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.243 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.243 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.243 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.243 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.243 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.243 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.243 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.243 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.243 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.243 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.243 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.243 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.243 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.243 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.243 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.244 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.244 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.244 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.244 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.244 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.244 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.244 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.244 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.244 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.244 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.244 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.244 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.244 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.244 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.244 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.245 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.245 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.245 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.245 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.245 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.245 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.245 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.245 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.245 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.245 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.245 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.245 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.245 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.245 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.246 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.246 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.246 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.246 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.246 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.246 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.246 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.246 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.246 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.246 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.246 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.246 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.246 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.246 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.247 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.247 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.247 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.247 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.247 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.247 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.247 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.247 INFO:teuthology.orchestra.run.smithi067.stdout: "min_size": { 2024-09-21T06:48:22.247 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "min_size", 2024-09-21T06:48:22.247 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.247 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.247 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.247 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.247 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.248 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.248 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.248 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default min_size for any created local pool", 2024-09-21T06:48:22.248 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T06:48:22.248 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.248 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.248 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.248 INFO:teuthology.orchestra.run.smithi067.stdout: "num_rep": { 2024-09-21T06:48:22.248 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "num_rep", 2024-09-21T06:48:22.248 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.248 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.248 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.248 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3", 2024-09-21T06:48:22.248 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.248 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.249 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.249 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default replica count for any created local pool", 2024-09-21T06:48:22.249 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.249 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.249 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.249 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.249 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_num": { 2024-09-21T06:48:22.249 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pg_num", 2024-09-21T06:48:22.249 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.249 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.249 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.249 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "128", 2024-09-21T06:48:22.249 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.249 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.249 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.249 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T06:48:22.249 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.250 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.250 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.250 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.250 INFO:teuthology.orchestra.run.smithi067.stdout: "prefix": { 2024-09-21T06:48:22.250 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prefix", 2024-09-21T06:48:22.250 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.250 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.250 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.250 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.250 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.250 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.250 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.250 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "name prefix for any created local pool", 2024-09-21T06:48:22.250 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.250 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.251 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.251 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.251 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.251 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.251 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.251 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.251 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.251 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.251 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.251 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.251 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.251 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.251 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.251 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.252 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.252 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.252 INFO:teuthology.orchestra.run.smithi067.stdout: "subtree": { 2024-09-21T06:48:22.252 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subtree", 2024-09-21T06:48:22.252 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.252 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.252 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.252 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "rack", 2024-09-21T06:48:22.252 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.252 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.252 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.252 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T06:48:22.252 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T06:48:22.252 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.252 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.253 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.253 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.253 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.253 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.253 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mds_autoscaler", 2024-09-21T06:48:22.253 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.253 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.253 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.253 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.253 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.253 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.253 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.253 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.253 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.253 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.253 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.254 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.254 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.254 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.254 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.254 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.254 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.254 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.254 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.254 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.254 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.254 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.254 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.254 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.254 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.255 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.255 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.255 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.255 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.255 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.255 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.255 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.255 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.255 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.255 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.255 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.255 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.255 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.255 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.255 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.256 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.256 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.256 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.256 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.256 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.256 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.256 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.256 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.256 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.256 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.256 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.256 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.256 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.256 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.257 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.257 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.257 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.257 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.257 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.257 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.257 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.257 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.257 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.257 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.257 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.257 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.257 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.257 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.257 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.258 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.258 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.258 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.258 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.258 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.258 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.258 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.258 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.258 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.258 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.258 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.258 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.258 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.258 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.258 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.259 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.259 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.259 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.259 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.259 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.259 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.259 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mirroring", 2024-09-21T06:48:22.259 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.259 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.259 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.259 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.259 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.259 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.259 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.260 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.260 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.260 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.260 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.260 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.260 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.260 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.260 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.260 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.260 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.260 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.260 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.260 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.260 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.260 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.261 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.261 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.261 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.261 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.261 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.261 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.261 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.261 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.261 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.261 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.261 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.261 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.261 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.261 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.261 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.262 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.262 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.262 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.262 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.262 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.262 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.262 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.262 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.262 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.262 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.262 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.262 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.262 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.262 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.262 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.263 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.263 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.263 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.263 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.263 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.263 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.263 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.263 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.263 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.263 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.263 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.263 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.263 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.263 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.263 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.264 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.264 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.264 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.264 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.264 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.264 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.264 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.264 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.264 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.264 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.264 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.264 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.264 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.264 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.264 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.265 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.265 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.265 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.265 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.265 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.265 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.265 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.265 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.265 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "nfs", 2024-09-21T06:48:22.265 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.265 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.265 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.265 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.265 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.265 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.266 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.266 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.266 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.266 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.266 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.266 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.266 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.266 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.266 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.266 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.266 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.266 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.266 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.266 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.266 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.267 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.267 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.267 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.267 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.267 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.267 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.267 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.267 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.267 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.267 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.267 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.267 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.267 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.267 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.267 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.268 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.268 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.268 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.268 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.268 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.268 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.268 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.268 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.268 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.268 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.268 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.268 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.268 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.268 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.269 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.269 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.269 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.269 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.269 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.269 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.269 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.269 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.269 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.269 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.269 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.269 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.269 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.269 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.270 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.270 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.270 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.270 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.270 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.270 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.270 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.270 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.270 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.270 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.270 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.270 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.270 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.270 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.271 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.271 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.271 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.271 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.271 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.271 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.271 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.271 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.271 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.271 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.271 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.271 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.271 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "orchestrator", 2024-09-21T06:48:22.271 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.271 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.272 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.272 INFO:teuthology.orchestra.run.smithi067.stdout: "fail_fs": { 2024-09-21T06:48:22.272 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "fail_fs", 2024-09-21T06:48:22.272 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.272 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.272 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.272 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.272 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.272 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.272 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.272 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T06:48:22.272 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.272 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.272 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.272 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.273 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.273 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.273 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.273 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.273 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.273 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.273 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.273 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.273 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.273 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.273 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.273 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.273 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.273 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.274 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.274 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.274 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.274 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.274 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.274 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.274 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.274 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.274 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.274 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.274 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.274 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.274 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.274 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.274 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.275 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.275 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.275 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.275 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.275 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.275 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.275 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.275 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.275 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.275 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.275 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.275 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.275 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.275 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.276 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.276 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.276 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.276 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.276 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.276 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.276 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.276 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.276 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.276 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.276 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.276 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.276 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.276 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.276 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.277 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.277 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.277 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.277 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.277 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.277 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.277 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.277 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.277 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.277 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.277 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.277 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.277 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator": { 2024-09-21T06:48:22.277 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "orchestrator", 2024-09-21T06:48:22.278 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.278 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.278 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.278 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.278 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.278 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.278 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.278 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm", 2024-09-21T06:48:22.278 INFO:teuthology.orchestra.run.smithi067.stdout: "rook", 2024-09-21T06:48:22.278 INFO:teuthology.orchestra.run.smithi067.stdout: "test_orchestrator" 2024-09-21T06:48:22.278 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.278 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Orchestrator backend", 2024-09-21T06:48:22.278 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.278 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.278 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.279 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.279 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.279 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.279 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.279 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.279 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.279 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.279 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.279 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.279 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.279 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.279 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.279 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.279 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.279 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.280 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.280 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.280 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.280 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "osd_perf_query", 2024-09-21T06:48:22.280 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.280 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.280 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.280 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.280 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.280 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.280 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.280 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.280 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.280 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.280 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.281 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.281 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.281 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.281 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.281 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.281 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.281 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.281 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.281 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.281 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.281 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.281 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.281 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.281 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.281 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.282 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.282 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.282 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.282 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.282 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.282 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.282 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.282 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.282 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.282 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.282 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.282 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.282 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.282 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.283 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.283 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.283 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.283 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.283 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.283 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.283 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.283 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.283 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.283 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.283 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.283 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.283 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.283 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.283 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.284 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.284 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.284 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.284 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.284 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.284 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.284 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.284 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.284 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.284 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.284 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.284 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.284 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.284 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.285 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.285 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.285 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.285 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.285 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.285 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.285 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.285 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.285 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.285 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.285 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.285 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.285 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.285 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.286 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.286 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.286 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.286 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.286 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.286 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.286 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.286 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "osd_support", 2024-09-21T06:48:22.286 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.286 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.286 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.286 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.286 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.286 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.287 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.287 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.287 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.287 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.287 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.287 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.287 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.287 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.287 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.287 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.287 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.287 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.287 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.287 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.288 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.288 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.288 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.288 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.288 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.288 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.288 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.288 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.288 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.288 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.288 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.288 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.288 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.288 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.288 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.289 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.289 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.289 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.289 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.289 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.289 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.289 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.289 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.289 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.289 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.289 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.289 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.289 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.289 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.289 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.290 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.290 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.290 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.290 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.290 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.290 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.290 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.290 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.290 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.290 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.290 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.290 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.290 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.290 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.290 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.291 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.291 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.291 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.291 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.291 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.291 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.291 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.291 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.291 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.291 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.291 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.291 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.291 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.291 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.292 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.292 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.292 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.292 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.292 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.292 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.292 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.292 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.292 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.292 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.292 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.292 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pg_autoscaler", 2024-09-21T06:48:22.292 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.292 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.293 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.293 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.293 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.293 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.293 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.293 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.293 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.293 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.293 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.293 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.293 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.293 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.293 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.293 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.294 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.294 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.294 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.294 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.294 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.294 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.294 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.294 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.294 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.294 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.294 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.294 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.294 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.294 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.294 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.295 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.295 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.295 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.295 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.295 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.295 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.295 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.295 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.295 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.295 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.295 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.295 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.295 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.295 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.296 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.296 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.296 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.296 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.296 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.296 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.296 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.296 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.296 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.296 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.296 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.296 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.296 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.296 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.297 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.297 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.297 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.297 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.297 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.297 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.297 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.297 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.297 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.297 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.297 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.297 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.297 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.297 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.298 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-09-21T06:48:22.298 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-09-21T06:48:22.298 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.298 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.298 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.298 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-09-21T06:48:22.298 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.298 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.298 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.298 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.298 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.298 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.298 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.298 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.298 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.299 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.299 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.299 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.299 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.299 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.299 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.299 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.299 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.299 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.299 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.299 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.299 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.299 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.299 INFO:teuthology.orchestra.run.smithi067.stdout: "threshold": { 2024-09-21T06:48:22.300 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "threshold", 2024-09-21T06:48:22.300 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-21T06:48:22.300 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.300 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.300 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3.0", 2024-09-21T06:48:22.300 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1.0", 2024-09-21T06:48:22.300 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.300 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.300 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "scaling threshold", 2024-09-21T06:48:22.300 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T06:48:22.300 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.300 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.300 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.301 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.301 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.301 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.301 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "progress", 2024-09-21T06:48:22.301 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.301 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.301 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.301 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_pg_recovery_event": { 2024-09-21T06:48:22.301 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_pg_recovery_event", 2024-09-21T06:48:22.301 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.301 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.301 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.301 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.301 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.301 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.302 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.302 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T06:48:22.302 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.302 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.302 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.302 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.302 INFO:teuthology.orchestra.run.smithi067.stdout: "enabled": { 2024-09-21T06:48:22.302 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enabled", 2024-09-21T06:48:22.302 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.302 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.302 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.302 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.302 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.302 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.302 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.303 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.303 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.303 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.303 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.303 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.303 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.303 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.303 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.303 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.303 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.303 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.303 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.303 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.303 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.304 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.304 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.304 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.304 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.304 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.304 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.304 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.304 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.304 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.304 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.304 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.304 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.304 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.304 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.304 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.305 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.305 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.305 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.305 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.305 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.305 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.305 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.305 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.305 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.305 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.305 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.305 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.305 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.305 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.306 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.306 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.306 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.306 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.306 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.306 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.306 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.306 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.306 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.306 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.306 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.306 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.306 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.306 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.306 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.307 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.307 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.307 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.307 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.307 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.307 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.307 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.307 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.307 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.307 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.307 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.307 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.307 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.307 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.307 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.308 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.308 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.308 INFO:teuthology.orchestra.run.smithi067.stdout: "max_completed_events": { 2024-09-21T06:48:22.308 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_completed_events", 2024-09-21T06:48:22.308 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.308 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.308 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.308 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "50", 2024-09-21T06:48:22.308 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.308 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.308 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.308 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "number of past completed events to remember", 2024-09-21T06:48:22.308 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.308 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.309 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.309 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.309 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-09-21T06:48:22.309 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-09-21T06:48:22.309 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.309 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.309 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.309 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-09-21T06:48:22.309 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.309 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.309 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.309 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long the module is going to sleep", 2024-09-21T06:48:22.309 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.309 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.309 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.310 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.310 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.310 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.310 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.310 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.310 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.310 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.310 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.310 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.310 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.310 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.310 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.311 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.311 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.311 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.311 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.311 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.311 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.311 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prometheus", 2024-09-21T06:48:22.311 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.311 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.311 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.311 INFO:teuthology.orchestra.run.smithi067.stdout: "cache": { 2024-09-21T06:48:22.311 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cache", 2024-09-21T06:48:22.311 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.311 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.311 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.311 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.312 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.312 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.312 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.312 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.312 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.312 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.312 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.312 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.312 INFO:teuthology.orchestra.run.smithi067.stdout: "exclude_perf_counters": { 2024-09-21T06:48:22.312 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "exclude_perf_counters", 2024-09-21T06:48:22.312 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.312 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.312 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.312 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.312 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.313 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.313 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.313 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T06:48:22.313 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T06:48:22.313 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.313 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.313 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.313 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.313 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.313 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.313 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.313 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.313 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.313 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.313 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.314 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.314 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.314 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.314 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.314 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.314 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.314 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.314 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.314 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.314 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.314 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.314 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.314 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.315 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.315 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.315 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.315 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.315 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.315 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.315 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.315 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.315 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.315 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.315 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.315 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.315 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.315 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.316 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.316 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.316 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.316 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.316 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.316 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.316 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.316 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.316 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.316 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.316 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.316 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.316 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.317 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.317 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.317 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.317 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.317 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.317 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.317 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.317 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.317 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.317 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.317 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.317 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.317 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.317 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.317 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.318 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.318 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.318 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.318 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.318 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.318 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.318 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.318 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_stats_pools": { 2024-09-21T06:48:22.318 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_stats_pools", 2024-09-21T06:48:22.318 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.318 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.318 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.318 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.318 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.319 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.319 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.319 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.319 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.319 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.319 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.319 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.319 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T06:48:22.319 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T06:48:22.319 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.319 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.319 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.319 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "300", 2024-09-21T06:48:22.319 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.320 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.320 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.320 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.320 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.320 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.320 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.320 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.320 INFO:teuthology.orchestra.run.smithi067.stdout: "scrape_interval": { 2024-09-21T06:48:22.320 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "scrape_interval", 2024-09-21T06:48:22.320 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-21T06:48:22.320 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.320 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.320 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15.0", 2024-09-21T06:48:22.320 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.320 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.320 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.321 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.321 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.321 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.321 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.321 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.321 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-09-21T06:48:22.321 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-09-21T06:48:22.321 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.321 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.321 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.321 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "::", 2024-09-21T06:48:22.321 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.321 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.321 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.322 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T06:48:22.322 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.322 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.322 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.322 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.322 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-09-21T06:48:22.322 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-09-21T06:48:22.322 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.322 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.322 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.322 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "9283", 2024-09-21T06:48:22.322 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.322 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.322 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.323 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T06:48:22.323 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.323 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.323 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.323 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.323 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.323 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.323 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.323 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.323 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.323 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.323 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.323 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.323 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.323 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.324 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.324 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.324 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.324 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.324 INFO:teuthology.orchestra.run.smithi067.stdout: "stale_cache_strategy": { 2024-09-21T06:48:22.324 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "stale_cache_strategy", 2024-09-21T06:48:22.324 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.324 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.324 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.324 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "log", 2024-09-21T06:48:22.324 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.324 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.324 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.324 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.325 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.325 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.325 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.325 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.325 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_behaviour": { 2024-09-21T06:48:22.325 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_behaviour", 2024-09-21T06:48:22.325 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.325 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.325 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.325 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "default", 2024-09-21T06:48:22.325 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.325 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.325 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.325 INFO:teuthology.orchestra.run.smithi067.stdout: "default", 2024-09-21T06:48:22.326 INFO:teuthology.orchestra.run.smithi067.stdout: "error" 2024-09-21T06:48:22.326 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.326 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.326 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.326 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.326 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.326 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.326 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_error_status_code": { 2024-09-21T06:48:22.326 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_error_status_code", 2024-09-21T06:48:22.326 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.326 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.326 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.326 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-09-21T06:48:22.326 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "400", 2024-09-21T06:48:22.327 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "599", 2024-09-21T06:48:22.327 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.327 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.327 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.327 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.327 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.327 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.327 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.327 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.327 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.327 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_support", 2024-09-21T06:48:22.327 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.327 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.327 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.328 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.328 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.328 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.328 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.328 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.328 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.328 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.328 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.328 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.328 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.328 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.328 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.328 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.328 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.329 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.329 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.329 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.329 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.329 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.329 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.329 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.329 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.329 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.329 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.329 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.329 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.329 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.329 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.330 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.330 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.330 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.330 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.330 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.330 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.330 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.330 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.330 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.330 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.330 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.330 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.330 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.330 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.330 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.331 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.331 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.331 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.331 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.331 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.331 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.331 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.331 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.331 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.331 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.331 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.331 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.331 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.331 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.332 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.332 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.332 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.332 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.332 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.332 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.332 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.332 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.332 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.332 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.332 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.332 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.332 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.332 INFO:teuthology.orchestra.run.smithi067.stdout: "max_concurrent_snap_create": { 2024-09-21T06:48:22.332 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_concurrent_snap_create", 2024-09-21T06:48:22.333 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.333 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.333 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.333 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-21T06:48:22.333 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.333 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.333 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.333 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.333 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.333 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.333 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.333 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.333 INFO:teuthology.orchestra.run.smithi067.stdout: "mirror_snapshot_schedule": { 2024-09-21T06:48:22.333 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T06:48:22.334 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.334 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.334 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.334 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.334 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.334 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.334 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.334 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.334 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.334 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.334 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.334 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.334 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.334 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.335 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.335 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.335 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.335 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.335 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.335 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.335 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.335 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.335 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.335 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.335 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.335 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.335 INFO:teuthology.orchestra.run.smithi067.stdout: "trash_purge_schedule": { 2024-09-21T06:48:22.335 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "trash_purge_schedule", 2024-09-21T06:48:22.336 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.336 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.336 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.336 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.336 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.336 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.336 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.336 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.336 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.336 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.336 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.336 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.336 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.336 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.337 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.337 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "restful", 2024-09-21T06:48:22.337 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.337 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.337 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.337 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_auth": { 2024-09-21T06:48:22.337 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enable_auth", 2024-09-21T06:48:22.337 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.337 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.337 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.337 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.337 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.337 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.337 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.337 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.338 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.338 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.338 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.338 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.338 INFO:teuthology.orchestra.run.smithi067.stdout: "key_file": { 2024-09-21T06:48:22.338 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "key_file", 2024-09-21T06:48:22.338 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.338 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.338 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.338 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.338 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.338 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.338 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.338 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.339 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.339 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.339 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.339 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.339 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.339 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.339 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.339 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.339 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.339 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.339 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.339 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.339 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.339 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.339 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.340 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.340 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.340 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.340 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.340 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.340 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.340 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.340 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.340 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.340 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.340 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.340 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.340 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.341 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.341 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.341 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.341 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.341 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.341 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.341 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.341 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.341 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.341 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.341 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.341 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.341 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.341 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.341 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.342 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.342 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.342 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.342 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.342 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.342 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.342 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.342 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.342 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.342 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.342 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.342 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.342 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.343 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.343 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.343 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.343 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.343 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.343 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.343 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.343 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.343 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.343 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.343 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.343 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.343 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.343 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.344 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.344 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.344 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.344 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.344 INFO:teuthology.orchestra.run.smithi067.stdout: "max_requests": { 2024-09-21T06:48:22.344 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_requests", 2024-09-21T06:48:22.344 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.344 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.344 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.344 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-09-21T06:48:22.344 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.344 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.344 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.344 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T06:48:22.345 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.345 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.345 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.345 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.345 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-09-21T06:48:22.345 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-09-21T06:48:22.345 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.345 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.345 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.345 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.345 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.345 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.345 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.345 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.346 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.346 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.346 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.346 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.346 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-09-21T06:48:22.346 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-09-21T06:48:22.346 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.346 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.346 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.346 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.346 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.346 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.346 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.346 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.347 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.347 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.347 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.347 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.347 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.347 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.347 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.347 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.347 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.347 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.347 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.347 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.347 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.347 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.347 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.348 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.348 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.348 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.348 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.348 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.348 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.348 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rgw", 2024-09-21T06:48:22.348 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.348 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.348 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.348 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.348 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.348 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.348 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.349 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.349 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.349 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.349 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.349 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.349 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.349 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.349 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.349 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.349 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.349 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.349 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.349 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.349 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.349 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.349 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.349 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.350 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.350 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.350 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.350 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.350 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.350 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.350 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.350 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.350 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.350 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.350 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.350 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.350 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.350 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.350 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.351 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.351 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.351 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.351 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.351 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.351 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.351 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.351 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.351 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.351 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.351 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.351 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.351 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.351 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.352 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.352 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.352 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.352 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.352 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.352 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.352 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.352 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.352 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.352 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.352 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.352 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.352 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.352 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.353 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.353 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.353 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.353 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.353 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.353 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.353 INFO:teuthology.orchestra.run.smithi067.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T06:48:22.353 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T06:48:22.353 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.353 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.353 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.353 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-09-21T06:48:22.353 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.354 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.354 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.354 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T06:48:22.354 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.354 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.354 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.354 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.354 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.354 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.354 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.354 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.354 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.354 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.354 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.354 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.355 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.355 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.355 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.355 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.355 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.355 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.355 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.355 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.355 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.355 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rook", 2024-09-21T06:48:22.355 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.355 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.355 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.355 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.356 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.356 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.356 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.356 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.356 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.356 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.356 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.356 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.356 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.356 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.356 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.356 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.356 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.356 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.356 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.356 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.357 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.357 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.357 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.357 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.357 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.357 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.357 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.357 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.357 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.357 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.357 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.357 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.357 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.357 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.357 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.358 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.358 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.358 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.358 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.358 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.358 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.358 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.358 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.358 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.358 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.358 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.358 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.358 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.358 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.359 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.359 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.359 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.359 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.359 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.359 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.359 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.359 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.359 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.359 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.359 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.359 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.359 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.359 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.359 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.359 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.360 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.360 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.360 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.360 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.360 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.360 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.360 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.360 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.360 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.360 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.360 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.360 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.360 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.360 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.360 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.360 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.360 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.361 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.361 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.361 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.361 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.361 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.361 INFO:teuthology.orchestra.run.smithi067.stdout: "storage_class": { 2024-09-21T06:48:22.361 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "storage_class", 2024-09-21T06:48:22.361 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.361 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.361 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.361 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "local", 2024-09-21T06:48:22.361 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.361 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.361 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.361 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T06:48:22.361 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.362 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.362 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.362 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.362 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.362 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.362 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.362 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "selftest", 2024-09-21T06:48:22.362 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.362 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.362 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.362 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.362 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.362 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.362 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.362 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.363 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.363 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.363 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.363 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.363 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.363 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.363 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.363 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.363 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.363 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.363 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.363 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.363 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.363 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.363 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.363 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.364 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.364 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.364 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.364 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.364 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.364 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.364 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.364 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.364 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.364 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.364 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.364 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.364 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.364 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.364 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.364 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.365 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.365 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.365 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.365 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.365 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.365 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.365 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.365 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.365 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.365 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.365 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.365 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.365 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.365 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.365 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.366 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.366 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.366 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.366 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.366 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.366 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.366 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.366 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.366 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.366 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.366 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.366 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.366 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.366 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.366 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.367 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.367 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.367 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.367 INFO:teuthology.orchestra.run.smithi067.stdout: "roption1": { 2024-09-21T06:48:22.367 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "roption1", 2024-09-21T06:48:22.367 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.367 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.367 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.367 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.367 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.367 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.367 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.367 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.368 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.368 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.368 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.368 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.368 INFO:teuthology.orchestra.run.smithi067.stdout: "roption2": { 2024-09-21T06:48:22.368 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "roption2", 2024-09-21T06:48:22.368 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.368 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.368 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.368 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "xyz", 2024-09-21T06:48:22.368 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.368 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.368 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.368 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.368 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.369 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.369 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.369 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.369 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption1": { 2024-09-21T06:48:22.369 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption1", 2024-09-21T06:48:22.369 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.369 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.369 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.369 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.369 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.369 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.369 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.369 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.369 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.369 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.369 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.370 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.370 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption2": { 2024-09-21T06:48:22.370 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption2", 2024-09-21T06:48:22.370 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.370 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.370 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.370 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.370 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.370 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.370 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.370 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.370 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.370 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.370 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.370 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.371 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption3": { 2024-09-21T06:48:22.371 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption3", 2024-09-21T06:48:22.371 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-21T06:48:22.371 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.371 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.371 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.371 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.371 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.371 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.371 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.371 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.371 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.371 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.371 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.372 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption4": { 2024-09-21T06:48:22.372 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption4", 2024-09-21T06:48:22.372 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.372 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.372 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.372 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.372 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.372 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.372 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.372 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.372 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.372 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.372 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.372 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.372 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption5": { 2024-09-21T06:48:22.372 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption5", 2024-09-21T06:48:22.372 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.373 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.373 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.373 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.373 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.373 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.373 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.373 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.373 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.373 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.373 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.373 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.373 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption6": { 2024-09-21T06:48:22.373 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption6", 2024-09-21T06:48:22.373 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.373 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.373 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.374 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.374 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.374 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.374 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.374 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.374 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.374 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.374 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.374 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.374 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption7": { 2024-09-21T06:48:22.374 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption7", 2024-09-21T06:48:22.374 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.374 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.374 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.374 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.375 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-09-21T06:48:22.375 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "42", 2024-09-21T06:48:22.375 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.375 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.375 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.375 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.375 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.375 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.375 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.375 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.375 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.375 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.375 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.375 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.375 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.376 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.376 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.376 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.376 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.376 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.376 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.376 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.376 INFO:teuthology.orchestra.run.smithi067.stdout: "testkey": { 2024-09-21T06:48:22.376 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testkey", 2024-09-21T06:48:22.376 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.376 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.376 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.376 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.376 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.377 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.377 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.377 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.377 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.377 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.377 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.377 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.377 INFO:teuthology.orchestra.run.smithi067.stdout: "testlkey": { 2024-09-21T06:48:22.377 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testlkey", 2024-09-21T06:48:22.377 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.377 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.377 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.377 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.377 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.377 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.378 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.378 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.378 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.378 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.378 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.378 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.378 INFO:teuthology.orchestra.run.smithi067.stdout: "testnewline": { 2024-09-21T06:48:22.378 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testnewline", 2024-09-21T06:48:22.378 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.378 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.378 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.378 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.378 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.379 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.379 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.379 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.379 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.379 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.379 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.379 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.379 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.379 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.379 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.379 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smb", 2024-09-21T06:48:22.379 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.379 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.380 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.380 INFO:teuthology.orchestra.run.smithi067.stdout: "internal_store_backend": { 2024-09-21T06:48:22.380 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "internal_store_backend", 2024-09-21T06:48:22.380 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.380 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.380 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.380 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.380 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.380 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.380 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.380 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T06:48:22.380 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.380 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.380 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.380 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.381 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.381 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.381 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.381 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.381 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.381 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.381 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.381 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.381 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.381 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.381 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.381 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.381 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.381 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.381 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.381 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.382 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.382 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.382 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.382 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.382 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.382 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.382 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.382 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.382 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.382 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.382 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.382 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.382 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.382 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.382 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.383 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.383 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.383 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.383 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.383 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.383 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.383 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.383 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.383 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.383 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.383 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.383 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.383 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.383 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.384 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.384 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.384 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.384 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.384 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.384 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.384 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.384 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.384 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.384 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.384 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.384 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.384 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.384 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.384 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.385 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.385 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.385 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.385 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.385 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.385 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.385 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.385 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.385 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.385 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.385 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.385 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.385 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.385 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.386 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.386 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.386 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.386 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.386 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.386 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.386 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.386 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.386 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.386 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.386 INFO:teuthology.orchestra.run.smithi067.stdout: "update_orchestration": { 2024-09-21T06:48:22.386 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "update_orchestration", 2024-09-21T06:48:22.386 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.386 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.386 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.387 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.387 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.387 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.387 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.387 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T06:48:22.387 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.387 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.387 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.387 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.387 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.387 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.387 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.387 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snap_schedule", 2024-09-21T06:48:22.388 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.388 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.388 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.388 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_m_granularity": { 2024-09-21T06:48:22.388 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_m_granularity", 2024-09-21T06:48:22.388 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.388 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.388 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.388 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.388 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.388 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.388 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.388 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T06:48:22.388 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.388 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.388 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.389 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.389 INFO:teuthology.orchestra.run.smithi067.stdout: "dump_on_update": { 2024-09-21T06:48:22.389 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "dump_on_update", 2024-09-21T06:48:22.389 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.389 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.389 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.389 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.389 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.389 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.389 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.389 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "dump database to debug log on update", 2024-09-21T06:48:22.389 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.389 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.389 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.389 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.390 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.390 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.390 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.390 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.390 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.390 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.390 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.390 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.390 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.390 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.390 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.390 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.390 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.390 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.390 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.391 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.391 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.391 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.391 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.391 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.391 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.391 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.391 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.391 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.391 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.391 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.391 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.391 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.391 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.391 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.392 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.392 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.392 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.392 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.392 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.392 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.392 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.392 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.392 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.392 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.392 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.392 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.392 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.392 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.392 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.392 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.393 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.393 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.393 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.393 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.393 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.393 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.393 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.393 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.393 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.393 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.393 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.393 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.393 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.393 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.393 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.393 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.394 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.394 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.394 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.394 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.394 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.394 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.394 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.394 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.394 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.394 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.394 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.394 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.394 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.394 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.394 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.394 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.395 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.395 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.395 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.395 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.395 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.395 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.395 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.395 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.395 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.395 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "stats", 2024-09-21T06:48:22.395 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.395 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.395 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.395 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.396 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.396 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.396 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.396 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.396 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.396 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.396 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.396 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.396 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.396 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.396 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.396 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.397 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.397 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.397 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.397 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.397 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.397 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.397 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.397 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.397 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.397 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.397 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.397 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.397 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.398 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.398 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.398 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.398 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.398 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.398 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.398 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.398 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.398 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.398 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.398 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.398 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.398 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.398 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.398 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.399 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.399 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.399 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.399 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.399 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.399 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.399 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.399 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.399 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.399 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.399 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.399 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.399 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.399 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.400 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.400 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.400 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.400 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.400 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.400 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.400 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.400 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.400 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.400 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.400 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.400 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.400 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.400 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.400 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.400 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.401 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.401 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.401 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.401 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.401 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.401 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.401 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.401 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.401 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.401 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.401 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.401 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.401 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.401 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.401 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.401 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.402 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "status", 2024-09-21T06:48:22.402 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.402 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.402 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.402 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.402 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.402 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.402 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.402 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.402 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.402 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.402 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.402 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.402 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.402 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.402 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.403 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.403 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.403 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.403 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.403 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.403 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.403 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.403 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.403 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.403 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.403 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.403 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.403 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.403 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.403 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.403 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.404 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.404 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.404 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.404 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.404 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.404 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.404 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.404 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.404 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.404 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.404 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.404 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.404 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.404 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.405 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.405 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.405 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.405 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.405 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.405 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.405 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.405 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.405 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.405 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.405 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.405 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.405 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.405 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.406 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.406 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.406 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.406 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.406 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.406 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.406 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.406 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.406 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.406 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.406 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.406 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.406 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.406 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.406 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.407 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.407 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.407 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.407 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.407 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.407 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.407 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.407 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.407 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.407 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.407 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.407 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.407 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.407 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.408 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.408 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.408 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "telegraf", 2024-09-21T06:48:22.408 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.408 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.408 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.408 INFO:teuthology.orchestra.run.smithi067.stdout: "address": { 2024-09-21T06:48:22.408 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "address", 2024-09-21T06:48:22.408 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.408 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.408 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.408 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T06:48:22.408 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.408 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.408 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.409 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.409 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.409 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.409 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.409 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.409 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-09-21T06:48:22.409 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-09-21T06:48:22.409 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.409 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.409 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.409 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15", 2024-09-21T06:48:22.409 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.409 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.409 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.409 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.410 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.410 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.410 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.410 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.410 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.410 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.410 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.410 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.410 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.410 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.410 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.410 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.410 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.410 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.411 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.411 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.411 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.411 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.411 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.411 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.411 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.411 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.411 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.411 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.411 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.411 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.411 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.411 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.412 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.412 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.412 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.412 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.412 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.412 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.412 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.412 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.412 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.412 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.413 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.414 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.414 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.414 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.414 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.414 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.414 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.414 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.414 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.414 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.414 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.414 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.414 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.414 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.415 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.415 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.415 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.415 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.415 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.415 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.415 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.415 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.415 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.415 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.415 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.415 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.415 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.415 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.415 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.416 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.416 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.416 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.416 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.416 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.416 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.416 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.416 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.416 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.416 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.416 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.416 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.416 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.416 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.417 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.417 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.417 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.417 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.417 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.417 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.417 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.417 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.417 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.417 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.417 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "telemetry", 2024-09-21T06:48:22.417 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.417 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.417 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.417 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_basic": { 2024-09-21T06:48:22.418 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_basic", 2024-09-21T06:48:22.418 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.418 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.418 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.418 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.418 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.418 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.418 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.418 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T06:48:22.418 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.418 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.418 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.419 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.419 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_crash": { 2024-09-21T06:48:22.419 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_crash", 2024-09-21T06:48:22.419 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.419 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.419 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.419 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.419 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.419 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.419 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.419 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T06:48:22.419 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.419 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.419 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.419 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.420 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_device": { 2024-09-21T06:48:22.420 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_device", 2024-09-21T06:48:22.420 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.420 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.420 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.420 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.420 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.420 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.420 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.420 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T06:48:22.420 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.420 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.420 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.420 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.421 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_ident": { 2024-09-21T06:48:22.421 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_ident", 2024-09-21T06:48:22.421 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.421 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.421 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.421 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.421 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.421 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.421 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.421 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T06:48:22.421 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.421 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.421 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.422 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.422 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_perf": { 2024-09-21T06:48:22.422 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_perf", 2024-09-21T06:48:22.422 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.422 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.422 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.422 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.422 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.422 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.422 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.422 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T06:48:22.422 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.422 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.422 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.422 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.423 INFO:teuthology.orchestra.run.smithi067.stdout: "contact": { 2024-09-21T06:48:22.423 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "contact", 2024-09-21T06:48:22.423 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.423 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.423 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.423 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.423 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.423 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.423 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.423 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.423 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.423 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.423 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.423 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.424 INFO:teuthology.orchestra.run.smithi067.stdout: "description": { 2024-09-21T06:48:22.424 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "description", 2024-09-21T06:48:22.424 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.424 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.424 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.424 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.424 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.424 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.424 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.424 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.424 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.424 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.424 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.424 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.425 INFO:teuthology.orchestra.run.smithi067.stdout: "device_url": { 2024-09-21T06:48:22.425 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_url", 2024-09-21T06:48:22.425 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.425 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.425 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.425 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T06:48:22.425 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.425 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.425 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.425 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.425 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.425 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.425 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.425 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.425 INFO:teuthology.orchestra.run.smithi067.stdout: "enabled": { 2024-09-21T06:48:22.426 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enabled", 2024-09-21T06:48:22.426 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.426 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.426 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.426 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.426 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.426 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.426 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.426 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.426 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.426 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.426 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.426 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.427 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-09-21T06:48:22.427 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-09-21T06:48:22.427 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.427 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.427 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.427 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "24", 2024-09-21T06:48:22.427 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "8", 2024-09-21T06:48:22.427 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.427 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.427 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.427 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.427 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.427 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.427 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.427 INFO:teuthology.orchestra.run.smithi067.stdout: "last_opt_revision": { 2024-09-21T06:48:22.428 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "last_opt_revision", 2024-09-21T06:48:22.428 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.428 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.428 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.428 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1", 2024-09-21T06:48:22.428 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.428 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.428 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.428 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.428 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.428 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.428 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.428 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.428 INFO:teuthology.orchestra.run.smithi067.stdout: "leaderboard": { 2024-09-21T06:48:22.429 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "leaderboard", 2024-09-21T06:48:22.429 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.429 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.429 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.429 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.429 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.429 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.429 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.429 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.429 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.429 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.429 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.429 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.429 INFO:teuthology.orchestra.run.smithi067.stdout: "leaderboard_description": { 2024-09-21T06:48:22.429 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "leaderboard_description", 2024-09-21T06:48:22.430 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.430 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.430 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.430 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.430 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.430 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.430 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.430 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.430 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.430 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.430 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.430 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.430 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.430 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.431 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.431 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.431 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.431 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.431 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.431 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.431 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.431 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.431 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.431 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.431 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.431 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.431 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.431 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.432 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.432 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.432 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.432 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.432 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.432 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.432 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.432 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.432 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.432 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.432 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.432 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.432 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.432 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.433 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.433 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.433 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.433 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.433 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.433 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.433 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.433 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.433 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.433 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.433 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.433 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.433 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.433 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.433 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.434 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.434 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.434 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.434 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.434 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.434 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.434 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.434 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.434 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.434 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.434 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.434 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.434 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.434 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.435 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.435 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.435 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.435 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.435 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.435 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.435 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.435 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.435 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.435 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.435 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.435 INFO:teuthology.orchestra.run.smithi067.stdout: "organization": { 2024-09-21T06:48:22.435 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "organization", 2024-09-21T06:48:22.435 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.435 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.436 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.436 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.436 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.436 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.436 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.436 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.436 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.436 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.436 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.436 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.436 INFO:teuthology.orchestra.run.smithi067.stdout: "proxy": { 2024-09-21T06:48:22.436 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "proxy", 2024-09-21T06:48:22.436 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.436 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.436 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.437 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.437 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.437 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.437 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.437 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.437 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.437 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.437 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.437 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.437 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.437 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.437 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.437 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.437 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.438 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.438 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.438 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.438 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.438 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.438 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.438 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.438 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.438 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.438 INFO:teuthology.orchestra.run.smithi067.stdout: "url": { 2024-09-21T06:48:22.438 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "url", 2024-09-21T06:48:22.438 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.438 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.438 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.439 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T06:48:22.439 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.439 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.439 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.439 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.439 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.439 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.439 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.439 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.439 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.439 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.439 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.439 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "test_orchestrator", 2024-09-21T06:48:22.439 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.440 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.440 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.440 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.440 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.440 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.440 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.440 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.440 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.440 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.440 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.440 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.440 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.440 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.440 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.441 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.441 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.441 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.441 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.441 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.441 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.441 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.441 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.441 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.441 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.441 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.441 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.441 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.441 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.442 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.442 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.442 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.442 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.442 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.442 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.442 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.442 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.442 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.442 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.442 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.442 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.442 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.443 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.443 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.443 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.443 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.443 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.443 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.443 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.443 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.443 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.443 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.443 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.443 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.443 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.444 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.444 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.444 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.444 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.444 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.444 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.444 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.444 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.444 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.444 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.444 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.444 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.444 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.444 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.444 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.445 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.445 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.445 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.445 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.445 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.445 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.445 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.445 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.445 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.445 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.445 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.445 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.445 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.446 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.446 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.446 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.446 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.446 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.446 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.446 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "volumes", 2024-09-21T06:48:22.446 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.446 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.446 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.446 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.446 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.446 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.446 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.447 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.447 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.447 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.447 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.447 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.447 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.447 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.447 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.447 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.447 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.447 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.447 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.447 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.447 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.448 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.448 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.448 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.448 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.448 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.448 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.448 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.448 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.448 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.448 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.448 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.448 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.448 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.448 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.449 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.449 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.449 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.449 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.449 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.449 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.449 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.449 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.449 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.449 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.449 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.449 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.449 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.449 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.450 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.450 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.450 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.450 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.450 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.450 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.450 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.450 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.450 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.450 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.450 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.450 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.450 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.450 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.451 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.451 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.451 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.451 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.451 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.451 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.451 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.451 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.451 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.451 INFO:teuthology.orchestra.run.smithi067.stdout: "max_concurrent_clones": { 2024-09-21T06:48:22.451 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_concurrent_clones", 2024-09-21T06:48:22.451 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.451 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.451 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.451 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "4", 2024-09-21T06:48:22.452 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.452 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.452 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.452 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T06:48:22.452 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.452 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.452 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.452 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.452 INFO:teuthology.orchestra.run.smithi067.stdout: "periodic_async_work": { 2024-09-21T06:48:22.452 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "periodic_async_work", 2024-09-21T06:48:22.452 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.452 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.452 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.452 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.452 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.453 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.453 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.453 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Periodically check for async work", 2024-09-21T06:48:22.453 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.453 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.453 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.453 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.453 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot_clone_delay": { 2024-09-21T06:48:22.453 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snapshot_clone_delay", 2024-09-21T06:48:22.453 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.453 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.453 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.453 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.453 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.453 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.454 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.454 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T06:48:22.454 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.454 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.454 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.454 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.454 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot_clone_no_wait": { 2024-09-21T06:48:22.454 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T06:48:22.454 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.454 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.454 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.454 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.454 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.454 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.455 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.455 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T06:48:22.455 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.455 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.455 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.455 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.455 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.455 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.455 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.455 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.455 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.455 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.455 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.455 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.456 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.456 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.456 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.456 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.456 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.456 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.456 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.456 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.456 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix", 2024-09-21T06:48:22.456 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.456 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.456 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.457 INFO:teuthology.orchestra.run.smithi067.stdout: "discovery_interval": { 2024-09-21T06:48:22.457 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "discovery_interval", 2024-09-21T06:48:22.457 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-09-21T06:48:22.457 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.457 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.457 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "100", 2024-09-21T06:48:22.457 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.457 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.457 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.457 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.457 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.457 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.457 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.457 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.458 INFO:teuthology.orchestra.run.smithi067.stdout: "identifier": { 2024-09-21T06:48:22.458 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "identifier", 2024-09-21T06:48:22.458 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.458 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.458 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.458 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.458 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.458 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.458 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.458 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.458 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.458 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.458 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.458 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.459 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-09-21T06:48:22.459 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-09-21T06:48:22.459 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.459 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.459 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.459 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-09-21T06:48:22.459 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.459 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.459 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.459 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.459 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.459 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.459 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.459 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.459 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.460 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.460 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.460 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.460 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.460 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.460 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.460 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.460 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.460 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.460 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.460 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.460 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.460 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.460 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.461 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.461 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.461 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.461 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.461 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.461 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.461 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.461 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.461 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.461 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.461 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.461 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.461 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.461 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.461 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.462 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.462 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.462 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.462 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.462 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.462 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.462 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.462 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.462 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.462 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.462 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.462 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.462 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.462 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.463 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.463 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.463 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.463 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.463 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.463 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.463 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.463 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.463 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.463 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.463 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.463 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.463 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.463 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.463 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.464 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.464 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.464 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.464 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.464 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.464 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.464 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.464 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.464 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.464 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.464 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.464 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.464 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.464 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.464 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.465 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.465 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.465 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.465 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.465 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.465 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.465 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.465 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.465 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.465 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.465 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_host": { 2024-09-21T06:48:22.465 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_host", 2024-09-21T06:48:22.465 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.465 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.465 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.466 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.466 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.466 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.466 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.466 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.466 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.466 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.466 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.466 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.466 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_port": { 2024-09-21T06:48:22.466 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_port", 2024-09-21T06:48:22.466 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.466 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.466 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.467 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10051", 2024-09-21T06:48:22.467 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.467 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.467 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.467 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.467 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.467 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.467 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.467 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.467 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_sender": { 2024-09-21T06:48:22.467 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_sender", 2024-09-21T06:48:22.467 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.467 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.467 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.467 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T06:48:22.467 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.468 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.468 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.468 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.468 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.468 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.468 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.468 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.468 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.468 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.468 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-21T06:48:22.468 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.468 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.468 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-09-21T06:48:22.468 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm", 2024-09-21T06:48:22.468 INFO:teuthology.orchestra.run.smithi067.stdout: "dashboard", 2024-09-21T06:48:22.469 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-09-21T06:48:22.469 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-09-21T06:48:22.469 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-09-21T06:48:22.469 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.469 INFO:teuthology.orchestra.run.smithi067.stdout: "available_modules": [ 2024-09-21T06:48:22.469 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.469 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "alerts", 2024-09-21T06:48:22.469 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.469 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.469 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.469 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-09-21T06:48:22.469 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-09-21T06:48:22.469 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.469 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.469 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.469 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-09-21T06:48:22.470 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.470 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.470 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.470 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T06:48:22.470 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.470 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.470 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.470 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.470 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.470 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.470 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.470 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.471 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.471 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.471 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.471 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.471 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.471 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.471 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.471 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.471 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.471 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.471 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.471 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.471 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.471 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.471 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.472 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.472 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.472 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.472 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.472 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.472 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.472 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.472 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.472 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.472 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.472 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.472 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.473 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.473 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.473 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.473 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.473 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.473 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.473 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.473 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.473 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.473 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.473 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.473 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.473 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.473 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.474 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.474 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.474 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.474 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.474 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.474 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.474 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.474 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.474 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.474 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.474 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.474 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.474 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.474 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.475 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.475 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.475 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.475 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.475 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.475 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.475 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.475 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.475 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.475 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.475 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_destination": { 2024-09-21T06:48:22.475 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_destination", 2024-09-21T06:48:22.475 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.475 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.475 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.476 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.476 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.476 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.476 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.476 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T06:48:22.476 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.476 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.476 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.476 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_from_name": { 2024-09-21T06:48:22.476 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_from_name", 2024-09-21T06:48:22.476 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.476 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.477 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.477 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "Ceph", 2024-09-21T06:48:22.477 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.477 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.477 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.477 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Email From: name", 2024-09-21T06:48:22.477 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.477 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.477 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.477 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.477 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_host": { 2024-09-21T06:48:22.477 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_host", 2024-09-21T06:48:22.477 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.477 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.477 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.478 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.478 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.478 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.478 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.478 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP server", 2024-09-21T06:48:22.478 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.478 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.478 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.478 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.478 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_password": { 2024-09-21T06:48:22.478 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_password", 2024-09-21T06:48:22.478 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.478 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.478 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.479 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.479 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.479 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.479 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.479 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Password to authenticate with", 2024-09-21T06:48:22.479 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.479 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.479 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.479 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.479 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_port": { 2024-09-21T06:48:22.479 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_port", 2024-09-21T06:48:22.479 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.479 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.479 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.480 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "465", 2024-09-21T06:48:22.480 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.480 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.480 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.480 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP port", 2024-09-21T06:48:22.480 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.480 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.480 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.480 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_sender": { 2024-09-21T06:48:22.480 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_sender", 2024-09-21T06:48:22.480 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.480 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.480 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.480 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.480 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP envelope sender", 2024-09-21T06:48:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_ssl": { 2024-09-21T06:48:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_ssl", 2024-09-21T06:48:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.482 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.482 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T06:48:22.482 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.482 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.482 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.482 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.482 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_user": { 2024-09-21T06:48:22.482 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_user", 2024-09-21T06:48:22.482 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.482 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.482 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.482 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.482 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.482 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.482 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.483 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "User to authenticate as", 2024-09-21T06:48:22.483 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.483 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.483 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.483 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.483 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.483 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.483 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.483 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.483 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.483 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.483 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.483 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.483 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.484 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.484 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.484 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.484 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.484 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.484 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.484 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.484 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.484 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "balancer", 2024-09-21T06:48:22.484 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.484 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.484 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.484 INFO:teuthology.orchestra.run.smithi067.stdout: "active": { 2024-09-21T06:48:22.484 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "active", 2024-09-21T06:48:22.484 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.485 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.485 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.485 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.485 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.485 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.485 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.485 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T06:48:22.485 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.485 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.485 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.485 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.485 INFO:teuthology.orchestra.run.smithi067.stdout: "begin_time": { 2024-09-21T06:48:22.485 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "begin_time", 2024-09-21T06:48:22.486 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.486 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.486 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.486 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0000", 2024-09-21T06:48:22.486 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.486 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.486 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.486 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T06:48:22.486 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T06:48:22.486 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.486 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.486 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.486 INFO:teuthology.orchestra.run.smithi067.stdout: "begin_weekday": { 2024-09-21T06:48:22.486 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "begin_weekday", 2024-09-21T06:48:22.486 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-09-21T06:48:22.487 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.487 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.487 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.487 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0", 2024-09-21T06:48:22.487 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "6", 2024-09-21T06:48:22.487 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.487 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T06:48:22.487 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T06:48:22.487 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.487 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.487 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.487 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_max_iterations": { 2024-09-21T06:48:22.487 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_max_iterations", 2024-09-21T06:48:22.487 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-09-21T06:48:22.488 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.488 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.488 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "25", 2024-09-21T06:48:22.488 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-09-21T06:48:22.488 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "250", 2024-09-21T06:48:22.488 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.488 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T06:48:22.488 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.488 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.488 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.488 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.488 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_metrics": { 2024-09-21T06:48:22.488 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_metrics", 2024-09-21T06:48:22.488 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.488 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.489 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.489 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T06:48:22.489 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.489 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.489 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.489 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T06:48:22.489 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T06:48:22.489 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.489 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.489 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.489 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_step": { 2024-09-21T06:48:22.489 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_step", 2024-09-21T06:48:22.489 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-21T06:48:22.489 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.490 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.490 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0.5", 2024-09-21T06:48:22.490 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0.001", 2024-09-21T06:48:22.490 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "0.999", 2024-09-21T06:48:22.490 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.490 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "aggressiveness of optimization", 2024-09-21T06:48:22.490 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T06:48:22.490 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.490 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.490 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.490 INFO:teuthology.orchestra.run.smithi067.stdout: "end_time": { 2024-09-21T06:48:22.490 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "end_time", 2024-09-21T06:48:22.490 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.490 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.491 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.491 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "2359", 2024-09-21T06:48:22.491 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.491 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.491 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.491 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T06:48:22.491 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T06:48:22.491 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.491 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.491 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.491 INFO:teuthology.orchestra.run.smithi067.stdout: "end_weekday": { 2024-09-21T06:48:22.491 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "end_weekday", 2024-09-21T06:48:22.491 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-09-21T06:48:22.491 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.492 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.492 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.492 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0", 2024-09-21T06:48:22.492 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "6", 2024-09-21T06:48:22.492 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.492 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T06:48:22.492 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T06:48:22.492 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.492 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.492 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.492 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.492 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.492 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.492 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.492 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.493 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.493 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.493 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.493 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.493 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.493 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.493 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.493 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.493 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.493 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.493 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.493 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.493 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.493 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.494 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.494 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.494 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.494 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.494 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.494 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.494 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.494 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.494 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.494 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.494 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.494 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.494 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.494 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.494 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.495 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.495 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.495 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.495 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.495 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.495 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.495 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.495 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.495 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.495 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.495 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.495 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.495 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.495 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.496 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.496 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.496 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.496 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.496 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.496 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.496 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.496 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.496 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.496 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.496 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.496 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.496 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.496 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.497 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.497 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.497 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.497 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.497 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.497 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.497 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.497 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.497 INFO:teuthology.orchestra.run.smithi067.stdout: "min_score": { 2024-09-21T06:48:22.497 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "min_score", 2024-09-21T06:48:22.497 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-21T06:48:22.497 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.497 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.497 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.497 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.498 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.498 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.498 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T06:48:22.498 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.498 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.498 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.498 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.498 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": { 2024-09-21T06:48:22.498 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mode", 2024-09-21T06:48:22.498 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.498 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.498 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.498 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "upmap", 2024-09-21T06:48:22.498 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.499 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.499 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.499 INFO:teuthology.orchestra.run.smithi067.stdout: "crush-compat", 2024-09-21T06:48:22.499 INFO:teuthology.orchestra.run.smithi067.stdout: "none", 2024-09-21T06:48:22.499 INFO:teuthology.orchestra.run.smithi067.stdout: "read", 2024-09-21T06:48:22.499 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap", 2024-09-21T06:48:22.499 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap-read" 2024-09-21T06:48:22.499 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.499 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Balancer mode", 2024-09-21T06:48:22.499 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.499 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.499 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.499 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.499 INFO:teuthology.orchestra.run.smithi067.stdout: "pool_ids": { 2024-09-21T06:48:22.500 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pool_ids", 2024-09-21T06:48:22.500 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.500 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.500 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.500 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.500 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.500 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.500 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.500 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T06:48:22.500 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.500 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.500 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.500 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.500 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-09-21T06:48:22.501 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-09-21T06:48:22.501 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.501 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.501 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.501 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-09-21T06:48:22.501 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.501 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.501 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.501 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T06:48:22.501 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.501 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.501 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.501 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.501 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.502 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.502 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.502 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.502 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.502 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.502 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.502 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.502 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.502 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.502 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.502 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.502 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.502 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.502 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap_max_deviation": { 2024-09-21T06:48:22.503 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "upmap_max_deviation", 2024-09-21T06:48:22.503 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.503 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.503 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.503 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-09-21T06:48:22.503 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-09-21T06:48:22.503 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.503 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.503 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T06:48:22.503 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T06:48:22.503 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.503 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.503 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.503 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap_max_optimizations": { 2024-09-21T06:48:22.504 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "upmap_max_optimizations", 2024-09-21T06:48:22.504 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-09-21T06:48:22.504 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.504 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.504 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-21T06:48:22.504 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.504 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.504 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.504 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T06:48:22.504 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.504 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.504 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.504 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.504 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.505 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.505 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.505 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephadm", 2024-09-21T06:48:22.505 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.505 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.505 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.505 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_down_multiplier": { 2024-09-21T06:48:22.505 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_down_multiplier", 2024-09-21T06:48:22.505 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-21T06:48:22.505 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.505 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.505 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3.0", 2024-09-21T06:48:22.505 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.505 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.506 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.506 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T06:48:22.506 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.506 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.506 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.506 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.506 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_refresh_rate": { 2024-09-21T06:48:22.506 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_refresh_rate", 2024-09-21T06:48:22.506 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.506 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.506 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.506 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "20", 2024-09-21T06:48:22.506 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.506 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.507 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.507 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T06:48:22.507 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.507 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.507 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.507 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.507 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_starting_port": { 2024-09-21T06:48:22.507 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_starting_port", 2024-09-21T06:48:22.507 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.507 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.507 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.507 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "4721", 2024-09-21T06:48:22.507 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.507 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.508 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.508 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T06:48:22.508 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.508 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.508 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.508 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.508 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_ptrace": { 2024-09-21T06:48:22.508 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_ptrace", 2024-09-21T06:48:22.508 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.508 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.508 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.508 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.508 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.509 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.509 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.509 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T06:48:22.509 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T06:48:22.509 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.509 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.509 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.509 INFO:teuthology.orchestra.run.smithi067.stdout: "autotune_interval": { 2024-09-21T06:48:22.509 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "autotune_interval", 2024-09-21T06:48:22.509 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.509 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.509 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.509 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-09-21T06:48:22.509 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.509 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.510 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.510 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T06:48:22.510 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.510 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.510 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.510 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.510 INFO:teuthology.orchestra.run.smithi067.stdout: "autotune_memory_target_ratio": { 2024-09-21T06:48:22.510 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T06:48:22.510 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-21T06:48:22.510 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.510 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.510 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0.7", 2024-09-21T06:48:22.510 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.510 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.511 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.511 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T06:48:22.511 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.511 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.511 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.511 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.511 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm_log_destination": { 2024-09-21T06:48:22.511 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephadm_log_destination", 2024-09-21T06:48:22.511 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.511 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.511 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.511 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.511 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.511 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.511 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.512 INFO:teuthology.orchestra.run.smithi067.stdout: "file", 2024-09-21T06:48:22.512 INFO:teuthology.orchestra.run.smithi067.stdout: "file,syslog", 2024-09-21T06:48:22.512 INFO:teuthology.orchestra.run.smithi067.stdout: "syslog" 2024-09-21T06:48:22.512 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.512 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T06:48:22.512 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.512 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.512 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.512 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.512 INFO:teuthology.orchestra.run.smithi067.stdout: "cgroups_split": { 2024-09-21T06:48:22.512 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cgroups_split", 2024-09-21T06:48:22.512 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.512 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.512 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.512 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.513 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.513 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.513 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.513 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T06:48:22.513 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.513 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.513 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.513 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.513 INFO:teuthology.orchestra.run.smithi067.stdout: "config_checks_enabled": { 2024-09-21T06:48:22.513 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_checks_enabled", 2024-09-21T06:48:22.513 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.513 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.513 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.513 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.513 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.514 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.514 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.514 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T06:48:22.514 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.514 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.514 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.514 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.514 INFO:teuthology.orchestra.run.smithi067.stdout: "config_dashboard": { 2024-09-21T06:48:22.514 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_dashboard", 2024-09-21T06:48:22.514 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.514 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.514 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.514 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.514 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.515 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.515 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.515 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T06:48:22.515 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.515 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.515 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.515 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.515 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_alertmanager": { 2024-09-21T06:48:22.515 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_alertmanager", 2024-09-21T06:48:22.515 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.515 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.515 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.515 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T06:48:22.515 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.515 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.516 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.516 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-09-21T06:48:22.516 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.516 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.516 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.516 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.516 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_base": { 2024-09-21T06:48:22.516 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_base", 2024-09-21T06:48:22.516 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.516 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.516 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.516 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T06:48:22.516 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.516 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.516 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.517 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Container image name, without the tag", 2024-09-21T06:48:22.517 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.517 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.517 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.517 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.517 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_elasticsearch": { 2024-09-21T06:48:22.517 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_elasticsearch", 2024-09-21T06:48:22.517 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.517 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.517 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.517 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T06:48:22.517 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.517 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.517 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.518 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "elasticsearch container image", 2024-09-21T06:48:22.518 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.518 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.518 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.518 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.518 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_grafana": { 2024-09-21T06:48:22.518 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_grafana", 2024-09-21T06:48:22.518 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.518 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.518 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.518 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T06:48:22.518 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.518 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.518 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.519 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-09-21T06:48:22.519 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.519 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.519 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.519 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.519 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_haproxy": { 2024-09-21T06:48:22.519 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_haproxy", 2024-09-21T06:48:22.519 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.519 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.519 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.519 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T06:48:22.519 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.519 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.519 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.520 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "HAproxy container image", 2024-09-21T06:48:22.520 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.520 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.520 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.520 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.520 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_agent": { 2024-09-21T06:48:22.520 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_agent", 2024-09-21T06:48:22.520 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.520 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.520 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.520 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T06:48:22.520 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.520 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.520 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.521 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger agent container image", 2024-09-21T06:48:22.521 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.521 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.521 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.521 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.521 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_collector": { 2024-09-21T06:48:22.521 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_collector", 2024-09-21T06:48:22.521 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.521 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.521 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.521 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T06:48:22.521 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.521 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.521 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.522 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger collector container image", 2024-09-21T06:48:22.522 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.522 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.522 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.522 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.522 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_query": { 2024-09-21T06:48:22.522 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_query", 2024-09-21T06:48:22.522 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.522 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.522 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.522 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T06:48:22.522 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.522 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.522 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.522 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger query container image", 2024-09-21T06:48:22.523 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.523 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.523 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.523 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.523 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_keepalived": { 2024-09-21T06:48:22.523 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_keepalived", 2024-09-21T06:48:22.523 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.523 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.523 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.523 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T06:48:22.523 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.523 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.523 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.523 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Keepalived container image", 2024-09-21T06:48:22.523 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.524 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.524 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.524 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.524 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_loki": { 2024-09-21T06:48:22.524 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_loki", 2024-09-21T06:48:22.524 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.524 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.524 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.524 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T06:48:22.524 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.524 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.524 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.524 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Loki container image", 2024-09-21T06:48:22.524 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.524 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.525 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.525 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.525 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_nginx": { 2024-09-21T06:48:22.525 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_nginx", 2024-09-21T06:48:22.525 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.525 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.525 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.525 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T06:48:22.525 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.525 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.525 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.525 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Nginx container image", 2024-09-21T06:48:22.525 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.525 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.525 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.526 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.526 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_node_exporter": { 2024-09-21T06:48:22.526 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_node_exporter", 2024-09-21T06:48:22.526 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.526 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.526 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.526 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T06:48:22.526 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.526 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.526 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.526 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-09-21T06:48:22.526 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.526 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.526 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.526 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.527 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_nvmeof": { 2024-09-21T06:48:22.527 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_nvmeof", 2024-09-21T06:48:22.527 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.527 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.527 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.527 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T06:48:22.527 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.527 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.527 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.527 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Nvme-of container image", 2024-09-21T06:48:22.527 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.527 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.527 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.527 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.527 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_oauth2_proxy": { 2024-09-21T06:48:22.528 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T06:48:22.528 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.528 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.528 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.528 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T06:48:22.528 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.528 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.528 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.528 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "oauth2-proxy container image", 2024-09-21T06:48:22.528 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.529 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.529 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.529 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.529 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_prometheus": { 2024-09-21T06:48:22.529 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_prometheus", 2024-09-21T06:48:22.529 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.529 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.529 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.529 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T06:48:22.529 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.529 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.529 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.529 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-09-21T06:48:22.530 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.530 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.530 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.530 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.530 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_promtail": { 2024-09-21T06:48:22.530 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_promtail", 2024-09-21T06:48:22.530 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.530 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.530 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.530 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T06:48:22.530 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.530 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.530 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.530 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Promtail container image", 2024-09-21T06:48:22.531 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.531 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.531 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.531 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.531 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_samba": { 2024-09-21T06:48:22.531 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_samba", 2024-09-21T06:48:22.531 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.531 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.531 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.531 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T06:48:22.531 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.531 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.531 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.531 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Samba/SMB container image", 2024-09-21T06:48:22.532 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.532 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.532 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.532 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.532 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_samba_metrics": { 2024-09-21T06:48:22.532 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_samba_metrics", 2024-09-21T06:48:22.532 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.532 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.532 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.532 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T06:48:22.532 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.532 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.532 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.532 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T06:48:22.533 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.533 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.533 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.533 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.533 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_snmp_gateway": { 2024-09-21T06:48:22.533 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_snmp_gateway", 2024-09-21T06:48:22.533 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.533 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.533 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.533 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T06:48:22.533 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.533 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.533 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.533 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SNMP Gateway container image", 2024-09-21T06:48:22.533 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.534 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.534 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.534 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.534 INFO:teuthology.orchestra.run.smithi067.stdout: "container_init": { 2024-09-21T06:48:22.534 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_init", 2024-09-21T06:48:22.534 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.534 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.534 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.534 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.534 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.534 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.534 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.534 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T06:48:22.534 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.535 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.535 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.535 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.535 INFO:teuthology.orchestra.run.smithi067.stdout: "daemon_cache_timeout": { 2024-09-21T06:48:22.535 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "daemon_cache_timeout", 2024-09-21T06:48:22.535 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.535 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.535 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.535 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-09-21T06:48:22.535 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.535 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.535 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.535 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T06:48:22.536 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.536 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.536 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.536 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.536 INFO:teuthology.orchestra.run.smithi067.stdout: "default_cephadm_command_timeout": { 2024-09-21T06:48:22.536 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T06:48:22.536 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.536 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.536 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.536 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "900", 2024-09-21T06:48:22.536 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.536 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.536 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.536 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T06:48:22.537 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.537 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.537 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.537 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.537 INFO:teuthology.orchestra.run.smithi067.stdout: "default_registry": { 2024-09-21T06:48:22.537 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "default_registry", 2024-09-21T06:48:22.537 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.537 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.537 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.537 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io", 2024-09-21T06:48:22.537 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.537 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.537 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.537 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T06:48:22.537 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.538 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.538 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.538 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.538 INFO:teuthology.orchestra.run.smithi067.stdout: "device_cache_timeout": { 2024-09-21T06:48:22.538 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_cache_timeout", 2024-09-21T06:48:22.538 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.538 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.538 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.538 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1800", 2024-09-21T06:48:22.538 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.538 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.538 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.538 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache device inventory", 2024-09-21T06:48:22.538 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.539 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.539 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.539 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.539 INFO:teuthology.orchestra.run.smithi067.stdout: "device_enhanced_scan": { 2024-09-21T06:48:22.539 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_enhanced_scan", 2024-09-21T06:48:22.539 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.539 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.539 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.539 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.539 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.539 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.539 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.539 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T06:48:22.539 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.539 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.540 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.540 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.540 INFO:teuthology.orchestra.run.smithi067.stdout: "facts_cache_timeout": { 2024-09-21T06:48:22.540 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "facts_cache_timeout", 2024-09-21T06:48:22.540 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.540 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.540 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.540 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-09-21T06:48:22.540 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.540 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.540 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.540 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache host facts data", 2024-09-21T06:48:22.540 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.540 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.541 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.541 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.541 INFO:teuthology.orchestra.run.smithi067.stdout: "grafana_dashboards_path": { 2024-09-21T06:48:22.541 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "grafana_dashboards_path", 2024-09-21T06:48:22.541 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.541 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.541 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.541 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T06:48:22.541 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.541 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.541 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.541 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T06:48:22.541 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.541 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.541 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.542 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.542 INFO:teuthology.orchestra.run.smithi067.stdout: "host_check_interval": { 2024-09-21T06:48:22.542 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "host_check_interval", 2024-09-21T06:48:22.542 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.542 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.542 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.542 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-09-21T06:48:22.542 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.542 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.542 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.542 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to perform a host check", 2024-09-21T06:48:22.542 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.542 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.542 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.543 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.543 INFO:teuthology.orchestra.run.smithi067.stdout: "hw_monitoring": { 2024-09-21T06:48:22.543 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "hw_monitoring", 2024-09-21T06:48:22.543 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.543 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.543 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.543 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.543 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.543 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.543 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.543 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T06:48:22.543 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.543 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.543 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.543 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.544 INFO:teuthology.orchestra.run.smithi067.stdout: "inventory_list_all": { 2024-09-21T06:48:22.544 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "inventory_list_all", 2024-09-21T06:48:22.544 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.544 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.544 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.544 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.544 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.544 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.544 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.544 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T06:48:22.544 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.544 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.544 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.544 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.544 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.545 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.545 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.545 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.545 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.545 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.545 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.545 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.545 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.545 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.545 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.545 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.545 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.545 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.545 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.545 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.546 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.546 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.546 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.546 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.546 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.546 INFO:teuthology.orchestra.run.smithi067.stdout: "log_refresh_metadata": { 2024-09-21T06:48:22.546 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_refresh_metadata", 2024-09-21T06:48:22.546 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.546 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.546 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.546 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.546 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.546 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.546 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.547 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T06:48:22.547 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.547 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.547 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.547 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.547 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.547 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.547 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.547 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.547 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.547 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.547 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.547 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.548 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.548 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T06:48:22.548 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.548 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.548 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.548 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.548 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.548 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.548 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.548 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.548 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.548 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.548 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.548 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.548 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.550 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.550 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.550 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.550 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.550 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.550 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.550 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.550 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.550 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.550 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.550 INFO:teuthology.orchestra.run.smithi067.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T06:48:22.550 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T06:48:22.550 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.550 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.550 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.551 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.551 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.551 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.551 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.551 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T06:48:22.551 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.551 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.551 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.551 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.551 INFO:teuthology.orchestra.run.smithi067.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T06:48:22.551 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T06:48:22.551 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.551 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.551 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.552 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "*", 2024-09-21T06:48:22.552 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.552 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.552 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.552 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T06:48:22.552 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.552 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.552 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.552 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.552 INFO:teuthology.orchestra.run.smithi067.stdout: "max_count_per_host": { 2024-09-21T06:48:22.552 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_count_per_host", 2024-09-21T06:48:22.552 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.552 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.552 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.552 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-21T06:48:22.553 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.553 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.553 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.553 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "max number of daemons per service per host", 2024-09-21T06:48:22.553 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.553 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.553 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.553 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.553 INFO:teuthology.orchestra.run.smithi067.stdout: "max_osd_draining_count": { 2024-09-21T06:48:22.553 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_osd_draining_count", 2024-09-21T06:48:22.553 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.553 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.553 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.553 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-21T06:48:22.553 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.554 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.554 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.554 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T06:48:22.554 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.554 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.554 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.554 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.554 INFO:teuthology.orchestra.run.smithi067.stdout: "migration_current": { 2024-09-21T06:48:22.554 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "migration_current", 2024-09-21T06:48:22.554 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.554 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.554 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.554 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.554 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.555 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.555 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.555 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "internal - do not modify", 2024-09-21T06:48:22.555 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.555 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.555 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.555 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.555 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": { 2024-09-21T06:48:22.555 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mode", 2024-09-21T06:48:22.555 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.555 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.555 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.555 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "root", 2024-09-21T06:48:22.555 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.555 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.556 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.556 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm-package", 2024-09-21T06:48:22.556 INFO:teuthology.orchestra.run.smithi067.stdout: "root" 2024-09-21T06:48:22.556 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.556 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T06:48:22.556 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.556 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.556 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.556 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.556 INFO:teuthology.orchestra.run.smithi067.stdout: "oob_default_addr": { 2024-09-21T06:48:22.556 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "oob_default_addr", 2024-09-21T06:48:22.556 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.556 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.556 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.557 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "169.254.1.1", 2024-09-21T06:48:22.557 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.557 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.557 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.557 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T06:48:22.557 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.557 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.557 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.557 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.557 INFO:teuthology.orchestra.run.smithi067.stdout: "prometheus_alerts_path": { 2024-09-21T06:48:22.557 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prometheus_alerts_path", 2024-09-21T06:48:22.557 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.557 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.557 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.558 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T06:48:22.558 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.558 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.558 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.558 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T06:48:22.558 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.558 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.558 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.558 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.558 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_insecure": { 2024-09-21T06:48:22.558 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_insecure", 2024-09-21T06:48:22.558 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.558 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.558 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.559 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.559 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.559 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.559 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.559 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T06:48:22.559 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.559 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.559 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.559 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.559 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_password": { 2024-09-21T06:48:22.559 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_password", 2024-09-21T06:48:22.559 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.559 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.559 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.559 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.559 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.560 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.560 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.560 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T06:48:22.560 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.560 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.560 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.560 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.560 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_url": { 2024-09-21T06:48:22.560 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_url", 2024-09-21T06:48:22.560 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.560 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.560 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.560 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.560 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.560 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.561 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.561 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T06:48:22.561 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.561 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.561 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.561 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.561 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_username": { 2024-09-21T06:48:22.561 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_username", 2024-09-21T06:48:22.561 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.561 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.561 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.561 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.561 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.561 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.561 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.561 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T06:48:22.562 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.562 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.562 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.562 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.562 INFO:teuthology.orchestra.run.smithi067.stdout: "secure_monitoring_stack": { 2024-09-21T06:48:22.562 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "secure_monitoring_stack", 2024-09-21T06:48:22.562 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.562 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.562 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.562 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.562 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.562 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.562 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.562 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T06:48:22.562 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.563 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.563 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.563 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.563 INFO:teuthology.orchestra.run.smithi067.stdout: "service_discovery_port": { 2024-09-21T06:48:22.563 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "service_discovery_port", 2024-09-21T06:48:22.563 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.563 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.563 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.563 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8765", 2024-09-21T06:48:22.563 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.563 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.563 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.563 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "cephadm service discovery port", 2024-09-21T06:48:22.563 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.563 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.564 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.564 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.564 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.564 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.564 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.564 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.564 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.564 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.564 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.564 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.564 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.564 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.564 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.564 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.565 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.565 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.565 INFO:teuthology.orchestra.run.smithi067.stdout: "ssh_config_file": { 2024-09-21T06:48:22.565 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssh_config_file", 2024-09-21T06:48:22.565 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.565 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.565 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.565 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.565 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.565 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.565 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.565 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T06:48:22.565 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.565 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.565 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.566 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.566 INFO:teuthology.orchestra.run.smithi067.stdout: "ssh_keepalive_count_max": { 2024-09-21T06:48:22.566 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T06:48:22.566 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.566 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.566 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.566 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3", 2024-09-21T06:48:22.566 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.566 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.566 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.566 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T06:48:22.566 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.566 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.566 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.566 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.567 INFO:teuthology.orchestra.run.smithi067.stdout: "ssh_keepalive_interval": { 2024-09-21T06:48:22.567 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssh_keepalive_interval", 2024-09-21T06:48:22.567 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.567 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.567 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.567 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7", 2024-09-21T06:48:22.567 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.567 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.567 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.567 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T06:48:22.567 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.567 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.567 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.567 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.567 INFO:teuthology.orchestra.run.smithi067.stdout: "use_agent": { 2024-09-21T06:48:22.568 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "use_agent", 2024-09-21T06:48:22.568 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.568 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.568 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.568 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.568 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.568 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.568 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.568 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T06:48:22.568 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.568 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.568 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.568 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.568 INFO:teuthology.orchestra.run.smithi067.stdout: "use_repo_digest": { 2024-09-21T06:48:22.568 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "use_repo_digest", 2024-09-21T06:48:22.568 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.569 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.569 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.569 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.569 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.569 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.569 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.569 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T06:48:22.569 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.569 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.569 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.569 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.569 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_failed_host_check": { 2024-09-21T06:48:22.569 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_failed_host_check", 2024-09-21T06:48:22.569 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.569 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.569 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.570 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.570 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.570 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.570 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.570 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T06:48:22.570 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.570 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.570 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.570 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.570 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_stray_daemons": { 2024-09-21T06:48:22.570 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_stray_daemons", 2024-09-21T06:48:22.570 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.570 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.570 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.571 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.571 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.571 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.571 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.571 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T06:48:22.571 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.571 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.571 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.571 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.571 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_stray_hosts": { 2024-09-21T06:48:22.571 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_stray_hosts", 2024-09-21T06:48:22.571 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.571 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.571 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.572 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.572 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.572 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.572 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.572 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T06:48:22.572 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.572 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.572 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.572 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.572 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.572 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.572 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.572 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crash", 2024-09-21T06:48:22.572 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.572 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.573 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.573 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.573 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.573 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.573 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.573 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.573 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.573 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.573 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.573 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.573 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.573 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.573 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.573 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.573 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.574 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.574 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.574 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.574 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.574 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.574 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.574 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.574 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.574 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.574 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.574 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.574 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.574 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.574 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.575 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.575 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.575 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.575 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.575 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.575 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.575 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.575 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.575 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.575 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.575 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.575 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.575 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.575 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.575 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.576 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.576 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.576 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.576 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.576 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.576 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.576 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.576 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.576 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.576 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.576 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.576 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.576 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.576 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.577 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.577 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.577 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.577 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.577 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.577 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.577 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.577 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.577 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.577 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.577 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.577 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.577 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.577 INFO:teuthology.orchestra.run.smithi067.stdout: "retain_interval": { 2024-09-21T06:48:22.577 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "retain_interval", 2024-09-21T06:48:22.578 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.578 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.578 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.578 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "31536000", 2024-09-21T06:48:22.578 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.578 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.578 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.578 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T06:48:22.578 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.578 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.578 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.578 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.578 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.578 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.579 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.579 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.579 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.579 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.579 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.579 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.579 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.579 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.579 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.579 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.579 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.579 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.579 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_recent_interval": { 2024-09-21T06:48:22.579 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_recent_interval", 2024-09-21T06:48:22.579 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.580 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.580 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.580 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1209600", 2024-09-21T06:48:22.580 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.580 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.580 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.580 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T06:48:22.580 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.580 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.580 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.580 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.580 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.580 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.580 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.580 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "dashboard", 2024-09-21T06:48:22.581 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.581 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.581 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.581 INFO:teuthology.orchestra.run.smithi067.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T06:48:22.581 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T06:48:22.581 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.581 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.581 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.581 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-21T06:48:22.581 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.581 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.581 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.581 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.581 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.581 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.581 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.582 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.582 INFO:teuthology.orchestra.run.smithi067.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T06:48:22.582 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T06:48:22.582 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.582 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.582 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.582 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.582 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.582 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.582 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.582 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.582 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.582 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.582 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.582 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.583 INFO:teuthology.orchestra.run.smithi067.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T06:48:22.583 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T06:48:22.583 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.583 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.583 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.583 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.583 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.583 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.583 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.583 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.583 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.583 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.583 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.583 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.584 INFO:teuthology.orchestra.run.smithi067.stdout: "AUDIT_API_ENABLED": { 2024-09-21T06:48:22.584 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T06:48:22.584 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.584 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.584 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.584 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.584 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.584 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.584 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.584 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.584 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.584 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.584 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.584 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.584 INFO:teuthology.orchestra.run.smithi067.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T06:48:22.584 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T06:48:22.585 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.585 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.585 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.585 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.585 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.585 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.585 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.585 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.585 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.585 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.585 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.585 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.585 INFO:teuthology.orchestra.run.smithi067.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T06:48:22.585 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T06:48:22.585 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.585 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.585 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.586 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.586 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.586 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.586 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.586 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.586 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.586 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.586 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.586 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.586 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T06:48:22.586 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T06:48:22.586 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.586 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.586 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.587 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.587 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.587 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.587 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.587 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.587 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.587 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.587 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.587 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.587 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T06:48:22.587 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T06:48:22.587 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.587 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.587 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.587 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.588 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.588 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.588 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.588 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.588 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.588 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.588 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.588 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.588 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T06:48:22.588 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T06:48:22.588 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.588 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.588 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.588 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.588 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.589 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.589 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.589 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.589 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.589 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.589 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.589 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.589 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T06:48:22.589 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T06:48:22.589 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.589 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.589 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.589 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.589 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.589 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.590 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.590 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.590 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.590 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.590 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.590 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.590 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T06:48:22.590 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T06:48:22.590 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.590 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.590 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.590 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.590 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.590 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.590 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.591 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.591 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.591 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.591 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.591 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.591 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T06:48:22.591 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T06:48:22.591 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.591 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.591 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.591 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.591 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.591 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.591 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.591 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.591 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.592 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.592 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.592 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.592 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T06:48:22.592 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T06:48:22.592 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.592 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.592 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.592 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.592 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.592 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.592 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.592 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.592 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.592 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.592 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.593 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.593 INFO:teuthology.orchestra.run.smithi067.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T06:48:22.593 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T06:48:22.593 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.593 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.593 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.593 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.593 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.593 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.593 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.593 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.593 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.593 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.593 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.593 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.593 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T06:48:22.593 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T06:48:22.594 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.594 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.594 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.594 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-09-21T06:48:22.594 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.594 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.594 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.594 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.594 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.594 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.594 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.594 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.594 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T06:48:22.594 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T06:48:22.595 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.595 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.595 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.595 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.595 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.595 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.595 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.595 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.595 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.595 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.595 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.595 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.595 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_URL": { 2024-09-21T06:48:22.595 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_URL", 2024-09-21T06:48:22.595 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.596 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.596 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.596 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.596 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.596 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.596 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.596 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.596 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.596 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.596 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.596 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.596 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T06:48:22.596 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T06:48:22.596 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.597 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.597 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.597 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-09-21T06:48:22.597 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.597 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.597 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.597 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.597 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.597 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.597 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.597 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.597 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T06:48:22.597 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T06:48:22.597 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.597 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.598 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.598 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.598 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.598 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.598 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.598 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.598 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.598 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.598 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.598 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.598 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T06:48:22.598 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T06:48:22.598 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.598 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.598 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.599 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.599 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.599 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.599 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.599 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.599 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.599 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.599 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.599 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.599 INFO:teuthology.orchestra.run.smithi067.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T06:48:22.599 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T06:48:22.599 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.599 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.599 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.599 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.600 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.600 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.600 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.600 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.600 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.600 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.600 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.600 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.600 INFO:teuthology.orchestra.run.smithi067.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T06:48:22.600 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T06:48:22.600 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.600 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.600 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.600 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.600 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.601 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.601 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.601 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.601 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.601 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.601 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.601 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.601 INFO:teuthology.orchestra.run.smithi067.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T06:48:22.601 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T06:48:22.601 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.601 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.601 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.601 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "[]", 2024-09-21T06:48:22.601 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.601 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.602 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.602 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.602 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.602 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.602 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.602 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.602 INFO:teuthology.orchestra.run.smithi067.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T06:48:22.602 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T06:48:22.602 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.602 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.602 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.602 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "{}", 2024-09-21T06:48:22.602 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.602 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.602 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.602 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.603 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.603 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.603 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.603 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.603 INFO:teuthology.orchestra.run.smithi067.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T06:48:22.603 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T06:48:22.603 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.603 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.603 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.603 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.603 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.603 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.603 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.603 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.603 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.604 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.604 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.604 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.604 INFO:teuthology.orchestra.run.smithi067.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T06:48:22.604 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T06:48:22.604 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.604 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.604 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.604 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.604 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.604 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.604 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.604 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.604 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.604 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.605 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.605 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.605 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T06:48:22.605 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T06:48:22.605 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.605 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.605 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.605 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.605 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.605 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.605 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.605 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.605 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.605 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.606 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.606 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.606 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T06:48:22.606 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T06:48:22.606 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.606 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.606 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.606 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.606 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.606 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.606 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.606 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.606 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.606 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.607 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.607 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.607 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T06:48:22.607 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T06:48:22.607 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.607 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.607 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.607 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.607 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.607 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.607 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.607 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.607 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.607 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.607 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.608 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.608 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T06:48:22.608 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T06:48:22.608 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.608 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.608 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.608 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.608 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.608 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.608 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.608 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.608 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.608 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.608 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.608 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.608 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T06:48:22.609 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T06:48:22.609 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.609 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.609 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.609 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.609 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.609 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.609 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.609 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.609 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.609 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.609 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.609 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.609 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T06:48:22.609 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T06:48:22.609 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.610 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.610 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.610 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.610 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.610 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.610 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.610 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.610 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.610 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.610 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.610 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.610 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T06:48:22.610 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T06:48:22.610 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.611 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.611 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.611 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.611 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.611 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.611 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.611 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.611 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.611 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.611 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.611 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.611 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T06:48:22.611 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T06:48:22.611 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.612 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.612 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.612 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.612 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.612 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.612 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.612 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.612 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.612 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.612 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.612 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.612 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T06:48:22.612 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T06:48:22.612 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.613 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.613 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.613 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T06:48:22.613 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.613 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.613 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.613 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.613 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.613 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.613 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.613 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.613 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T06:48:22.613 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T06:48:22.613 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.613 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.614 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.614 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-21T06:48:22.614 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.614 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.614 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.614 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.614 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.614 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.614 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.614 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.614 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T06:48:22.614 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T06:48:22.614 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.614 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.614 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.615 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8", 2024-09-21T06:48:22.615 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.615 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.615 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.615 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.615 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.615 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.615 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.615 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.615 INFO:teuthology.orchestra.run.smithi067.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T06:48:22.615 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T06:48:22.615 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.615 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.615 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.616 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "45", 2024-09-21T06:48:22.616 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.616 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.616 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.616 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.616 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.616 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.616 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.616 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.616 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T06:48:22.616 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T06:48:22.616 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.617 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.617 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.617 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.617 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.618 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.618 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.618 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.618 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.618 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.618 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.618 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.618 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T06:48:22.618 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T06:48:22.618 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.618 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.618 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.618 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-09-21T06:48:22.618 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.619 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.619 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.619 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.619 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.619 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.619 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.619 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.619 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T06:48:22.619 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T06:48:22.619 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.619 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.619 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.619 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.619 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.619 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.620 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.620 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.620 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.620 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.620 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.620 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.620 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T06:48:22.620 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T06:48:22.620 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.620 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.620 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.620 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.620 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.620 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.621 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.621 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.621 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.621 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.621 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.621 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.621 INFO:teuthology.orchestra.run.smithi067.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T06:48:22.621 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T06:48:22.621 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.621 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.621 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.621 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.621 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.621 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.621 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.622 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.622 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.622 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.622 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.622 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.622 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T06:48:22.622 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T06:48:22.622 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.622 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.622 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.622 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.622 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.622 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.622 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.623 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.623 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.623 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.623 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.623 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.623 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T06:48:22.623 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T06:48:22.623 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.623 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.623 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.623 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-21T06:48:22.623 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.623 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.623 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.624 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.624 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.624 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.624 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.624 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.624 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T06:48:22.624 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T06:48:22.624 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.624 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.624 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.624 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-09-21T06:48:22.624 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.624 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.624 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.624 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.625 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.625 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.625 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.625 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.625 INFO:teuthology.orchestra.run.smithi067.stdout: "cross_origin_url": { 2024-09-21T06:48:22.625 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cross_origin_url", 2024-09-21T06:48:22.625 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.625 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.625 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.625 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.625 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.625 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.625 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.625 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.625 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.626 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.626 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.626 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.626 INFO:teuthology.orchestra.run.smithi067.stdout: "crt_file": { 2024-09-21T06:48:22.626 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crt_file", 2024-09-21T06:48:22.626 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.626 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.626 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.626 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.626 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.626 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.626 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.626 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.626 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.626 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.627 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.627 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.627 INFO:teuthology.orchestra.run.smithi067.stdout: "debug": { 2024-09-21T06:48:22.627 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "debug", 2024-09-21T06:48:22.627 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.627 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.627 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.627 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.627 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.627 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.627 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.627 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable/disable debug options", 2024-09-21T06:48:22.627 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.627 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.627 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.628 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.628 INFO:teuthology.orchestra.run.smithi067.stdout: "jwt_token_ttl": { 2024-09-21T06:48:22.628 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "jwt_token_ttl", 2024-09-21T06:48:22.628 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.628 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.628 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.628 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "28800", 2024-09-21T06:48:22.628 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.628 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.628 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.628 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.628 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.628 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.628 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.628 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.629 INFO:teuthology.orchestra.run.smithi067.stdout: "key_file": { 2024-09-21T06:48:22.629 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "key_file", 2024-09-21T06:48:22.629 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.629 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.629 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.629 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.629 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.629 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.629 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.629 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.629 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.629 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.629 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.629 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.629 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.630 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.630 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.630 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.630 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.630 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.630 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.630 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.630 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.630 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.630 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.630 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.630 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.630 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.631 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.631 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.631 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.631 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.631 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.631 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.631 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.631 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.631 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.631 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.631 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.631 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.631 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.631 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.631 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.631 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.632 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.632 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.632 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.632 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.632 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.632 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.632 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.632 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.632 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.632 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.632 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.632 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.632 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.632 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.633 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.633 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.633 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.633 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.633 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.633 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.633 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.633 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.633 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.633 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.633 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.633 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.633 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.633 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.633 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.634 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.634 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.634 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.634 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.634 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.634 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.634 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.634 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.634 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.634 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.634 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.634 INFO:teuthology.orchestra.run.smithi067.stdout: "motd": { 2024-09-21T06:48:22.634 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "motd", 2024-09-21T06:48:22.634 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.635 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.635 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.635 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.635 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.635 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.635 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.635 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "The message of the day", 2024-09-21T06:48:22.635 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.635 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.635 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.635 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.635 INFO:teuthology.orchestra.run.smithi067.stdout: "redirect_resolve_ip_addr": { 2024-09-21T06:48:22.635 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T06:48:22.635 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.635 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.636 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.636 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.636 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.636 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.636 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.636 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.636 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.636 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.636 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.636 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.636 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-09-21T06:48:22.636 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-09-21T06:48:22.636 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.636 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.636 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.637 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "::", 2024-09-21T06:48:22.637 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.637 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.637 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.637 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.637 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.637 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.637 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.637 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.637 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-09-21T06:48:22.637 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-09-21T06:48:22.637 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.637 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.637 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.637 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8080", 2024-09-21T06:48:22.638 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.638 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.638 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.638 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.638 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.638 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.638 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.638 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.638 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.638 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.638 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.638 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.638 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.638 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.639 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.639 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.639 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.639 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.639 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.639 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.639 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.639 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.639 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl": { 2024-09-21T06:48:22.639 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl", 2024-09-21T06:48:22.639 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.639 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.639 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.639 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.639 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.640 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.640 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.640 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.640 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.640 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.640 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.640 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.640 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl_server_port": { 2024-09-21T06:48:22.640 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl_server_port", 2024-09-21T06:48:22.640 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.640 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.640 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.640 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8443", 2024-09-21T06:48:22.640 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.640 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.641 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.641 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.641 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.641 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.641 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.641 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.641 INFO:teuthology.orchestra.run.smithi067.stdout: "sso_oauth2": { 2024-09-21T06:48:22.641 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sso_oauth2", 2024-09-21T06:48:22.641 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.641 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.641 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.641 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.641 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.641 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.641 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.641 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.642 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.642 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.642 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.642 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.642 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_behaviour": { 2024-09-21T06:48:22.642 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_behaviour", 2024-09-21T06:48:22.642 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.642 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.642 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.642 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "redirect", 2024-09-21T06:48:22.642 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.642 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.642 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.642 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.642 INFO:teuthology.orchestra.run.smithi067.stdout: "redirect" 2024-09-21T06:48:22.643 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.643 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.643 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.643 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.643 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.643 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.643 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_error_status_code": { 2024-09-21T06:48:22.643 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_error_status_code", 2024-09-21T06:48:22.643 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.643 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.643 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.643 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-09-21T06:48:22.643 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "400", 2024-09-21T06:48:22.643 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "599", 2024-09-21T06:48:22.643 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.644 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.644 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.644 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.644 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.644 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.644 INFO:teuthology.orchestra.run.smithi067.stdout: "url_prefix": { 2024-09-21T06:48:22.644 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "url_prefix", 2024-09-21T06:48:22.644 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.644 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.644 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.644 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.644 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.644 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.644 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.644 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.645 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.645 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.645 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.645 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.645 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.645 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.645 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.645 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "devicehealth", 2024-09-21T06:48:22.645 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.645 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.645 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.645 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_monitoring": { 2024-09-21T06:48:22.645 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enable_monitoring", 2024-09-21T06:48:22.645 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.645 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.645 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.646 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.646 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.646 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.646 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.646 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "monitor device health metrics", 2024-09-21T06:48:22.646 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.646 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.646 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.646 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.646 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.646 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.646 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.646 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.646 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.647 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.647 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.647 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.647 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.647 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.647 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.647 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.647 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.647 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.647 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.647 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.647 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.647 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.647 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.648 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.648 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.648 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.648 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.648 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.648 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.648 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.648 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.648 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.648 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.648 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.648 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.648 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.648 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.649 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.649 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.649 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.649 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.649 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.649 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.649 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.649 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.649 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.649 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.649 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.649 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.649 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.649 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.649 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.650 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.650 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.650 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.650 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.650 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.650 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.650 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.650 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.650 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.650 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.650 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.650 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.650 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.650 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.650 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.651 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.651 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.651 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.651 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.651 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.651 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.651 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.651 INFO:teuthology.orchestra.run.smithi067.stdout: "mark_out_threshold": { 2024-09-21T06:48:22.651 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mark_out_threshold", 2024-09-21T06:48:22.651 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.651 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.651 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.651 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "2419200", 2024-09-21T06:48:22.651 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.652 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.652 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.652 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T06:48:22.652 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.652 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.652 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.652 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.652 INFO:teuthology.orchestra.run.smithi067.stdout: "pool_name": { 2024-09-21T06:48:22.652 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pool_name", 2024-09-21T06:48:22.652 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.652 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.652 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.652 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "device_health_metrics", 2024-09-21T06:48:22.652 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.653 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.653 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.653 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T06:48:22.653 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.653 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.653 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.653 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.653 INFO:teuthology.orchestra.run.smithi067.stdout: "retention_period": { 2024-09-21T06:48:22.653 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "retention_period", 2024-09-21T06:48:22.653 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.653 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.653 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.653 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15552000", 2024-09-21T06:48:22.653 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.654 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.654 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.654 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long to retain device health metrics", 2024-09-21T06:48:22.654 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.654 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.654 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.654 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.654 INFO:teuthology.orchestra.run.smithi067.stdout: "scrape_frequency": { 2024-09-21T06:48:22.654 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "scrape_frequency", 2024-09-21T06:48:22.654 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.654 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.654 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.654 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "86400", 2024-09-21T06:48:22.654 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.654 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.655 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.655 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T06:48:22.655 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.655 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.655 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.655 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.655 INFO:teuthology.orchestra.run.smithi067.stdout: "self_heal": { 2024-09-21T06:48:22.655 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "self_heal", 2024-09-21T06:48:22.655 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.655 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.655 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.655 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.655 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.655 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.656 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.656 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T06:48:22.656 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.656 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.656 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.656 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.656 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-09-21T06:48:22.656 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-09-21T06:48:22.656 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.656 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.656 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.656 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-09-21T06:48:22.656 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.656 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.656 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.657 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T06:48:22.657 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.657 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.657 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.657 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.657 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.657 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.657 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.657 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.657 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.657 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.657 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.657 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.657 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.657 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.658 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.658 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.658 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.658 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.658 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_threshold": { 2024-09-21T06:48:22.658 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_threshold", 2024-09-21T06:48:22.658 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.658 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.658 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.658 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7257600", 2024-09-21T06:48:22.658 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.658 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.658 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.658 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T06:48:22.659 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.659 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.659 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.659 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.659 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.659 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.659 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.659 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "diskprediction_local", 2024-09-21T06:48:22.659 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.659 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.659 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.659 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.659 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.659 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.659 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.660 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.660 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.660 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.660 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.660 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.660 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.660 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.660 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.660 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.660 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.660 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.660 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.660 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.660 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.661 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.661 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.661 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.661 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.661 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.661 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.661 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.661 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.661 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.661 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.661 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.661 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.661 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.661 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.661 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.662 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.662 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.662 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.662 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.662 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.662 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.662 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.662 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.662 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.662 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.662 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.662 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.662 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.662 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.663 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.663 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.663 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.663 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.663 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.663 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.663 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.663 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.663 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.663 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.663 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.663 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.663 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.663 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.663 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.663 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.664 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.664 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.664 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.664 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.664 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.664 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.664 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.664 INFO:teuthology.orchestra.run.smithi067.stdout: "predict_interval": { 2024-09-21T06:48:22.664 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "predict_interval", 2024-09-21T06:48:22.664 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.664 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.664 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.664 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "86400", 2024-09-21T06:48:22.664 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.665 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.665 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.665 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.665 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.665 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.665 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.665 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.665 INFO:teuthology.orchestra.run.smithi067.stdout: "predictor_model": { 2024-09-21T06:48:22.665 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "predictor_model", 2024-09-21T06:48:22.665 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.665 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.665 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.665 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "prophetstor", 2024-09-21T06:48:22.665 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.665 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.666 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.666 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.666 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.666 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.666 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.666 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.666 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-09-21T06:48:22.666 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-09-21T06:48:22.666 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.666 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.666 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.666 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-09-21T06:48:22.666 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.666 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.667 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.667 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.667 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.667 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.667 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.667 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.667 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.667 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.667 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.667 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.667 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.667 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.667 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.667 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.667 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.668 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.668 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.668 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.668 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.668 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.668 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.668 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.668 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.668 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "influx", 2024-09-21T06:48:22.668 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": false, 2024-09-21T06:48:22.668 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "influxdb python module not found", 2024-09-21T06:48:22.668 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.668 INFO:teuthology.orchestra.run.smithi067.stdout: "batch_size": { 2024-09-21T06:48:22.668 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "batch_size", 2024-09-21T06:48:22.668 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.669 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.669 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.669 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5000", 2024-09-21T06:48:22.669 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.669 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.669 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.669 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T06:48:22.669 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.669 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.669 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.669 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.669 INFO:teuthology.orchestra.run.smithi067.stdout: "database": { 2024-09-21T06:48:22.669 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "database", 2024-09-21T06:48:22.669 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.669 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.670 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.670 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "ceph", 2024-09-21T06:48:22.670 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.670 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.670 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.670 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T06:48:22.670 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.670 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.670 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.670 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.670 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": { 2024-09-21T06:48:22.670 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "hostname", 2024-09-21T06:48:22.670 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.670 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.671 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.671 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.671 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.671 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.671 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.671 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB server hostname", 2024-09-21T06:48:22.671 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.671 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.671 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.671 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.671 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-09-21T06:48:22.671 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-09-21T06:48:22.671 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.671 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.671 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.672 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "30", 2024-09-21T06:48:22.672 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "5", 2024-09-21T06:48:22.672 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.672 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.672 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T06:48:22.672 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.672 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.672 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.672 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.672 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.672 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.672 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.672 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.672 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.673 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.673 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.673 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.673 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.673 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.673 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.673 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.673 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.673 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.673 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.673 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.673 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.673 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.673 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.673 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.674 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.674 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.674 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.674 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.674 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.674 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.674 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.674 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.674 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.674 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.674 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.674 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.674 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.674 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.674 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.675 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.675 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.675 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.675 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.675 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.675 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.675 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.675 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.675 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.675 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.675 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.675 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.675 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.675 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.676 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.676 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.676 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.676 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.676 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.676 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.676 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.676 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.676 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.676 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.676 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.676 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.676 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.676 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.676 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.677 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.677 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.677 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.677 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.677 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.677 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.677 INFO:teuthology.orchestra.run.smithi067.stdout: "password": { 2024-09-21T06:48:22.677 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "password", 2024-09-21T06:48:22.677 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.677 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.677 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.677 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.677 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.677 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.677 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.678 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "password of InfluxDB server user", 2024-09-21T06:48:22.678 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.678 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.678 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.678 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.678 INFO:teuthology.orchestra.run.smithi067.stdout: "port": { 2024-09-21T06:48:22.678 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "port", 2024-09-21T06:48:22.678 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.678 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.678 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.678 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8086", 2024-09-21T06:48:22.678 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.678 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.678 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.678 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB server port", 2024-09-21T06:48:22.679 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.679 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.679 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.679 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.679 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.679 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.679 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.679 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.679 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.679 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.679 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.679 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.679 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.679 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.679 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.680 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.680 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.680 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.680 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl": { 2024-09-21T06:48:22.680 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl", 2024-09-21T06:48:22.680 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.680 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.680 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.680 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "false", 2024-09-21T06:48:22.680 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.680 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.680 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.680 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T06:48:22.680 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.681 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.681 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.681 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.681 INFO:teuthology.orchestra.run.smithi067.stdout: "threads": { 2024-09-21T06:48:22.681 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "threads", 2024-09-21T06:48:22.681 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.681 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.681 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.681 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-09-21T06:48:22.681 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-09-21T06:48:22.681 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "32", 2024-09-21T06:48:22.681 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.681 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T06:48:22.681 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.681 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.682 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.682 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.682 INFO:teuthology.orchestra.run.smithi067.stdout: "username": { 2024-09-21T06:48:22.682 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "username", 2024-09-21T06:48:22.682 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.682 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.682 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.682 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.682 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.682 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.682 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.682 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "username of InfluxDB server user", 2024-09-21T06:48:22.682 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.682 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.683 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.683 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.683 INFO:teuthology.orchestra.run.smithi067.stdout: "verify_ssl": { 2024-09-21T06:48:22.683 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "verify_ssl", 2024-09-21T06:48:22.683 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.683 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.683 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.683 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "true", 2024-09-21T06:48:22.683 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.683 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.683 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.683 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T06:48:22.683 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.683 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.683 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.684 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.684 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.684 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.684 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.684 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "insights", 2024-09-21T06:48:22.684 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.684 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.684 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.684 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.684 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.684 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.684 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.684 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.684 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.685 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.685 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.685 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.685 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.685 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.685 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.685 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.685 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.685 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.685 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.685 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.685 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.685 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.685 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.686 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.686 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.686 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.686 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.686 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.686 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.686 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.686 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.686 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.686 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.686 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.686 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.686 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.686 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.687 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.687 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.687 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.687 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.687 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.687 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.687 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.687 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.687 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.687 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.687 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.687 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.687 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.687 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.687 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.688 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.688 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.688 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.688 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.688 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.688 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.688 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.688 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.688 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.688 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.688 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.688 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.688 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.688 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.689 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.689 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.689 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.689 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.689 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.689 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.689 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.689 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.689 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.689 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.689 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.689 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.689 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.689 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.689 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.690 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.690 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.690 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.690 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.690 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.690 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.690 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.690 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.690 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.690 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "iostat", 2024-09-21T06:48:22.690 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.690 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.690 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.690 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.691 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.691 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.691 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.691 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.691 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.691 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.691 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.691 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.691 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.691 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.691 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.691 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.691 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.691 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.692 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.692 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.692 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.692 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.692 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.692 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.692 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.692 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.692 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.692 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.692 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.692 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.692 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.692 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.692 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.693 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.693 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.693 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.693 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.693 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.693 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.693 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.693 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.693 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.693 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.693 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.693 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.693 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.693 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.694 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.694 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.694 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.694 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.694 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.694 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.694 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.694 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.694 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.694 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.694 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.694 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.694 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.694 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.694 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.695 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.695 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.695 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.695 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.695 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.695 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.695 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.695 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.695 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.695 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.695 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.695 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.695 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.695 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.695 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.696 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.696 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.696 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.696 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.696 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.696 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.696 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.696 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.696 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.696 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.696 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.696 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.696 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.696 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "k8sevents", 2024-09-21T06:48:22.697 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.697 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.697 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.697 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_event_retention_days": { 2024-09-21T06:48:22.697 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ceph_event_retention_days", 2024-09-21T06:48:22.697 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.697 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.697 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.697 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7", 2024-09-21T06:48:22.697 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.697 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.697 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.697 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T06:48:22.697 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.697 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.698 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.698 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.698 INFO:teuthology.orchestra.run.smithi067.stdout: "config_check_secs": { 2024-09-21T06:48:22.698 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_check_secs", 2024-09-21T06:48:22.698 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.698 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.698 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.698 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-21T06:48:22.698 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "10", 2024-09-21T06:48:22.698 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.698 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.698 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T06:48:22.698 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.698 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.699 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.699 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.699 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.699 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.699 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.699 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.699 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.699 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.699 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.699 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.699 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.699 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.699 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.699 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.700 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.700 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.700 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.700 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.700 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.700 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.700 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.700 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.700 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.700 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.700 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.700 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.700 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.700 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.701 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.701 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.701 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.701 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.701 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.701 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.701 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.701 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.701 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.701 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.701 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.701 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.701 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.701 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.702 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.702 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.702 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.702 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.702 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.702 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.702 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.702 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.702 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.702 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.702 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.702 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.702 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.703 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.703 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.703 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.703 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.703 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.703 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.703 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.703 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.703 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.703 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.703 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.703 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.703 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.703 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.703 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.704 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.704 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.704 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.704 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.704 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.704 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.704 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.704 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.704 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.704 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.704 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.704 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.704 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.704 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.704 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.705 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.705 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.705 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.705 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.705 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "localpool", 2024-09-21T06:48:22.705 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.705 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.705 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.705 INFO:teuthology.orchestra.run.smithi067.stdout: "failure_domain": { 2024-09-21T06:48:22.705 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "failure_domain", 2024-09-21T06:48:22.705 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.705 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.705 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.705 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "host", 2024-09-21T06:48:22.705 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.706 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.706 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.706 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "failure domain for any created local pool", 2024-09-21T06:48:22.706 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T06:48:22.706 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.706 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.706 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.706 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.706 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.706 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.706 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.706 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.706 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.706 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.707 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.707 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.707 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.707 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.707 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.707 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.707 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.707 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.707 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.707 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.707 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.707 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.707 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.707 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.707 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.707 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.708 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.708 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.708 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.708 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.708 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.708 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.708 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.708 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.708 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.708 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.708 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.708 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.708 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.708 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.709 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.709 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.709 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.709 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.709 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.709 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.709 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.709 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.709 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.709 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.709 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.709 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.709 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.709 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.709 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.710 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.710 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.710 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.710 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.710 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.710 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.710 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.710 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.710 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.710 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.710 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.710 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.710 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.710 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.710 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.711 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.711 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.711 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.711 INFO:teuthology.orchestra.run.smithi067.stdout: "min_size": { 2024-09-21T06:48:22.711 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "min_size", 2024-09-21T06:48:22.711 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.711 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.711 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.711 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.711 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.711 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.711 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.711 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default min_size for any created local pool", 2024-09-21T06:48:22.711 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T06:48:22.711 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.712 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.712 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.712 INFO:teuthology.orchestra.run.smithi067.stdout: "num_rep": { 2024-09-21T06:48:22.712 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "num_rep", 2024-09-21T06:48:22.712 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.712 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.712 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.712 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3", 2024-09-21T06:48:22.712 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.712 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.712 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.712 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default replica count for any created local pool", 2024-09-21T06:48:22.712 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.712 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.712 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.713 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.713 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_num": { 2024-09-21T06:48:22.713 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pg_num", 2024-09-21T06:48:22.713 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.713 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.713 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.713 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "128", 2024-09-21T06:48:22.713 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.713 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.713 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.713 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T06:48:22.713 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.713 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.713 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.714 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.714 INFO:teuthology.orchestra.run.smithi067.stdout: "prefix": { 2024-09-21T06:48:22.714 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prefix", 2024-09-21T06:48:22.714 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.714 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.714 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.714 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.714 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.714 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.714 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.714 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "name prefix for any created local pool", 2024-09-21T06:48:22.714 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.714 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.714 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.714 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.715 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.715 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.715 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.715 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.715 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.715 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.715 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.715 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.715 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.715 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.715 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.715 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.715 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.715 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.716 INFO:teuthology.orchestra.run.smithi067.stdout: "subtree": { 2024-09-21T06:48:22.716 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subtree", 2024-09-21T06:48:22.716 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.716 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.716 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.716 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "rack", 2024-09-21T06:48:22.716 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.716 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.716 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.716 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T06:48:22.716 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T06:48:22.716 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.716 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.716 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.716 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.717 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.717 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.717 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mds_autoscaler", 2024-09-21T06:48:22.717 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.717 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.717 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.717 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.717 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.717 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.717 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.717 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.717 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.717 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.718 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.718 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.718 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.718 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.718 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.718 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.718 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.718 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.718 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.718 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.718 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.718 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.718 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.718 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.718 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.719 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.719 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.719 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.719 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.719 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.719 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.719 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.719 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.719 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.719 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.719 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.719 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.719 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.720 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.720 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.720 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.720 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.720 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.720 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.720 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.720 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.720 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.720 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.720 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.720 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.720 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.720 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.720 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.721 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.721 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.721 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.721 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.721 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.721 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.721 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.721 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.721 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.721 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.721 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.721 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.721 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.721 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.721 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.722 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.722 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.722 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.722 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.722 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.722 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.722 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.722 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.722 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.722 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.722 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.722 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.722 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.722 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.723 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.723 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.723 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.723 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.723 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.723 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.723 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.723 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.723 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mirroring", 2024-09-21T06:48:22.723 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.723 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.723 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.723 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.723 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.724 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.724 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.724 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.724 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.724 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.724 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.724 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.724 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.724 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.724 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.724 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.724 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.724 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.724 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.724 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.724 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.725 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.725 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.725 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.725 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.725 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.725 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.725 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.725 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.725 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.725 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.725 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.725 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.725 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.725 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.725 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.726 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.726 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.726 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.726 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.726 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.726 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.726 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.726 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.726 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.726 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.726 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.726 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.726 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.726 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.727 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.727 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.727 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.727 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.727 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.727 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.727 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.727 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.727 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.727 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.727 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.727 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.727 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.728 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.728 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.728 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.728 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.728 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.728 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.728 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.728 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.728 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.728 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.728 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.728 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.728 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.728 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.728 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.729 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.729 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.729 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.729 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.729 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.729 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.729 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.729 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.729 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.729 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.729 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.729 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.729 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "nfs", 2024-09-21T06:48:22.729 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.730 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.730 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.730 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.730 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.730 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.730 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.730 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.730 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.730 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.730 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.730 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.730 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.730 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.730 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.730 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.731 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.731 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.731 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.731 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.731 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.731 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.731 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.731 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.731 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.731 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.731 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.731 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.731 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.731 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.732 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.732 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.732 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.732 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.732 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.732 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.732 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.732 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.732 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.732 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.732 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.732 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.732 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.732 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.732 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.733 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.733 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.733 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.733 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.733 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.733 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.733 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.733 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.733 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.733 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.733 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.733 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.733 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.733 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.734 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.734 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.734 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.734 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.734 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.734 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.734 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.734 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.734 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.734 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.734 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.734 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.734 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.734 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.735 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.735 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.735 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.735 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.735 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.735 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.735 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.735 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.735 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.735 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.735 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.735 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.735 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.735 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.735 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.736 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.736 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.736 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "orchestrator", 2024-09-21T06:48:22.736 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.736 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.736 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.736 INFO:teuthology.orchestra.run.smithi067.stdout: "fail_fs": { 2024-09-21T06:48:22.736 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "fail_fs", 2024-09-21T06:48:22.736 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.736 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.736 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.736 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.736 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.736 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.737 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.737 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T06:48:22.737 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.737 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.737 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.737 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.737 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.737 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.737 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.737 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.737 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.737 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.737 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.737 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.737 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.738 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.738 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.738 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.738 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.738 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.738 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.738 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.738 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.738 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.738 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.738 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.738 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.738 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.738 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.739 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.739 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.739 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.739 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.739 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.739 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.739 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.739 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.739 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.739 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.739 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.739 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.739 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.739 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.739 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.740 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.740 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.740 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.740 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.740 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.740 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.740 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.740 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.740 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.740 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.740 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.740 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.740 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.740 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.741 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.741 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.741 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.741 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.741 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.741 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.741 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.741 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.741 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.741 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.741 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.741 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.741 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.741 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.741 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.742 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.742 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.742 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.742 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator": { 2024-09-21T06:48:22.742 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "orchestrator", 2024-09-21T06:48:22.742 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.742 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.742 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.742 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.742 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.742 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.742 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.742 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm", 2024-09-21T06:48:22.742 INFO:teuthology.orchestra.run.smithi067.stdout: "rook", 2024-09-21T06:48:22.743 INFO:teuthology.orchestra.run.smithi067.stdout: "test_orchestrator" 2024-09-21T06:48:22.743 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.743 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Orchestrator backend", 2024-09-21T06:48:22.743 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.743 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.743 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.743 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.743 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.743 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.743 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.743 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.743 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.743 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.743 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.743 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.743 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.744 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.744 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.744 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.744 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.744 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.744 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.744 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.744 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.744 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "osd_perf_query", 2024-09-21T06:48:22.744 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.744 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.744 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.744 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.744 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.745 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.745 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.745 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.745 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.745 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.745 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.745 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.745 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.745 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.745 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.745 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.745 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.745 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.745 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.745 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.746 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.746 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.746 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.746 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.746 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.746 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.746 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.746 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.746 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.746 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.746 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.746 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.746 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.746 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.747 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.747 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.747 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.747 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.747 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.747 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.747 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.747 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.747 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.747 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.747 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.747 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.747 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.747 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.747 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.748 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.748 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.748 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.748 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.748 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.748 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.748 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.748 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.748 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.748 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.748 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.748 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.748 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.748 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.749 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.749 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.749 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.749 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.749 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.749 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.749 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.749 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.749 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.749 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.749 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.749 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.749 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.749 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.749 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.750 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.750 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.750 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.750 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.750 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.750 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.750 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.750 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.750 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.750 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.750 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.750 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.751 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "osd_support", 2024-09-21T06:48:22.751 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.751 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.751 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.751 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.751 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.751 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.751 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.751 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.751 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.751 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.751 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.751 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.751 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.751 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.751 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.752 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.752 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.752 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.752 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.752 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.752 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.752 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.752 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.752 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.752 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.752 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.752 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.752 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.752 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.752 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.752 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.753 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.753 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.753 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.753 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.753 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.753 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.753 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.753 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.753 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.753 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.753 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.753 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.753 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.753 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.753 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.754 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.754 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.754 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.754 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.754 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.754 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.754 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.754 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.754 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.754 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.754 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.754 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.754 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.754 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.754 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.754 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.755 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.755 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.755 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.755 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.755 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.755 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.755 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.755 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.755 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.755 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.755 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.755 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.755 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.755 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.755 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.756 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.756 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.756 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.756 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.756 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.756 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.756 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.756 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.756 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.756 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.756 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.756 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.756 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.756 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pg_autoscaler", 2024-09-21T06:48:22.757 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.757 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.757 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.757 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.757 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.757 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.757 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.757 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.757 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.757 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.757 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.757 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.757 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.757 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.757 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.758 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.758 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.758 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.758 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.758 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.758 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.758 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.758 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.758 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.758 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.758 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.758 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.758 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.758 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.758 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.759 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.759 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.759 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.759 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.759 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.759 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.759 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.759 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.759 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.759 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.759 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.759 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.759 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.759 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.759 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.759 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.760 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.760 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.760 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.760 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.760 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.760 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.760 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.760 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.760 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.760 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.760 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.760 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.760 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.761 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.761 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.761 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.761 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.761 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.761 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.761 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.761 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.761 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.761 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.761 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.761 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.761 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.761 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.762 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-09-21T06:48:22.762 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-09-21T06:48:22.762 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.762 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.762 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.762 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-09-21T06:48:22.762 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.762 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.762 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.762 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.762 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.762 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.762 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.762 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.762 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.763 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.763 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.763 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.763 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.763 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.763 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.763 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.763 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.763 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.763 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.763 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.763 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.763 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.763 INFO:teuthology.orchestra.run.smithi067.stdout: "threshold": { 2024-09-21T06:48:22.764 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "threshold", 2024-09-21T06:48:22.764 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-21T06:48:22.764 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.764 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.764 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3.0", 2024-09-21T06:48:22.764 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1.0", 2024-09-21T06:48:22.764 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.764 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.764 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "scaling threshold", 2024-09-21T06:48:22.764 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T06:48:22.764 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.764 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.764 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.765 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.765 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.765 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.765 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "progress", 2024-09-21T06:48:22.765 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.765 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.765 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.765 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_pg_recovery_event": { 2024-09-21T06:48:22.765 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_pg_recovery_event", 2024-09-21T06:48:22.765 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.765 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.765 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.765 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.765 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.766 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.766 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.766 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T06:48:22.766 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.766 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.766 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.766 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.766 INFO:teuthology.orchestra.run.smithi067.stdout: "enabled": { 2024-09-21T06:48:22.766 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enabled", 2024-09-21T06:48:22.766 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.766 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.766 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.766 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.767 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.767 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.767 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.767 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.767 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.767 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.767 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.767 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.767 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.767 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.767 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.767 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.767 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.767 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.768 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.768 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.768 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.768 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.768 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.768 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.768 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.768 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.768 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.768 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.768 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.768 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.768 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.768 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.768 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.769 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.769 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.769 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.769 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.769 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.769 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.769 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.769 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.769 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.769 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.769 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.769 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.769 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.769 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.770 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.770 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.770 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.770 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.770 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.770 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.770 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.770 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.770 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.770 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.770 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.770 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.770 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.770 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.770 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.771 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.771 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.771 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.771 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.771 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.771 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.771 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.771 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.771 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.771 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.771 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.771 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.771 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.771 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.772 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.772 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.772 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.772 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.772 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.772 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.772 INFO:teuthology.orchestra.run.smithi067.stdout: "max_completed_events": { 2024-09-21T06:48:22.772 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_completed_events", 2024-09-21T06:48:22.772 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.772 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.772 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.772 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "50", 2024-09-21T06:48:22.772 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.772 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.773 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.773 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "number of past completed events to remember", 2024-09-21T06:48:22.773 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.773 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.773 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.773 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.773 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-09-21T06:48:22.773 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-09-21T06:48:22.773 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.773 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.773 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.773 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-09-21T06:48:22.773 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.773 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.774 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.774 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long the module is going to sleep", 2024-09-21T06:48:22.774 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.774 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.774 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.774 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.774 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.774 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.774 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.774 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.774 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.774 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.774 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.774 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.774 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.775 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.775 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.775 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.775 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.775 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.775 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.775 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.775 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.775 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prometheus", 2024-09-21T06:48:22.775 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.775 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.775 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.775 INFO:teuthology.orchestra.run.smithi067.stdout: "cache": { 2024-09-21T06:48:22.775 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cache", 2024-09-21T06:48:22.776 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.776 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.776 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.776 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.776 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.776 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.776 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.776 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.776 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.776 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.776 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.776 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.776 INFO:teuthology.orchestra.run.smithi067.stdout: "exclude_perf_counters": { 2024-09-21T06:48:22.776 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "exclude_perf_counters", 2024-09-21T06:48:22.776 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.777 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.777 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.777 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.777 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.777 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.777 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.777 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T06:48:22.777 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T06:48:22.777 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.777 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.777 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.777 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.777 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.777 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.778 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.778 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.778 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.778 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.778 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.778 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.778 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.778 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.778 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.778 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.778 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.778 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.778 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.779 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.779 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.779 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.779 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.779 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.779 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.779 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.779 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.779 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.779 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.779 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.779 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.779 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.779 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.779 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.780 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.780 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.780 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.780 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.780 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.780 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.780 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.780 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.780 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.780 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.780 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.780 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.780 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.780 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.781 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.781 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.781 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.781 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.781 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.781 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.781 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.781 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.781 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.781 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.781 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.781 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.781 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.781 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.781 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.782 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.782 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.782 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.782 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.782 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.782 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.782 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.782 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.782 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.782 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.782 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_stats_pools": { 2024-09-21T06:48:22.782 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_stats_pools", 2024-09-21T06:48:22.782 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.783 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.783 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.783 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.783 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.783 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.783 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.783 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.783 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.783 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.783 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.783 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.783 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T06:48:22.783 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T06:48:22.783 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.783 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.784 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.784 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "300", 2024-09-21T06:48:22.784 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.784 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.784 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.784 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.784 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.784 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.784 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.784 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.784 INFO:teuthology.orchestra.run.smithi067.stdout: "scrape_interval": { 2024-09-21T06:48:22.784 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "scrape_interval", 2024-09-21T06:48:22.784 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-21T06:48:22.784 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.785 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.785 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15.0", 2024-09-21T06:48:22.785 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.785 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.785 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.785 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.785 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.785 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.785 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.785 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.785 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-09-21T06:48:22.785 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-09-21T06:48:22.785 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.785 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.785 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.786 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "::", 2024-09-21T06:48:22.786 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.786 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.786 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.786 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T06:48:22.786 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.786 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.786 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.786 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.786 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-09-21T06:48:22.786 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-09-21T06:48:22.786 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.786 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.786 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.787 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "9283", 2024-09-21T06:48:22.787 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.787 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.787 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.787 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T06:48:22.787 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.787 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.787 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.787 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.787 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.787 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.787 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.787 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.787 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.787 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.788 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.788 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.788 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.788 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.788 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.788 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.788 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.788 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.788 INFO:teuthology.orchestra.run.smithi067.stdout: "stale_cache_strategy": { 2024-09-21T06:48:22.788 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "stale_cache_strategy", 2024-09-21T06:48:22.788 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.788 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.788 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.788 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "log", 2024-09-21T06:48:22.788 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.789 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.789 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.789 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.789 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.789 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.789 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.789 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.789 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_behaviour": { 2024-09-21T06:48:22.789 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_behaviour", 2024-09-21T06:48:22.789 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.789 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.789 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.789 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "default", 2024-09-21T06:48:22.789 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.790 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.790 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.790 INFO:teuthology.orchestra.run.smithi067.stdout: "default", 2024-09-21T06:48:22.790 INFO:teuthology.orchestra.run.smithi067.stdout: "error" 2024-09-21T06:48:22.790 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.790 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.790 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.790 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.790 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.790 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.790 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_error_status_code": { 2024-09-21T06:48:22.790 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_error_status_code", 2024-09-21T06:48:22.790 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.790 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.790 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.791 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-09-21T06:48:22.791 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "400", 2024-09-21T06:48:22.791 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "599", 2024-09-21T06:48:22.791 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.791 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.791 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.791 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.791 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.791 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.791 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.791 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.791 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.791 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_support", 2024-09-21T06:48:22.791 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.791 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.792 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.792 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.792 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.792 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.792 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.792 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.792 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.792 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.792 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.792 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.792 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.792 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.792 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.792 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.792 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.792 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.793 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.793 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.793 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.793 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.793 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.793 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.793 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.793 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.793 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.793 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.793 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.793 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.793 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.793 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.794 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.794 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.794 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.794 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.794 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.794 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.794 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.794 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.794 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.794 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.794 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.794 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.794 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.794 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.794 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.795 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.795 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.795 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.795 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.795 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.795 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.795 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.795 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.795 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.795 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.795 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.795 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.795 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.795 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.795 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.796 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.796 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.796 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.796 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.796 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.796 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.796 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.796 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.796 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.796 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.796 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.796 INFO:teuthology.orchestra.run.smithi067.stdout: "max_concurrent_snap_create": { 2024-09-21T06:48:22.796 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_concurrent_snap_create", 2024-09-21T06:48:22.796 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.796 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.797 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.797 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-21T06:48:22.797 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.797 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.797 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.797 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.797 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.797 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.797 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.797 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.797 INFO:teuthology.orchestra.run.smithi067.stdout: "mirror_snapshot_schedule": { 2024-09-21T06:48:22.797 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T06:48:22.797 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.798 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.798 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.798 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.798 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.798 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.798 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.798 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.798 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.798 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.798 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.798 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.798 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.798 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.798 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.798 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.799 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.799 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.799 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.799 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.799 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.799 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.799 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.799 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.799 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.799 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.799 INFO:teuthology.orchestra.run.smithi067.stdout: "trash_purge_schedule": { 2024-09-21T06:48:22.799 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "trash_purge_schedule", 2024-09-21T06:48:22.799 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.799 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.799 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.800 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.800 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.800 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.800 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.800 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.800 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.800 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.800 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.800 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.800 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.800 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.800 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.800 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "restful", 2024-09-21T06:48:22.800 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.801 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.801 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.801 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_auth": { 2024-09-21T06:48:22.801 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enable_auth", 2024-09-21T06:48:22.801 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.801 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.801 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.801 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.801 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.801 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.801 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.801 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.801 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.801 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.801 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.802 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.802 INFO:teuthology.orchestra.run.smithi067.stdout: "key_file": { 2024-09-21T06:48:22.802 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "key_file", 2024-09-21T06:48:22.802 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.802 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.802 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.802 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.802 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.802 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.802 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.802 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.802 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.802 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.802 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.802 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.803 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.803 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.803 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.803 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.803 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.803 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.803 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.803 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.803 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.803 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.803 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.803 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.803 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.803 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.804 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.804 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.804 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.804 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.804 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.804 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.804 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.804 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.804 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.804 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.804 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.804 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.804 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.804 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.804 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.804 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.805 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.805 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.805 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.805 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.805 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.805 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.805 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.805 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.805 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.805 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.805 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.805 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.805 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.805 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.806 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.806 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.806 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.806 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.806 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.806 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.806 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.806 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.806 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.806 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.806 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.806 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.806 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.806 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.807 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.807 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.807 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.807 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.807 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.807 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.807 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.807 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.807 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.807 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.807 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.807 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.807 INFO:teuthology.orchestra.run.smithi067.stdout: "max_requests": { 2024-09-21T06:48:22.807 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_requests", 2024-09-21T06:48:22.807 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.807 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.808 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.808 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-09-21T06:48:22.808 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.808 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.808 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.808 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T06:48:22.808 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.808 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.808 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.808 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.808 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-09-21T06:48:22.808 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-09-21T06:48:22.808 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.808 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.808 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.809 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.809 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.809 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.809 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.809 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.809 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.809 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.809 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.809 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.809 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-09-21T06:48:22.809 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-09-21T06:48:22.809 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.809 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.809 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.810 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.810 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.810 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.810 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.810 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.810 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.810 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.810 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.810 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.810 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.810 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.810 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.810 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.810 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.811 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.811 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.811 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.811 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.811 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.811 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.811 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.811 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.811 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.811 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.811 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.811 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.811 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rgw", 2024-09-21T06:48:22.811 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.812 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.812 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.812 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.812 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.812 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.812 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.812 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.812 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.812 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.812 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.812 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.812 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.812 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.812 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.812 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.812 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.813 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.813 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.813 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.813 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.813 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.813 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.813 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.813 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.813 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.813 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.813 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.813 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.813 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.813 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.813 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.814 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.814 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.814 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.814 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.814 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.814 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.814 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.814 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.814 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.814 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.814 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.814 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.814 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.814 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.815 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.815 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.815 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.815 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.815 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.815 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.815 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.815 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.815 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.815 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.815 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.815 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.815 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.815 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.815 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.816 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.816 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.816 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.816 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.816 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.816 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.816 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.816 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.816 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.816 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.816 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.816 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.816 INFO:teuthology.orchestra.run.smithi067.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T06:48:22.816 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T06:48:22.816 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.817 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.817 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.817 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-09-21T06:48:22.817 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.817 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.817 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.817 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T06:48:22.817 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.817 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.817 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.817 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.817 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.817 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.817 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.817 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.818 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.818 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.818 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.818 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.818 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.818 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.818 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.818 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.818 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.818 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.818 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.818 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.818 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.818 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rook", 2024-09-21T06:48:22.819 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.819 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.819 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.819 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.819 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.819 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.819 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.819 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.819 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.819 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.819 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.819 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.819 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.819 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.819 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.820 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.820 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.820 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.820 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.820 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.820 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.820 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.820 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.820 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.820 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.820 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.820 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.820 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.820 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.821 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.821 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.821 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.821 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.821 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.821 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.821 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.821 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.821 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.821 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.821 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.821 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.821 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.821 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.821 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.821 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.822 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.822 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.822 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.822 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.822 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.822 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.822 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.822 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.822 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.822 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.822 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.822 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.822 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.823 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.823 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.823 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.823 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.823 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.823 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.823 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.823 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.823 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.823 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.823 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.823 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.823 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.823 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.823 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.823 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.824 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.824 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.824 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.824 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.824 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.824 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.824 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.824 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.824 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.824 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.824 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.824 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.824 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.824 INFO:teuthology.orchestra.run.smithi067.stdout: "storage_class": { 2024-09-21T06:48:22.825 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "storage_class", 2024-09-21T06:48:22.825 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.825 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.826 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.826 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "local", 2024-09-21T06:48:22.826 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.826 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.826 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.826 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T06:48:22.826 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.826 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.826 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.826 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.826 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.826 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.827 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.827 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "selftest", 2024-09-21T06:48:22.827 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.827 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.827 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.827 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.827 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.827 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.827 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.827 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.827 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.827 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.827 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.827 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.827 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.828 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.828 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.828 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.828 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.828 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.828 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.828 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.828 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.828 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.828 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.828 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.828 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.828 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.828 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.828 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.829 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.829 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.829 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.829 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.829 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.829 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.829 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.829 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.829 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.829 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.829 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.829 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.829 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.829 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.829 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.829 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.830 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.830 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.830 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.830 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.830 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.830 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.830 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.830 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.830 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.830 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.830 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.830 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.830 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.830 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.830 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.831 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.831 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.831 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.831 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.831 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.831 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.831 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.831 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.831 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.831 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.831 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.831 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.831 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.831 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.832 INFO:teuthology.orchestra.run.smithi067.stdout: "roption1": { 2024-09-21T06:48:22.832 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "roption1", 2024-09-21T06:48:22.832 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.832 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.832 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.832 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.832 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.832 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.832 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.832 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.832 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.832 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.832 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.832 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.832 INFO:teuthology.orchestra.run.smithi067.stdout: "roption2": { 2024-09-21T06:48:22.832 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "roption2", 2024-09-21T06:48:22.832 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.833 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.833 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.833 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "xyz", 2024-09-21T06:48:22.833 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.833 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.833 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.833 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.833 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.833 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.833 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.833 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.833 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption1": { 2024-09-21T06:48:22.833 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption1", 2024-09-21T06:48:22.833 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.833 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.834 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.834 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.834 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.834 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.834 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.834 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.834 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.834 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.834 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.834 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.834 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption2": { 2024-09-21T06:48:22.834 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption2", 2024-09-21T06:48:22.834 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.834 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.834 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.835 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.835 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.835 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.835 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.835 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.835 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.835 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.835 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.835 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.835 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption3": { 2024-09-21T06:48:22.835 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption3", 2024-09-21T06:48:22.835 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-21T06:48:22.835 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.835 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.835 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.836 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.836 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.836 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.836 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.836 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.836 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.836 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.836 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.836 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption4": { 2024-09-21T06:48:22.836 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption4", 2024-09-21T06:48:22.836 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.836 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.836 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.836 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.837 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.837 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.837 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.837 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.837 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.837 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.837 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.837 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.837 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption5": { 2024-09-21T06:48:22.837 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption5", 2024-09-21T06:48:22.837 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.837 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.837 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.837 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.837 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.838 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.838 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.838 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.838 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.838 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.838 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.838 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.838 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption6": { 2024-09-21T06:48:22.838 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption6", 2024-09-21T06:48:22.838 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.838 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.838 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.838 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.838 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.838 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.838 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.839 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.839 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.839 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.839 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.839 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.839 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption7": { 2024-09-21T06:48:22.839 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption7", 2024-09-21T06:48:22.839 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.839 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.839 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.839 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.839 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-09-21T06:48:22.839 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "42", 2024-09-21T06:48:22.839 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.840 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.840 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.840 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.840 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.840 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.840 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.840 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.840 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.840 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.840 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.840 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.840 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.840 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.840 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.840 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.841 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.841 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.841 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.841 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.841 INFO:teuthology.orchestra.run.smithi067.stdout: "testkey": { 2024-09-21T06:48:22.841 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testkey", 2024-09-21T06:48:22.841 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.841 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.841 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.841 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.841 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.841 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.841 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.841 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.842 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.842 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.842 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.842 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.842 INFO:teuthology.orchestra.run.smithi067.stdout: "testlkey": { 2024-09-21T06:48:22.842 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testlkey", 2024-09-21T06:48:22.842 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.842 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.842 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.842 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.842 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.842 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.842 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.842 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.842 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.843 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.843 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.843 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.843 INFO:teuthology.orchestra.run.smithi067.stdout: "testnewline": { 2024-09-21T06:48:22.843 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testnewline", 2024-09-21T06:48:22.843 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.843 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.843 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.843 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.843 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.843 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.843 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.843 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.843 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.843 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.844 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.844 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.844 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.844 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.844 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.844 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smb", 2024-09-21T06:48:22.844 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.844 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.844 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.844 INFO:teuthology.orchestra.run.smithi067.stdout: "internal_store_backend": { 2024-09-21T06:48:22.844 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "internal_store_backend", 2024-09-21T06:48:22.844 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.844 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.844 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.845 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.845 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.845 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.845 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.845 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T06:48:22.845 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.845 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.845 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.845 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.845 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.845 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.845 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.845 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.845 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.845 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.846 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.846 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.846 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.846 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.846 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.846 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.846 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.846 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.846 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.846 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.846 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.846 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.846 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.846 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.846 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.847 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.847 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.847 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.847 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.847 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.847 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.847 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.847 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.847 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.847 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.847 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.847 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.847 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.847 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.848 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.848 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.848 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.848 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.848 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.848 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.848 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.848 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.848 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.848 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.848 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.848 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.848 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.848 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.848 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.849 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.849 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.849 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.849 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.849 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.849 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.849 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.849 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.849 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.849 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.849 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.849 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.849 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.849 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.849 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.850 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.850 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.850 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.850 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.850 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.850 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.850 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.850 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.850 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.850 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.850 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.850 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.850 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.850 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.851 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.851 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.851 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.851 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.851 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.851 INFO:teuthology.orchestra.run.smithi067.stdout: "update_orchestration": { 2024-09-21T06:48:22.851 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "update_orchestration", 2024-09-21T06:48:22.851 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.851 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.851 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.851 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.851 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.851 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.851 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.851 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T06:48:22.852 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.852 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.852 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.852 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.852 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.852 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.852 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.852 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snap_schedule", 2024-09-21T06:48:22.852 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.852 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.852 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.852 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_m_granularity": { 2024-09-21T06:48:22.852 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_m_granularity", 2024-09-21T06:48:22.852 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.852 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.853 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.853 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.853 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.853 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.853 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.853 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T06:48:22.853 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.853 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.853 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.853 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.853 INFO:teuthology.orchestra.run.smithi067.stdout: "dump_on_update": { 2024-09-21T06:48:22.853 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "dump_on_update", 2024-09-21T06:48:22.853 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.853 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.854 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.854 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.854 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.854 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.854 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.854 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "dump database to debug log on update", 2024-09-21T06:48:22.854 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.854 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.854 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.854 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.854 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.854 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.854 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.854 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.854 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.855 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.855 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.855 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.855 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.855 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.855 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.855 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.855 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.855 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.855 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.855 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.855 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.855 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.855 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.856 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.856 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.856 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.856 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.856 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.856 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.856 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.856 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.856 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.856 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.856 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.856 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.856 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.856 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.856 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.857 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.857 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.857 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.857 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.857 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.857 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.857 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.857 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.857 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.857 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.857 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.857 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.857 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.857 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.858 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.858 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.858 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.858 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.858 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.858 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.858 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.858 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.858 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.858 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.858 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.858 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.858 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.858 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.859 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.859 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.859 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.859 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.859 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.859 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.859 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.859 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.859 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.859 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.859 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.859 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.859 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.859 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.860 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.860 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.860 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.860 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.860 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.860 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.860 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.860 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.860 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.860 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.860 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.860 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "stats", 2024-09-21T06:48:22.860 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.860 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.860 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.860 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.861 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.861 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.861 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.861 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.861 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.861 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.861 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.861 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.861 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.861 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.861 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.861 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.861 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.861 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.862 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.862 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.862 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.862 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.862 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.862 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.862 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.862 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.862 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.862 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.862 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.862 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.862 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.862 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.863 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.863 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.863 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.863 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.863 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.863 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.863 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.863 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.863 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.863 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.863 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.863 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.863 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.863 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.864 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.864 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.864 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.864 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.864 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.864 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.864 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.864 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.864 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.864 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.864 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.864 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.864 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.864 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.865 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.865 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.865 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.865 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.865 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.865 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.865 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.865 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.865 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.865 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.865 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.865 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.865 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.866 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.866 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.866 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.866 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.866 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.866 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.866 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.866 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.866 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.866 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.866 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.866 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.866 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.866 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.866 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.867 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.867 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.867 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "status", 2024-09-21T06:48:22.867 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.867 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.867 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.867 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.867 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.867 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.867 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.867 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.867 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.867 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.867 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.867 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.867 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.868 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.868 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.868 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.868 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.868 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.868 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.868 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.868 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.868 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.868 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.868 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.868 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.868 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.868 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.869 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.869 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.869 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.869 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.869 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.869 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.869 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.869 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.869 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.869 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.869 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.869 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.869 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.869 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.870 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.870 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.870 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.870 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.870 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.870 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.870 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.870 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.870 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.870 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.870 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.870 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.870 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.870 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.871 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.871 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.871 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.871 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.871 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.871 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.871 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.871 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.871 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.871 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.871 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.871 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.871 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.871 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.872 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.872 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.872 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.872 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.872 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.872 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.872 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.872 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.872 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.872 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.872 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.872 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.872 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.872 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.873 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.873 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.873 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.873 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.873 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.873 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.873 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.873 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "telegraf", 2024-09-21T06:48:22.873 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.873 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.873 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.873 INFO:teuthology.orchestra.run.smithi067.stdout: "address": { 2024-09-21T06:48:22.873 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "address", 2024-09-21T06:48:22.873 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.873 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.874 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.874 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T06:48:22.874 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.874 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.874 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.874 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.874 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.874 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.874 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.874 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.874 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-09-21T06:48:22.874 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-09-21T06:48:22.874 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.874 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.875 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.875 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15", 2024-09-21T06:48:22.875 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.875 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.875 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.875 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.875 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.875 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.875 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.875 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.875 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.875 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.875 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.875 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.876 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.876 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.876 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.876 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.876 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.876 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.876 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.876 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.876 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.876 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.876 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.876 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.876 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.876 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.877 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.877 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.877 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.877 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.877 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.877 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.877 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.877 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.877 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.877 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.877 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.877 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.877 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.877 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.878 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.878 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.878 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.878 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.878 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.878 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.878 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.878 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.878 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.878 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.878 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.878 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.878 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.879 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.879 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.879 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.879 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.879 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.879 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.879 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.879 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.879 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.879 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.879 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.879 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.879 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.879 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.879 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.879 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.880 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.880 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.880 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.880 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.880 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.880 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.880 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.880 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.880 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.880 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.880 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.880 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.880 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.880 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.880 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.881 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.881 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.881 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.881 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.881 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.881 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.881 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.881 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.881 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.881 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.881 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.881 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "telemetry", 2024-09-21T06:48:22.881 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.881 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.882 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.882 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_basic": { 2024-09-21T06:48:22.882 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_basic", 2024-09-21T06:48:22.882 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.882 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.882 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.882 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.882 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.882 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.882 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.882 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T06:48:22.882 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.882 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.882 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.882 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.883 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_crash": { 2024-09-21T06:48:22.883 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_crash", 2024-09-21T06:48:22.883 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.883 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.883 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.883 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.883 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.883 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.883 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.883 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T06:48:22.883 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.883 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.883 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.883 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.884 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_device": { 2024-09-21T06:48:22.884 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_device", 2024-09-21T06:48:22.884 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.884 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.884 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.884 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.884 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.884 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.884 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.884 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T06:48:22.884 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.884 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.884 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.884 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.884 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_ident": { 2024-09-21T06:48:22.885 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_ident", 2024-09-21T06:48:22.885 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.885 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.885 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.885 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.885 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.885 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.885 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.885 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T06:48:22.885 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.885 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.885 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.885 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.885 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_perf": { 2024-09-21T06:48:22.885 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_perf", 2024-09-21T06:48:22.885 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.886 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.886 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.886 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.886 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.886 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.886 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.886 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T06:48:22.886 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.886 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.886 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.886 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.886 INFO:teuthology.orchestra.run.smithi067.stdout: "contact": { 2024-09-21T06:48:22.886 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "contact", 2024-09-21T06:48:22.886 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.887 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.887 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.887 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.887 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.887 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.887 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.887 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.887 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.887 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.887 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.887 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.887 INFO:teuthology.orchestra.run.smithi067.stdout: "description": { 2024-09-21T06:48:22.887 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "description", 2024-09-21T06:48:22.887 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.888 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.888 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.888 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.888 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.888 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.888 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.888 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.888 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.888 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.888 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.888 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.888 INFO:teuthology.orchestra.run.smithi067.stdout: "device_url": { 2024-09-21T06:48:22.888 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_url", 2024-09-21T06:48:22.888 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.889 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.889 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.889 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T06:48:22.889 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.889 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.889 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.889 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.889 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.889 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.889 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.889 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.889 INFO:teuthology.orchestra.run.smithi067.stdout: "enabled": { 2024-09-21T06:48:22.889 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enabled", 2024-09-21T06:48:22.889 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.889 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.890 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.890 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.890 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.890 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.890 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.890 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.890 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.890 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.890 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.890 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.890 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-09-21T06:48:22.890 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-09-21T06:48:22.890 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.890 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.891 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.891 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "24", 2024-09-21T06:48:22.891 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "8", 2024-09-21T06:48:22.891 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.891 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.891 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.891 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.891 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.891 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.891 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.891 INFO:teuthology.orchestra.run.smithi067.stdout: "last_opt_revision": { 2024-09-21T06:48:22.891 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "last_opt_revision", 2024-09-21T06:48:22.891 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.891 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.892 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.892 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1", 2024-09-21T06:48:22.892 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.892 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.892 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.892 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.892 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.892 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.892 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.892 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.892 INFO:teuthology.orchestra.run.smithi067.stdout: "leaderboard": { 2024-09-21T06:48:22.892 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "leaderboard", 2024-09-21T06:48:22.892 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.892 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.893 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.893 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.893 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.893 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.893 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.893 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.893 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.893 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.893 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.893 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.893 INFO:teuthology.orchestra.run.smithi067.stdout: "leaderboard_description": { 2024-09-21T06:48:22.893 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "leaderboard_description", 2024-09-21T06:48:22.893 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.893 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.894 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.894 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.894 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.894 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.894 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.894 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.894 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.894 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.894 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.894 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.894 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.894 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.894 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.894 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.895 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.895 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.895 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.895 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.895 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.895 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.895 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.895 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.895 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.895 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.895 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.895 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.895 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.895 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.896 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.896 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.896 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.896 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.896 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.896 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.896 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.896 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.896 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.896 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.896 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.896 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.896 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.896 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.897 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.897 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.897 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.897 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.897 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.897 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.897 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.897 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.897 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.897 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.897 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.897 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.897 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.897 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.898 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.898 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.898 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.898 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.898 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.898 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.898 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.898 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.898 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.898 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.898 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.898 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.898 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.898 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.899 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.899 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.899 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.899 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.899 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.899 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.899 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.899 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.899 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.899 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.899 INFO:teuthology.orchestra.run.smithi067.stdout: "organization": { 2024-09-21T06:48:22.899 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "organization", 2024-09-21T06:48:22.899 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.899 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.900 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.900 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.900 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.900 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.900 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.900 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.900 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.900 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.900 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.900 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.900 INFO:teuthology.orchestra.run.smithi067.stdout: "proxy": { 2024-09-21T06:48:22.900 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "proxy", 2024-09-21T06:48:22.900 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.900 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.901 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.901 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.901 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.901 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.901 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.901 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.901 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.901 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.901 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.901 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.901 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.901 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.901 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.901 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.902 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.902 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.902 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.902 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.902 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.902 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.902 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.902 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.902 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.902 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.902 INFO:teuthology.orchestra.run.smithi067.stdout: "url": { 2024-09-21T06:48:22.902 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "url", 2024-09-21T06:48:22.902 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.902 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.902 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.903 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T06:48:22.903 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.903 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.903 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.903 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.903 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.903 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.903 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.903 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.903 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.903 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.903 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.903 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "test_orchestrator", 2024-09-21T06:48:22.903 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.903 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.903 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.904 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.904 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.904 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.904 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.904 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.904 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.904 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.904 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.904 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.904 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.904 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.904 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.904 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.904 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.904 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.904 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.905 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.905 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.905 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.905 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.905 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.905 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.905 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.905 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.905 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.905 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.905 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.905 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.905 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.905 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.905 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.905 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.905 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.906 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.906 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.906 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.906 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.906 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.906 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.906 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.906 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.906 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.906 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.906 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.906 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.906 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.906 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.906 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.907 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.907 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.907 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.907 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.907 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.907 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.907 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.907 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.907 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.907 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.907 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.907 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.907 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.907 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.907 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.907 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.908 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.908 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.908 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.908 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.908 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.908 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.908 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.908 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.908 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.908 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.908 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.908 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.908 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.908 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.908 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.908 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.909 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.909 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.909 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.909 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.909 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.909 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.909 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.909 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "volumes", 2024-09-21T06:48:22.909 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.909 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.909 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.909 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.909 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.909 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.909 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.910 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.910 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.910 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.910 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.910 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.910 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.910 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.910 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.910 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.910 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.910 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.910 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.910 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.910 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.910 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.911 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.911 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.911 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.911 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.911 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.911 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.911 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.911 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.911 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.911 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.911 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.911 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.911 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.911 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.912 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.912 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.912 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.912 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.912 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.912 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.912 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.912 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.912 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.912 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.912 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.912 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.912 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.912 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.913 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.913 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.913 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.913 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.913 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.913 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.913 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.913 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.913 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.913 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.913 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.913 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.913 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.913 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.913 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.914 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.914 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.914 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.914 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.914 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.914 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.914 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.914 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.914 INFO:teuthology.orchestra.run.smithi067.stdout: "max_concurrent_clones": { 2024-09-21T06:48:22.914 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_concurrent_clones", 2024-09-21T06:48:22.914 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.914 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.914 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.914 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "4", 2024-09-21T06:48:22.915 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.915 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.915 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.915 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T06:48:22.915 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.915 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.915 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.915 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.915 INFO:teuthology.orchestra.run.smithi067.stdout: "periodic_async_work": { 2024-09-21T06:48:22.915 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "periodic_async_work", 2024-09-21T06:48:22.915 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.915 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.915 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.915 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.915 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.915 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.916 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.916 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Periodically check for async work", 2024-09-21T06:48:22.916 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.916 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.916 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.916 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.916 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot_clone_delay": { 2024-09-21T06:48:22.916 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snapshot_clone_delay", 2024-09-21T06:48:22.916 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.916 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.916 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.916 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.916 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.916 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.916 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.917 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T06:48:22.917 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.917 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.917 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.917 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.917 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot_clone_no_wait": { 2024-09-21T06:48:22.917 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T06:48:22.917 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.917 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.917 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.917 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-21T06:48:22.917 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.917 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.917 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.917 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T06:48:22.918 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.918 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.918 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.918 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.918 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.918 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.918 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.918 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.918 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.918 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.918 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.918 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.918 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.918 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.918 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.919 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.919 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.919 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.919 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.919 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.919 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.919 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix", 2024-09-21T06:48:22.919 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-21T06:48:22.919 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-21T06:48:22.919 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-21T06:48:22.919 INFO:teuthology.orchestra.run.smithi067.stdout: "discovery_interval": { 2024-09-21T06:48:22.919 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "discovery_interval", 2024-09-21T06:48:22.919 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-09-21T06:48:22.919 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.919 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.920 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "100", 2024-09-21T06:48:22.920 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.920 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.920 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.920 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.920 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.920 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.920 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.920 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.920 INFO:teuthology.orchestra.run.smithi067.stdout: "identifier": { 2024-09-21T06:48:22.920 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "identifier", 2024-09-21T06:48:22.920 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.920 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.920 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.920 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.921 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.921 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.921 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.921 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.921 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.921 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.921 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.921 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.921 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-09-21T06:48:22.921 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-09-21T06:48:22.921 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-21T06:48:22.921 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.921 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.921 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-09-21T06:48:22.921 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.921 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.922 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.922 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.922 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.922 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.922 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.922 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.922 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-21T06:48:22.922 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-21T06:48:22.922 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.922 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.922 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.922 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.922 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.922 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.922 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.923 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.923 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.923 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.923 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.923 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.923 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.923 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.923 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.923 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.923 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.923 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.923 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.923 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-21T06:48:22.923 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-21T06:48:22.923 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.924 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.924 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.924 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.924 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.924 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.924 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.924 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.924 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.924 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.924 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.924 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.924 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-21T06:48:22.924 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-21T06:48:22.924 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.924 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.925 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.925 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-21T06:48:22.925 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.925 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.925 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-21T06:48:22.925 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-21T06:48:22.925 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-21T06:48:22.925 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-21T06:48:22.925 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-21T06:48:22.925 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-21T06:48:22.925 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-21T06:48:22.925 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.925 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.925 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.926 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.926 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.926 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.926 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-21T06:48:22.926 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-21T06:48:22.926 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-21T06:48:22.926 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.926 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.926 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-21T06:48:22.926 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.926 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.926 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.926 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.927 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.927 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.927 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.927 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.927 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-21T06:48:22.927 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:48:22.927 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.927 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-21T06:48:22.927 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-21T06:48:22.927 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-21T06:48:22.927 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.927 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.927 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.927 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.928 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.928 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.928 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.928 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.928 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_host": { 2024-09-21T06:48:22.928 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_host", 2024-09-21T06:48:22.928 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.928 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.928 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.928 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-21T06:48:22.928 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.928 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.928 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.928 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.929 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.929 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.929 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.929 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.929 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_port": { 2024-09-21T06:48:22.929 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_port", 2024-09-21T06:48:22.929 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-21T06:48:22.929 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.929 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.929 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10051", 2024-09-21T06:48:22.929 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.929 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.929 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.929 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.929 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.930 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.930 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.930 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.930 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_sender": { 2024-09-21T06:48:22.930 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_sender", 2024-09-21T06:48:22.930 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-21T06:48:22.930 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-21T06:48:22.930 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-21T06:48:22.930 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T06:48:22.930 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-21T06:48:22.930 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-21T06:48:22.930 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-21T06:48:22.930 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-21T06:48:22.930 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-21T06:48:22.931 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-21T06:48:22.931 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-21T06:48:22.931 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.931 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.931 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.931 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.931 INFO:teuthology.orchestra.run.smithi067.stdout: "services": { 2024-09-21T06:48:22.931 INFO:teuthology.orchestra.run.smithi067.stdout: "dashboard": "https://172.21.15.67:8443/" 2024-09-21T06:48:22.931 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.931 INFO:teuthology.orchestra.run.smithi067.stdout: "always_on_modules": { 2024-09-21T06:48:22.931 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus": [ 2024-09-21T06:48:22.931 INFO:teuthology.orchestra.run.smithi067.stdout: "balancer", 2024-09-21T06:48:22.931 INFO:teuthology.orchestra.run.smithi067.stdout: "crash", 2024-09-21T06:48:22.931 INFO:teuthology.orchestra.run.smithi067.stdout: "devicehealth", 2024-09-21T06:48:22.931 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator", 2024-09-21T06:48:22.932 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_autoscaler", 2024-09-21T06:48:22.932 INFO:teuthology.orchestra.run.smithi067.stdout: "progress", 2024-09-21T06:48:22.932 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_support", 2024-09-21T06:48:22.932 INFO:teuthology.orchestra.run.smithi067.stdout: "status", 2024-09-21T06:48:22.932 INFO:teuthology.orchestra.run.smithi067.stdout: "telemetry", 2024-09-21T06:48:22.932 INFO:teuthology.orchestra.run.smithi067.stdout: "volumes" 2024-09-21T06:48:22.932 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.932 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific": [ 2024-09-21T06:48:22.932 INFO:teuthology.orchestra.run.smithi067.stdout: "balancer", 2024-09-21T06:48:22.932 INFO:teuthology.orchestra.run.smithi067.stdout: "crash", 2024-09-21T06:48:22.932 INFO:teuthology.orchestra.run.smithi067.stdout: "devicehealth", 2024-09-21T06:48:22.932 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator", 2024-09-21T06:48:22.932 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_autoscaler", 2024-09-21T06:48:22.932 INFO:teuthology.orchestra.run.smithi067.stdout: "progress", 2024-09-21T06:48:22.933 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_support", 2024-09-21T06:48:22.933 INFO:teuthology.orchestra.run.smithi067.stdout: "status", 2024-09-21T06:48:22.933 INFO:teuthology.orchestra.run.smithi067.stdout: "telemetry", 2024-09-21T06:48:22.933 INFO:teuthology.orchestra.run.smithi067.stdout: "volumes" 2024-09-21T06:48:22.933 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.933 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy": [ 2024-09-21T06:48:22.933 INFO:teuthology.orchestra.run.smithi067.stdout: "balancer", 2024-09-21T06:48:22.933 INFO:teuthology.orchestra.run.smithi067.stdout: "crash", 2024-09-21T06:48:22.933 INFO:teuthology.orchestra.run.smithi067.stdout: "devicehealth", 2024-09-21T06:48:22.933 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator", 2024-09-21T06:48:22.933 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_autoscaler", 2024-09-21T06:48:22.933 INFO:teuthology.orchestra.run.smithi067.stdout: "progress", 2024-09-21T06:48:22.933 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_support", 2024-09-21T06:48:22.933 INFO:teuthology.orchestra.run.smithi067.stdout: "status", 2024-09-21T06:48:22.933 INFO:teuthology.orchestra.run.smithi067.stdout: "telemetry", 2024-09-21T06:48:22.934 INFO:teuthology.orchestra.run.smithi067.stdout: "volumes" 2024-09-21T06:48:22.934 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.934 INFO:teuthology.orchestra.run.smithi067.stdout: "reef": [ 2024-09-21T06:48:22.934 INFO:teuthology.orchestra.run.smithi067.stdout: "balancer", 2024-09-21T06:48:22.934 INFO:teuthology.orchestra.run.smithi067.stdout: "crash", 2024-09-21T06:48:22.934 INFO:teuthology.orchestra.run.smithi067.stdout: "devicehealth", 2024-09-21T06:48:22.934 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator", 2024-09-21T06:48:22.934 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_autoscaler", 2024-09-21T06:48:22.934 INFO:teuthology.orchestra.run.smithi067.stdout: "progress", 2024-09-21T06:48:22.934 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_support", 2024-09-21T06:48:22.934 INFO:teuthology.orchestra.run.smithi067.stdout: "status", 2024-09-21T06:48:22.934 INFO:teuthology.orchestra.run.smithi067.stdout: "telemetry", 2024-09-21T06:48:22.934 INFO:teuthology.orchestra.run.smithi067.stdout: "volumes" 2024-09-21T06:48:22.934 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-21T06:48:22.935 INFO:teuthology.orchestra.run.smithi067.stdout: "squid": [ 2024-09-21T06:48:22.935 INFO:teuthology.orchestra.run.smithi067.stdout: "balancer", 2024-09-21T06:48:22.935 INFO:teuthology.orchestra.run.smithi067.stdout: "crash", 2024-09-21T06:48:22.935 INFO:teuthology.orchestra.run.smithi067.stdout: "devicehealth", 2024-09-21T06:48:22.935 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator", 2024-09-21T06:48:22.935 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_autoscaler", 2024-09-21T06:48:22.935 INFO:teuthology.orchestra.run.smithi067.stdout: "progress", 2024-09-21T06:48:22.935 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_support", 2024-09-21T06:48:22.935 INFO:teuthology.orchestra.run.smithi067.stdout: "status", 2024-09-21T06:48:22.935 INFO:teuthology.orchestra.run.smithi067.stdout: "telemetry", 2024-09-21T06:48:22.935 INFO:teuthology.orchestra.run.smithi067.stdout: "volumes" 2024-09-21T06:48:22.935 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-21T06:48:22.935 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.935 INFO:teuthology.orchestra.run.smithi067.stdout: "last_failure_osd_epoch": 3, 2024-09-21T06:48:22.935 INFO:teuthology.orchestra.run.smithi067.stdout: "active_clients": [ 2024-09-21T06:48:22.936 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.936 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "devicehealth", 2024-09-21T06:48:22.936 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-09-21T06:48:22.936 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.936 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-09-21T06:48:22.936 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:0", 2024-09-21T06:48:22.936 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 1315043503 2024-09-21T06:48:22.936 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.936 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-21T06:48:22.936 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.936 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.936 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "libcephsqlite", 2024-09-21T06:48:22.936 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-09-21T06:48:22.936 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.936 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-09-21T06:48:22.937 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:0", 2024-09-21T06:48:22.937 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 2762272475 2024-09-21T06:48:22.937 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.937 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-21T06:48:22.937 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.937 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.937 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_support", 2024-09-21T06:48:22.937 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-09-21T06:48:22.937 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.937 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-09-21T06:48:22.937 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:0", 2024-09-21T06:48:22.937 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 2644215278 2024-09-21T06:48:22.937 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.937 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-21T06:48:22.937 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-21T06:48:22.938 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.938 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "volumes", 2024-09-21T06:48:22.938 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-09-21T06:48:22.938 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-21T06:48:22.938 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-09-21T06:48:22.938 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:0", 2024-09-21T06:48:22.938 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 993849907 2024-09-21T06:48:22.938 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.938 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-21T06:48:22.938 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-21T06:48:22.938 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-21T06:48:22.938 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-21T06:48:22.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.510+0000 7f7f8c669640 1 -- 172.21.15.67:0/652228116 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f7f700b6590 msgr2=0x7f7f700b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:22.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.510+0000 7f7f8c669640 1 --2- 172.21.15.67:0/652228116 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f7f700b6590 0x7f7f700b8aa0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7f80005f70 tx=0x7f7f80000bc0 comp rx=0 tx=0).stop 2024-09-21T06:48:22.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.510+0000 7f7f8c669640 1 -- 172.21.15.67:0/652228116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f84103fc0 msgr2=0x7f7f841a10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:22.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.510+0000 7f7f8c669640 1 --2- 172.21.15.67:0/652228116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f84103fc0 0x7f7f841a10f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c00eb40 tx=0x7f7f7c00c2d0 comp rx=0 tx=0).stop 2024-09-21T06:48:22.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.510+0000 7f7f8c669640 1 -- 172.21.15.67:0/652228116 shutdown_connections 2024-09-21T06:48:22.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.510+0000 7f7f8c669640 1 --2- 172.21.15.67:0/652228116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f84103620 0x7f7f8419eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:22.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.510+0000 7f7f8c669640 1 --2- 172.21.15.67:0/652228116 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f7f700b6590 0x7f7f700b8aa0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:22.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.510+0000 7f7f8c669640 1 --2- 172.21.15.67:0/652228116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f84103fc0 0x7f7f841a10f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:22.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.510+0000 7f7f8c669640 1 --2- 172.21.15.67:0/652228116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f841025a0 0x7f7f8406c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:22.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.510+0000 7f7f8c669640 1 -- 172.21.15.67:0/652228116 >> 172.21.15.67:0/652228116 conn(0x7f7f840fdee0 msgr2=0x7f7f84106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:22.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.511+0000 7f7f8c669640 1 -- 172.21.15.67:0/652228116 shutdown_connections 2024-09-21T06:48:22.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:21.511+0000 7f7f8c669640 1 -- 172.21.15.67:0/652228116 wait complete. 2024-09-21T06:48:22.987 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-21T06:48:23.002 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-21T06:48:23.003 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-21T06:48:23.003 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:48:23.003 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:21 smithi067 ceph-mon[31232]: pgmap v713: 129 pgs: 129 active+clean; 2.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 11 MiB/s wr, 392 op/s 2024-09-21T06:48:23.003 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:21 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1566856140' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T06:48:23.003 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:21 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/661546867' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-21T06:48:23.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:21 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/652228116' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T06:48:23.004 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:21 smithi100 ceph-mon[33401]: pgmap v713: 129 pgs: 129 active+clean; 2.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 11 MiB/s wr, 392 op/s 2024-09-21T06:48:23.004 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:21 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1566856140' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T06:48:23.005 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:21 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/661546867' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-21T06:48:23.005 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:21 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/652228116' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T06:48:23.005 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:21 smithi203 ceph-mon[33735]: pgmap v713: 129 pgs: 129 active+clean; 2.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 11 MiB/s wr, 392 op/s 2024-09-21T06:48:23.005 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:21 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1566856140' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T06:48:23.005 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:21 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/661546867' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-21T06:48:23.005 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:21 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/652228116' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T06:48:23.034 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-21T06:48:23.034 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:48:23.094 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T06:48:23.402 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:48:23.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:23 smithi067 ceph-mon[31232]: pgmap v714: 129 pgs: 129 active+clean; 109 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.8 MiB/s wr, 406 op/s 2024-09-21T06:48:23.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.926+0000 7fc3398fd640 1 -- 172.21.15.67:0/2563195704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc334108cf0 msgr2=0x7fc33410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:23.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.926+0000 7fc3398fd640 1 --2- 172.21.15.67:0/2563195704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc334108cf0 0x7fc33410f520 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc324009900 tx=0x7fc32402a3f0 comp rx=0 tx=0).stop 2024-09-21T06:48:23.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.926+0000 7fc3398fd640 1 -- 172.21.15.67:0/2563195704 shutdown_connections 2024-09-21T06:48:23.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.926+0000 7fc3398fd640 1 --2- 172.21.15.67:0/2563195704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3341083a0 0x7fc3341087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:23.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.926+0000 7fc3398fd640 1 --2- 172.21.15.67:0/2563195704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc334108cf0 0x7fc33410f520 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:23.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.926+0000 7fc3398fd640 1 --2- 172.21.15.67:0/2563195704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc334102a60 0x7fc334102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:23.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.926+0000 7fc3398fd640 1 -- 172.21.15.67:0/2563195704 >> 172.21.15.67:0/2563195704 conn(0x7fc334079730 msgr2=0x7fc334079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:23.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.926+0000 7fc3398fd640 1 -- 172.21.15.67:0/2563195704 shutdown_connections 2024-09-21T06:48:23.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.926+0000 7fc3398fd640 1 -- 172.21.15.67:0/2563195704 wait complete. 2024-09-21T06:48:23.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.926+0000 7fc3398fd640 1 Processor -- start 2024-09-21T06:48:23.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.927+0000 7fc3398fd640 1 -- start start 2024-09-21T06:48:23.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.927+0000 7fc3398fd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc334102a60 0x7fc33419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:23.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.927+0000 7fc3398fd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3341083a0 0x7fc33419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:23.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.927+0000 7fc3398fd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc334108cf0 0x7fc33419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:23.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.927+0000 7fc3398fd640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc334113930 con 0x7fc3341083a0 2024-09-21T06:48:23.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.927+0000 7fc3398fd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc3341137b0 con 0x7fc334108cf0 2024-09-21T06:48:23.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.927+0000 7fc3398fd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc334113ab0 con 0x7fc334102a60 2024-09-21T06:48:23.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.927+0000 7fc3327fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3341083a0 0x7fc33419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:23.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.927+0000 7fc3327fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3341083a0 0x7fc33419a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54740/0 (socket says 172.21.15.67:54740) 2024-09-21T06:48:23.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.927+0000 7fc3327fc640 1 -- 172.21.15.67:0/935798624 learned_addr learned my addr 172.21.15.67:0/935798624 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:48:23.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.927+0000 7fc332ffd640 1 --2- 172.21.15.67:0/935798624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc334102a60 0x7fc33419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:23.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.927+0000 7fc3337fe640 1 --2- 172.21.15.67:0/935798624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc334108cf0 0x7fc33419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:23.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.927+0000 7fc3327fc640 1 -- 172.21.15.67:0/935798624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc334102a60 msgr2=0x7fc33419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:23.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.927+0000 7fc3327fc640 1 --2- 172.21.15.67:0/935798624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc334102a60 0x7fc33419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:23.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.927+0000 7fc3327fc640 1 -- 172.21.15.67:0/935798624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc334108cf0 msgr2=0x7fc33419fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:23.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.927+0000 7fc3327fc640 1 --2- 172.21.15.67:0/935798624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc334108cf0 0x7fc33419fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:23.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.927+0000 7fc3327fc640 1 -- 172.21.15.67:0/935798624 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3341a02d0 con 0x7fc3341083a0 2024-09-21T06:48:23.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.928+0000 7fc3327fc640 1 --2- 172.21.15.67:0/935798624 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3341083a0 0x7fc33419a9c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc31c00c970 tx=0x7fc31c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:23.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.928+0000 7fc3388fb640 1 -- 172.21.15.67:0/935798624 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c014070 con 0x7fc3341083a0 2024-09-21T06:48:23.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.928+0000 7fc3398fd640 1 -- 172.21.15.67:0/935798624 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3341a05c0 con 0x7fc3341083a0 2024-09-21T06:48:23.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.928+0000 7fc3388fb640 1 -- 172.21.15.67:0/935798624 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc31c004440 con 0x7fc3341083a0 2024-09-21T06:48:23.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.928+0000 7fc3388fb640 1 -- 172.21.15.67:0/935798624 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c003d40 con 0x7fc3341083a0 2024-09-21T06:48:23.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.928+0000 7fc3398fd640 1 -- 172.21.15.67:0/935798624 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3341a6de0 con 0x7fc3341083a0 2024-09-21T06:48:23.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.929+0000 7fc3398fd640 1 -- 172.21.15.67:0/935798624 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc334102e70 con 0x7fc3341083a0 2024-09-21T06:48:23.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.930+0000 7fc3388fb640 1 -- 172.21.15.67:0/935798624 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fc31c017070 con 0x7fc3341083a0 2024-09-21T06:48:23.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.931+0000 7fc3388fb640 1 --2- 172.21.15.67:0/935798624 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc3200b6540 0x7fc3200b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:23.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.931+0000 7fc3388fb640 1 -- 172.21.15.67:0/935798624 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fc31c0e78a0 con 0x7fc3341083a0 2024-09-21T06:48:23.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.931+0000 7fc332ffd640 1 --2- 172.21.15.67:0/935798624 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc3200b6540 0x7fc3200b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:23.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.932+0000 7fc332ffd640 1 --2- 172.21.15.67:0/935798624 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc3200b6540 0x7fc3200b8a50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc32c005fd0 tx=0x7fc32c009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:23.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:23.935+0000 7fc3388fb640 1 -- 172.21.15.67:0/935798624 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc31c01e030 con 0x7fc3341083a0 2024-09-21T06:48:23.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:23 smithi203 ceph-mon[33735]: pgmap v714: 129 pgs: 129 active+clean; 109 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.8 MiB/s wr, 406 op/s 2024-09-21T06:48:24.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:23 smithi100 ceph-mon[33401]: pgmap v714: 129 pgs: 129 active+clean; 109 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.8 MiB/s wr, 406 op/s 2024-09-21T06:48:24.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:24.136+0000 7fc3398fd640 1 -- 172.21.15.67:0/935798624 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc33419b550 con 0x7fc3341083a0 2024-09-21T06:48:24.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:24.136+0000 7fc3388fb640 1 -- 172.21.15.67:0/935798624 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v85) ==== 81+0+592 (secure 0 0 0) 0x7fc31c0add60 con 0x7fc3341083a0 2024-09-21T06:48:24.139 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/1181476774 2024-09-22T06:27:03.740475+0000 2024-09-21T06:48:24.139 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3100321138 2024-09-22T06:27:03.740475+0000 2024-09-21T06:48:24.139 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/2309364298 2024-09-22T06:27:03.740475+0000 2024-09-21T06:48:24.139 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3458717076 2024-09-22T06:27:03.740475+0000 2024-09-21T06:48:24.139 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/4110206171 2024-09-22T06:26:37.041377+0000 2024-09-21T06:48:24.139 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/1181476774 2024-09-22T06:27:03.740475+0000 2024-09-21T06:48:24.139 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/2910038982 2024-09-22T06:26:37.041377+0000 2024-09-21T06:48:24.140 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/2622729103 2024-09-22T06:26:37.041377+0000 2024-09-21T06:48:24.140 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/2622729103 2024-09-22T06:26:37.041377+0000 2024-09-21T06:48:24.140 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3346784528 2024-09-22T06:26:37.041377+0000 2024-09-21T06:48:24.140 INFO:teuthology.orchestra.run.smithi067.stderr:listed 10 entries 2024-09-21T06:48:24.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:24.139+0000 7fc3398fd640 1 -- 172.21.15.67:0/935798624 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc3200b6540 msgr2=0x7fc3200b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:24.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:24.139+0000 7fc3398fd640 1 --2- 172.21.15.67:0/935798624 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc3200b6540 0x7fc3200b8a50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc32c005fd0 tx=0x7fc32c009680 comp rx=0 tx=0).stop 2024-09-21T06:48:24.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:24.139+0000 7fc3398fd640 1 -- 172.21.15.67:0/935798624 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3341083a0 msgr2=0x7fc33419a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:24.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:24.139+0000 7fc3398fd640 1 --2- 172.21.15.67:0/935798624 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3341083a0 0x7fc33419a9c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc31c00c970 tx=0x7fc31c00ce30 comp rx=0 tx=0).stop 2024-09-21T06:48:24.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:24.139+0000 7fc3398fd640 1 -- 172.21.15.67:0/935798624 shutdown_connections 2024-09-21T06:48:24.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:24.139+0000 7fc3398fd640 1 --2- 172.21.15.67:0/935798624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc334108cf0 0x7fc33419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:24.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:24.139+0000 7fc3398fd640 1 --2- 172.21.15.67:0/935798624 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fc3200b6540 0x7fc3200b8a50 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:24.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:24.139+0000 7fc3398fd640 1 --2- 172.21.15.67:0/935798624 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3341083a0 0x7fc33419a9c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:24.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:24.139+0000 7fc3398fd640 1 --2- 172.21.15.67:0/935798624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc334102a60 0x7fc33419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:24.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:24.139+0000 7fc3398fd640 1 -- 172.21.15.67:0/935798624 >> 172.21.15.67:0/935798624 conn(0x7fc334079730 msgr2=0x7fc334104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:24.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:24.139+0000 7fc3398fd640 1 -- 172.21.15.67:0/935798624 shutdown_connections 2024-09-21T06:48:24.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:24.139+0000 7fc3398fd640 1 -- 172.21.15.67:0/935798624 wait complete. 2024-09-21T06:48:24.829 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:24 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/935798624' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T06:48:24.830 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-21T06:48:24.830 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:48:24.847 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T06:48:24.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:24 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/935798624' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T06:48:25.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:24 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/935798624' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T06:48:25.143 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:48:25.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.665+0000 7fb679939640 1 -- 172.21.15.67:0/2688691051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6741094f0 msgr2=0x7fb674109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:25.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.665+0000 7fb679939640 1 --2- 172.21.15.67:0/2688691051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6741094f0 0x7fb674109900 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb668009900 tx=0x7fb66802a3f0 comp rx=0 tx=0).stop 2024-09-21T06:48:25.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.665+0000 7fb679939640 1 -- 172.21.15.67:0/2688691051 shutdown_connections 2024-09-21T06:48:25.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.665+0000 7fb679939640 1 --2- 172.21.15.67:0/2688691051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb674109e40 0x7fb674110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:25.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.665+0000 7fb679939640 1 --2- 172.21.15.67:0/2688691051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6741094f0 0x7fb674109900 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:25.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.665+0000 7fb679939640 1 --2- 172.21.15.67:0/2688691051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb674103bb0 0x7fb674103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:25.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.665+0000 7fb679939640 1 -- 172.21.15.67:0/2688691051 >> 172.21.15.67:0/2688691051 conn(0x7fb6740ff600 msgr2=0x7fb674101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:25.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.665+0000 7fb679939640 1 -- 172.21.15.67:0/2688691051 shutdown_connections 2024-09-21T06:48:25.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.665+0000 7fb679939640 1 -- 172.21.15.67:0/2688691051 wait complete. 2024-09-21T06:48:25.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.665+0000 7fb679939640 1 Processor -- start 2024-09-21T06:48:25.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.666+0000 7fb679939640 1 -- start start 2024-09-21T06:48:25.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.666+0000 7fb679939640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb674103bb0 0x7fb674075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:25.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.666+0000 7fb679939640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6741094f0 0x7fb674075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:25.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.666+0000 7fb679939640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb674109e40 0x7fb674076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:25.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.666+0000 7fb679939640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb6741139d0 con 0x7fb674103bb0 2024-09-21T06:48:25.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.666+0000 7fb679939640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb674113850 con 0x7fb674109e40 2024-09-21T06:48:25.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.666+0000 7fb679939640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb674113b50 con 0x7fb6741094f0 2024-09-21T06:48:25.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.666+0000 7fb672ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb674103bb0 0x7fb674075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:25.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.666+0000 7fb672ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb674103bb0 0x7fb674075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54748/0 (socket says 172.21.15.67:54748) 2024-09-21T06:48:25.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.666+0000 7fb672ffd640 1 -- 172.21.15.67:0/587519075 learned_addr learned my addr 172.21.15.67:0/587519075 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:48:25.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.666+0000 7fb6737fe640 1 --2- 172.21.15.67:0/587519075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb674109e40 0x7fb674076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:25.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.666+0000 7fb6727fc640 1 --2- 172.21.15.67:0/587519075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6741094f0 0x7fb674075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:25.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.666+0000 7fb672ffd640 1 -- 172.21.15.67:0/587519075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6741094f0 msgr2=0x7fb674075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:25.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.666+0000 7fb672ffd640 1 --2- 172.21.15.67:0/587519075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6741094f0 0x7fb674075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:25.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.666+0000 7fb672ffd640 1 -- 172.21.15.67:0/587519075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb674109e40 msgr2=0x7fb674076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:25.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.666+0000 7fb672ffd640 1 --2- 172.21.15.67:0/587519075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb674109e40 0x7fb674076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:25.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.666+0000 7fb672ffd640 1 -- 172.21.15.67:0/587519075 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6741a8700 con 0x7fb674103bb0 2024-09-21T06:48:25.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.667+0000 7fb672ffd640 1 --2- 172.21.15.67:0/587519075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb674103bb0 0x7fb674075710 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb66400d950 tx=0x7fb66400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:25.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.667+0000 7fb678937640 1 -- 172.21.15.67:0/587519075 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb664017070 con 0x7fb674103bb0 2024-09-21T06:48:25.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.667+0000 7fb678937640 1 -- 172.21.15.67:0/587519075 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb664009c30 con 0x7fb674103bb0 2024-09-21T06:48:25.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.667+0000 7fb678937640 1 -- 172.21.15.67:0/587519075 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66400f620 con 0x7fb674103bb0 2024-09-21T06:48:25.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.667+0000 7fb679939640 1 -- 172.21.15.67:0/587519075 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6741a89f0 con 0x7fb674103bb0 2024-09-21T06:48:25.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.667+0000 7fb679939640 1 -- 172.21.15.67:0/587519075 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6741a8f80 con 0x7fb674103bb0 2024-09-21T06:48:25.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.668+0000 7fb679939640 1 -- 172.21.15.67:0/587519075 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb674103fc0 con 0x7fb674103bb0 2024-09-21T06:48:25.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.669+0000 7fb678937640 1 -- 172.21.15.67:0/587519075 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fb664010460 con 0x7fb674103bb0 2024-09-21T06:48:25.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.670+0000 7fb678937640 1 --2- 172.21.15.67:0/587519075 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb65c0b6510 0x7fb65c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:25.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.670+0000 7fb6727fc640 1 --2- 172.21.15.67:0/587519075 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb65c0b6510 0x7fb65c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:25.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.670+0000 7fb678937640 1 -- 172.21.15.67:0/587519075 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fb6640e7220 con 0x7fb674103bb0 2024-09-21T06:48:25.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.671+0000 7fb6727fc640 1 --2- 172.21.15.67:0/587519075 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb65c0b6510 0x7fb65c0b8a20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb668005940 tx=0x7fb6680058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:25.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.675+0000 7fb678937640 1 -- 172.21.15.67:0/587519075 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb664009de0 con 0x7fb674103bb0 2024-09-21T06:48:25.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.876+0000 7fb679939640 1 -- 172.21.15.67:0/587519075 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb67402e640 con 0x7fb674103bb0 2024-09-21T06:48:25.878 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:25 smithi067 ceph-mon[31232]: pgmap v715: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 1.6 MiB/s wr, 253 op/s 2024-09-21T06:48:25.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.876+0000 7fb678937640 1 -- 172.21.15.67:0/587519075 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v85) ==== 81+0+592 (secure 0 0 0) 0x7fb6640b1ac0 con 0x7fb674103bb0 2024-09-21T06:48:25.879 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/1181476774 2024-09-22T06:27:03.740475+0000 2024-09-21T06:48:25.879 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3100321138 2024-09-22T06:27:03.740475+0000 2024-09-21T06:48:25.879 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/2309364298 2024-09-22T06:27:03.740475+0000 2024-09-21T06:48:25.879 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3458717076 2024-09-22T06:27:03.740475+0000 2024-09-21T06:48:25.879 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/4110206171 2024-09-22T06:26:37.041377+0000 2024-09-21T06:48:25.879 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/1181476774 2024-09-22T06:27:03.740475+0000 2024-09-21T06:48:25.880 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/2910038982 2024-09-22T06:26:37.041377+0000 2024-09-21T06:48:25.880 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/2622729103 2024-09-22T06:26:37.041377+0000 2024-09-21T06:48:25.880 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/2622729103 2024-09-22T06:26:37.041377+0000 2024-09-21T06:48:25.880 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3346784528 2024-09-22T06:26:37.041377+0000 2024-09-21T06:48:25.880 INFO:teuthology.orchestra.run.smithi067.stderr:listed 10 entries 2024-09-21T06:48:25.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.878+0000 7fb679939640 1 -- 172.21.15.67:0/587519075 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb65c0b6510 msgr2=0x7fb65c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:25.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.878+0000 7fb679939640 1 --2- 172.21.15.67:0/587519075 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb65c0b6510 0x7fb65c0b8a20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb668005940 tx=0x7fb6680058d0 comp rx=0 tx=0).stop 2024-09-21T06:48:25.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.878+0000 7fb679939640 1 -- 172.21.15.67:0/587519075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb674103bb0 msgr2=0x7fb674075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:25.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.878+0000 7fb679939640 1 --2- 172.21.15.67:0/587519075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb674103bb0 0x7fb674075710 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb66400d950 tx=0x7fb66400de10 comp rx=0 tx=0).stop 2024-09-21T06:48:25.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.878+0000 7fb679939640 1 -- 172.21.15.67:0/587519075 shutdown_connections 2024-09-21T06:48:25.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.878+0000 7fb679939640 1 --2- 172.21.15.67:0/587519075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb674109e40 0x7fb674076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:25.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.878+0000 7fb679939640 1 --2- 172.21.15.67:0/587519075 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fb65c0b6510 0x7fb65c0b8a20 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:25.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.878+0000 7fb679939640 1 --2- 172.21.15.67:0/587519075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6741094f0 0x7fb674075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:25.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.878+0000 7fb679939640 1 --2- 172.21.15.67:0/587519075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb674103bb0 0x7fb674075710 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:25.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.878+0000 7fb679939640 1 -- 172.21.15.67:0/587519075 >> 172.21.15.67:0/587519075 conn(0x7fb6740ff600 msgr2=0x7fb674101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:25.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.879+0000 7fb679939640 1 -- 172.21.15.67:0/587519075 shutdown_connections 2024-09-21T06:48:25.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:25.879+0000 7fb679939640 1 -- 172.21.15.67:0/587519075 wait complete. 2024-09-21T06:48:25.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:25 smithi203 ceph-mon[33735]: pgmap v715: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 1.6 MiB/s wr, 253 op/s 2024-09-21T06:48:26.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:25 smithi100 ceph-mon[33401]: pgmap v715: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 1.6 MiB/s wr, 253 op/s 2024-09-21T06:48:26.597 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi067.front.sepia.ceph.com... 2024-09-21T06:48:26.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:48:26.597 DEBUG:teuthology.orchestra.run.smithi067:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-21T06:48:26.635 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[84451]: fuse finished with error 0 and tester_r 0 2024-09-21T06:48:26.636 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:48:26.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:26 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/587519075' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T06:48:27.041 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:26 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/587519075' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T06:48:27.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:26 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/587519075' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T06:48:27.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:27 smithi203 ceph-mon[33735]: pgmap v716: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 48 KiB/s wr, 210 op/s 2024-09-21T06:48:28.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:27 smithi067 ceph-mon[31232]: pgmap v716: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 48 KiB/s wr, 210 op/s 2024-09-21T06:48:28.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:27 smithi100 ceph-mon[33401]: pgmap v716: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 48 KiB/s wr, 210 op/s 2024-09-21T06:48:29.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:29 smithi203 ceph-mon[33735]: pgmap v717: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.9 KiB/s wr, 167 op/s 2024-09-21T06:48:30.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:29 smithi067 ceph-mon[31232]: pgmap v717: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.9 KiB/s wr, 167 op/s 2024-09-21T06:48:30.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:29 smithi100 ceph-mon[33401]: pgmap v717: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.9 KiB/s wr, 167 op/s 2024-09-21T06:48:31.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:31 smithi203 ceph-mon[33735]: pgmap v718: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.9 KiB/s wr, 76 op/s 2024-09-21T06:48:32.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:31 smithi067 ceph-mon[31232]: pgmap v718: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.9 KiB/s wr, 76 op/s 2024-09-21T06:48:32.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:31 smithi100 ceph-mon[33401]: pgmap v718: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.9 KiB/s wr, 76 op/s 2024-09-21T06:48:32.737 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi067.front.sepia.ceph.com 2024-09-21T06:48:32.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:48:32.738 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T06:48:32.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:48:32.777 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-09-21T06:48:32.857 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T06:48:32.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:48:32.857 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T06:48:32.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:48:32.927 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link delete ceph-brx 2024-09-21T06:48:33.044 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-21T06:48:33.057 INFO:tasks.cephadm:Teardown begin 2024-09-21T06:48:33.058 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T06:48:33.058 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:48:33.320 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:48:33.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.823+0000 7f9c51149640 1 -- 172.21.15.67:0/1605083416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c4c103b60 msgr2=0x7f9c4c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:33.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.823+0000 7f9c51149640 1 --2- 172.21.15.67:0/1605083416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c4c103b60 0x7f9c4c103f70 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c009900 tx=0x7f9c3c02a410 comp rx=0 tx=0).stop 2024-09-21T06:48:33.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.823+0000 7f9c51149640 1 -- 172.21.15.67:0/1605083416 shutdown_connections 2024-09-21T06:48:33.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.823+0000 7f9c51149640 1 --2- 172.21.15.67:0/1605083416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c4c109df0 0x7f9c4c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:33.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.823+0000 7f9c51149640 1 --2- 172.21.15.67:0/1605083416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c4c1094a0 0x7f9c4c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:33.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.823+0000 7f9c51149640 1 --2- 172.21.15.67:0/1605083416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c4c103b60 0x7f9c4c103f70 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:33.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.823+0000 7f9c51149640 1 -- 172.21.15.67:0/1605083416 >> 172.21.15.67:0/1605083416 conn(0x7f9c4c07b850 msgr2=0x7f9c4c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:33.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.823+0000 7f9c51149640 1 -- 172.21.15.67:0/1605083416 shutdown_connections 2024-09-21T06:48:33.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.823+0000 7f9c51149640 1 -- 172.21.15.67:0/1605083416 wait complete. 2024-09-21T06:48:33.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.824+0000 7f9c51149640 1 Processor -- start 2024-09-21T06:48:33.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.824+0000 7f9c51149640 1 -- start start 2024-09-21T06:48:33.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.825+0000 7f9c51149640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c4c103b60 0x7f9c4c19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:33.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.825+0000 7f9c51149640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c4c1094a0 0x7f9c4c19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:33.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.825+0000 7f9c51149640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c4c109df0 0x7f9c4c19fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:33.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.825+0000 7f9c51149640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c113960 con 0x7f9c4c109df0 2024-09-21T06:48:33.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.825+0000 7f9c51149640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c1137e0 con 0x7f9c4c1094a0 2024-09-21T06:48:33.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.825+0000 7f9c51149640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c113ae0 con 0x7f9c4c103b60 2024-09-21T06:48:33.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.825+0000 7f9c4b59e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c4c109df0 0x7f9c4c19fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:33.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.825+0000 7f9c4b59e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c4c109df0 0x7f9c4c19fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53230/0 (socket says 172.21.15.67:53230) 2024-09-21T06:48:33.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.825+0000 7f9c4b59e640 1 -- 172.21.15.67:0/2776423851 learned_addr learned my addr 172.21.15.67:0/2776423851 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:48:33.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.825+0000 7f9c4ad9d640 1 --2- 172.21.15.67:0/2776423851 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c4c103b60 0x7f9c4c19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:33.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.825+0000 7f9c4a59c640 1 --2- 172.21.15.67:0/2776423851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c4c1094a0 0x7f9c4c19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:33.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.825+0000 7f9c4b59e640 1 -- 172.21.15.67:0/2776423851 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c4c103b60 msgr2=0x7f9c4c19a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:33.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.825+0000 7f9c4b59e640 1 --2- 172.21.15.67:0/2776423851 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c4c103b60 0x7f9c4c19a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:33.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.825+0000 7f9c4b59e640 1 -- 172.21.15.67:0/2776423851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c4c1094a0 msgr2=0x7f9c4c19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:33.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.825+0000 7f9c4b59e640 1 --2- 172.21.15.67:0/2776423851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c4c1094a0 0x7f9c4c19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:33.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.825+0000 7f9c4b59e640 1 -- 172.21.15.67:0/2776423851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c4c1a03a0 con 0x7f9c4c109df0 2024-09-21T06:48:33.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.825+0000 7f9c4b59e640 1 --2- 172.21.15.67:0/2776423851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c4c109df0 0x7f9c4c19fc70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009fd0 tx=0x7f9c4400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:33.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.825+0000 7f9c33fff640 1 -- 172.21.15.67:0/2776423851 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44017070 con 0x7f9c4c109df0 2024-09-21T06:48:33.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.826+0000 7f9c51149640 1 -- 172.21.15.67:0/2776423851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c4c1a0690 con 0x7f9c4c109df0 2024-09-21T06:48:33.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.826+0000 7f9c33fff640 1 -- 172.21.15.67:0/2776423851 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c44003e00 con 0x7f9c4c109df0 2024-09-21T06:48:33.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.826+0000 7f9c33fff640 1 -- 172.21.15.67:0/2776423851 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44010460 con 0x7f9c4c109df0 2024-09-21T06:48:33.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.826+0000 7f9c51149640 1 -- 172.21.15.67:0/2776423851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c4c1a6df0 con 0x7f9c4c109df0 2024-09-21T06:48:33.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.827+0000 7f9c51149640 1 -- 172.21.15.67:0/2776423851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c4c103f70 con 0x7f9c4c109df0 2024-09-21T06:48:33.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.829+0000 7f9c33fff640 1 -- 172.21.15.67:0/2776423851 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f9c44003970 con 0x7f9c4c109df0 2024-09-21T06:48:33.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.830+0000 7f9c33fff640 1 --2- 172.21.15.67:0/2776423851 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9c340b6510 0x7f9c340b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:33.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.830+0000 7f9c4ad9d640 1 --2- 172.21.15.67:0/2776423851 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9c340b6510 0x7f9c340b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:33.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.830+0000 7f9c33fff640 1 -- 172.21.15.67:0/2776423851 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f9c440e7740 con 0x7f9c4c109df0 2024-09-21T06:48:33.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.831+0000 7f9c4ad9d640 1 --2- 172.21.15.67:0/2776423851 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9c340b6510 0x7f9c340b8a20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c0057c0 tx=0x7f9c3c0058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:33.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:33.833+0000 7f9c33fff640 1 -- 172.21.15.67:0/2776423851 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9c440b1fe0 con 0x7f9c4c109df0 2024-09-21T06:48:33.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:33 smithi203 ceph-mon[33735]: pgmap v719: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 KiB/s wr, 57 op/s 2024-09-21T06:48:34.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:33 smithi067 ceph-mon[31232]: pgmap v719: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 KiB/s wr, 57 op/s 2024-09-21T06:48:34.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:33 smithi100 ceph-mon[33401]: pgmap v719: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 KiB/s wr, 57 op/s 2024-09-21T06:48:34.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:34.059+0000 7f9c51149640 1 -- 172.21.15.67:0/2776423851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c4c19b560 con 0x7f9c4c109df0 2024-09-21T06:48:34.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:34.060+0000 7f9c33fff640 1 -- 172.21.15.67:0/2776423851 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7f9c440b6f20 con 0x7f9c4c109df0 2024-09-21T06:48:34.062 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:48:34.063 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:48:34.063 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:48:34.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:34.062+0000 7f9c51149640 1 -- 172.21.15.67:0/2776423851 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9c340b6510 msgr2=0x7f9c340b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:34.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:34.062+0000 7f9c51149640 1 --2- 172.21.15.67:0/2776423851 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9c340b6510 0x7f9c340b8a20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c0057c0 tx=0x7f9c3c0058d0 comp rx=0 tx=0).stop 2024-09-21T06:48:34.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:34.062+0000 7f9c51149640 1 -- 172.21.15.67:0/2776423851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c4c109df0 msgr2=0x7f9c4c19fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:34.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:34.062+0000 7f9c51149640 1 --2- 172.21.15.67:0/2776423851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c4c109df0 0x7f9c4c19fc70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009fd0 tx=0x7f9c4400b260 comp rx=0 tx=0).stop 2024-09-21T06:48:34.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:34.062+0000 7f9c51149640 1 -- 172.21.15.67:0/2776423851 shutdown_connections 2024-09-21T06:48:34.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:34.062+0000 7f9c51149640 1 --2- 172.21.15.67:0/2776423851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c4c1094a0 0x7f9c4c19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:34.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:34.062+0000 7f9c51149640 1 --2- 172.21.15.67:0/2776423851 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f9c340b6510 0x7f9c340b8a20 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:34.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:34.062+0000 7f9c51149640 1 --2- 172.21.15.67:0/2776423851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c4c109df0 0x7f9c4c19fc70 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:34.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:34.062+0000 7f9c51149640 1 --2- 172.21.15.67:0/2776423851 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c4c103b60 0x7f9c4c19a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:34.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:34.062+0000 7f9c51149640 1 -- 172.21.15.67:0/2776423851 >> 172.21.15.67:0/2776423851 conn(0x7f9c4c07b850 msgr2=0x7f9c4c107640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:34.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:34.062+0000 7f9c51149640 1 -- 172.21.15.67:0/2776423851 shutdown_connections 2024-09-21T06:48:34.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:34.062+0000 7f9c51149640 1 -- 172.21.15.67:0/2776423851 wait complete. 2024-09-21T06:48:34.743 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:48:34.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:34 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2776423851' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:48:35.003 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:48:35.041 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:34 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2776423851' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:48:35.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:34 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2776423851' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:48:35.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:35 smithi203 ceph-mon[33735]: pgmap v720: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-21T06:48:36.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.483+0000 7fad349ef640 1 -- 172.21.15.67:0/322795535 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad30109df0 msgr2=0x7fad30110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:36.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.483+0000 7fad349ef640 1 --2- 172.21.15.67:0/322795535 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad30109df0 0x7fad30110620 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fad28009ca0 tx=0x7fad2802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:48:36.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.483+0000 7fad349ef640 1 -- 172.21.15.67:0/322795535 shutdown_connections 2024-09-21T06:48:36.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.483+0000 7fad349ef640 1 --2- 172.21.15.67:0/322795535 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad30109df0 0x7fad30110620 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:36.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.483+0000 7fad349ef640 1 --2- 172.21.15.67:0/322795535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad301094a0 0x7fad301098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:36.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.483+0000 7fad349ef640 1 --2- 172.21.15.67:0/322795535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad30103b60 0x7fad30103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:36.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.483+0000 7fad349ef640 1 -- 172.21.15.67:0/322795535 >> 172.21.15.67:0/322795535 conn(0x7fad3007b850 msgr2=0x7fad3007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:36.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.483+0000 7fad349ef640 1 -- 172.21.15.67:0/322795535 shutdown_connections 2024-09-21T06:48:36.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.483+0000 7fad349ef640 1 -- 172.21.15.67:0/322795535 wait complete. 2024-09-21T06:48:36.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.484+0000 7fad349ef640 1 Processor -- start 2024-09-21T06:48:36.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.484+0000 7fad349ef640 1 -- start start 2024-09-21T06:48:36.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.484+0000 7fad349ef640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad30103b60 0x7fad3019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:36.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.484+0000 7fad349ef640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad301094a0 0x7fad3019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:36.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.484+0000 7fad349ef640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad30109df0 0x7fad3019fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:36.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.484+0000 7fad349ef640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fad301139b0 con 0x7fad301094a0 2024-09-21T06:48:36.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.484+0000 7fad349ef640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fad30113830 con 0x7fad30103b60 2024-09-21T06:48:36.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.484+0000 7fad349ef640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fad30113b30 con 0x7fad30109df0 2024-09-21T06:48:36.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.484+0000 7fad2dd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad301094a0 0x7fad3019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:36.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.484+0000 7fad2dd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad301094a0 0x7fad3019a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53260/0 (socket says 172.21.15.67:53260) 2024-09-21T06:48:36.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.484+0000 7fad2dd9b640 1 -- 172.21.15.67:0/1770818548 learned_addr learned my addr 172.21.15.67:0/1770818548 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:48:36.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.485+0000 7fad2e59c640 1 --2- 172.21.15.67:0/1770818548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad30103b60 0x7fad3019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:36.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.485+0000 7fad2ed9d640 1 --2- 172.21.15.67:0/1770818548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad30109df0 0x7fad3019fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:36.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.485+0000 7fad2dd9b640 1 -- 172.21.15.67:0/1770818548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad30109df0 msgr2=0x7fad3019fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:36.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.485+0000 7fad2dd9b640 1 --2- 172.21.15.67:0/1770818548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad30109df0 0x7fad3019fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:36.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.485+0000 7fad2dd9b640 1 -- 172.21.15.67:0/1770818548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad30103b60 msgr2=0x7fad3019a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:36.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.485+0000 7fad2dd9b640 1 --2- 172.21.15.67:0/1770818548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad30103b60 0x7fad3019a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:36.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.485+0000 7fad2dd9b640 1 -- 172.21.15.67:0/1770818548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad301a0250 con 0x7fad301094a0 2024-09-21T06:48:36.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.485+0000 7fad2ed9d640 1 --2- 172.21.15.67:0/1770818548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad30109df0 0x7fad3019fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:48:36.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.485+0000 7fad2dd9b640 1 --2- 172.21.15.67:0/1770818548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad301094a0 0x7fad3019a940 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fad2400c8c0 tx=0x7fad2400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:36.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.485+0000 7fad1f7fe640 1 -- 172.21.15.67:0/1770818548 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad24007bb0 con 0x7fad301094a0 2024-09-21T06:48:36.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.485+0000 7fad1f7fe640 1 -- 172.21.15.67:0/1770818548 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad24007d50 con 0x7fad301094a0 2024-09-21T06:48:36.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.485+0000 7fad349ef640 1 -- 172.21.15.67:0/1770818548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad301a0540 con 0x7fad301094a0 2024-09-21T06:48:36.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.485+0000 7fad1f7fe640 1 -- 172.21.15.67:0/1770818548 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad240186a0 con 0x7fad301094a0 2024-09-21T06:48:36.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.485+0000 7fad349ef640 1 -- 172.21.15.67:0/1770818548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad301a6d60 con 0x7fad301094a0 2024-09-21T06:48:36.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.487+0000 7fad349ef640 1 -- 172.21.15.67:0/1770818548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad30103f70 con 0x7fad301094a0 2024-09-21T06:48:36.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.488+0000 7fad1f7fe640 1 -- 172.21.15.67:0/1770818548 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fad2400fa00 con 0x7fad301094a0 2024-09-21T06:48:36.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.489+0000 7fad1f7fe640 1 --2- 172.21.15.67:0/1770818548 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fad180b6540 0x7fad180b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:36.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.489+0000 7fad2e59c640 1 --2- 172.21.15.67:0/1770818548 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fad180b6540 0x7fad180b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:36.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.489+0000 7fad1f7fe640 1 -- 172.21.15.67:0/1770818548 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fad240e7730 con 0x7fad301094a0 2024-09-21T06:48:36.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.489+0000 7fad2e59c640 1 --2- 172.21.15.67:0/1770818548 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fad180b6540 0x7fad180b8a50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fad200058c0 tx=0x7fad2000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:36.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.493+0000 7fad1f7fe640 1 -- 172.21.15.67:0/1770818548 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fad24014030 con 0x7fad301094a0 2024-09-21T06:48:36.019 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:48:36.019 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"btime":"2024-09-21T06:35:10:013981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/4145954943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4145954943},{"type":"v1","addr":"172.21.15.67:6835","nonce":4145954943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1744346240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1744346240},{"type":"v1","addr":"172.21.15.67:6837","nonce":1744346240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/1168518926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":1168518926},{"type":"v1","addr":"172.21.15.67:6839","nonce":1168518926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2331956394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2331956394},{"type":"v1","addr":"172.21.15.67:6841","nonce":2331956394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3852485418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3852485418},{"type":"v1","addr":"172.21.15.203:6833","nonce":3852485418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24587,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/4088716909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":4088716909},{"type":"v1","addr":"172.21.15.100:6833","nonce":4088716909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2689490358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2689490358},{"type":"v1","addr":"172.21.15.203:6835","nonce":2689490358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24620,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1518007169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1518007169},{"type":"v1","addr":"172.21.15.100:6835","nonce":1518007169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426013144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426013144},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426013144}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2443174511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2443174511},{"type":"v1","addr":"172.21.15.203:6839","nonce":2443174511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:34:33.306959+0000","modified":"2024-09-21T06:35:09.805211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24641":{"gid":24641,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/259395872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":259395872},{"type":"v1","addr":"172.21.15.100:6837","nonce":259395872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/778060936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":778060936},{"type":"v1","addr":"172.21.15.100:6839","nonce":778060936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-09-21T06:48:36.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.722+0000 7fad349ef640 1 -- 172.21.15.67:0/1770818548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad3019b4d0 con 0x7fad301094a0 2024-09-21T06:48:36.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.722+0000 7fad1f7fe640 1 -- 172.21.15.67:0/1770818548 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11861 (secure 0 0 0) 0x7fad240adbf0 con 0x7fad301094a0 2024-09-21T06:48:36.019 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 21 2024-09-21T06:48:36.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.724+0000 7fad349ef640 1 -- 172.21.15.67:0/1770818548 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fad180b6540 msgr2=0x7fad180b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:36.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.724+0000 7fad349ef640 1 --2- 172.21.15.67:0/1770818548 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fad180b6540 0x7fad180b8a50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fad200058c0 tx=0x7fad2000a2d0 comp rx=0 tx=0).stop 2024-09-21T06:48:36.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.724+0000 7fad349ef640 1 -- 172.21.15.67:0/1770818548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad301094a0 msgr2=0x7fad3019a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:36.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.724+0000 7fad349ef640 1 --2- 172.21.15.67:0/1770818548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad301094a0 0x7fad3019a940 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fad2400c8c0 tx=0x7fad2400cd80 comp rx=0 tx=0).stop 2024-09-21T06:48:36.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.725+0000 7fad349ef640 1 -- 172.21.15.67:0/1770818548 shutdown_connections 2024-09-21T06:48:36.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.725+0000 7fad349ef640 1 --2- 172.21.15.67:0/1770818548 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7fad180b6540 0x7fad180b8a50 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:36.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.725+0000 7fad349ef640 1 --2- 172.21.15.67:0/1770818548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad30109df0 0x7fad3019fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:36.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.725+0000 7fad349ef640 1 --2- 172.21.15.67:0/1770818548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad301094a0 0x7fad3019a940 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:36.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.725+0000 7fad349ef640 1 --2- 172.21.15.67:0/1770818548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad30103b60 0x7fad3019a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:36.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.725+0000 7fad349ef640 1 -- 172.21.15.67:0/1770818548 >> 172.21.15.67:0/1770818548 conn(0x7fad3007b850 msgr2=0x7fad301054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:36.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.725+0000 7fad349ef640 1 -- 172.21.15.67:0/1770818548 shutdown_connections 2024-09-21T06:48:36.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:35.725+0000 7fad349ef640 1 -- 172.21.15.67:0/1770818548 wait complete. 2024-09-21T06:48:36.020 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:35 smithi067 ceph-mon[31232]: pgmap v720: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-21T06:48:36.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:35 smithi100 ceph-mon[33401]: pgmap v720: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-21T06:48:36.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:36 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1770818548' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:48:36.862 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:48:36.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:36 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1770818548' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:48:37.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:36 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1770818548' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:48:37.129 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:48:37.880 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:37 smithi067 ceph-mon[31232]: pgmap v721: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 426 B/s wr, 3 op/s 2024-09-21T06:48:37.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.878+0000 7f232a0e7640 1 -- 172.21.15.67:0/2955080207 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2324108cf0 msgr2=0x7f232410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:37.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.878+0000 7f232a0e7640 1 --2- 172.21.15.67:0/2955080207 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2324108cf0 0x7f232410f520 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f231c009ca0 tx=0x7f231c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:48:37.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.878+0000 7f232a0e7640 1 -- 172.21.15.67:0/2955080207 shutdown_connections 2024-09-21T06:48:37.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.878+0000 7f232a0e7640 1 --2- 172.21.15.67:0/2955080207 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2324108cf0 0x7f232410f520 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:37.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.878+0000 7f232a0e7640 1 --2- 172.21.15.67:0/2955080207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23241083a0 0x7f23241087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:37.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.878+0000 7f232a0e7640 1 --2- 172.21.15.67:0/2955080207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2324102a60 0x7f2324102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:37.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.878+0000 7f232a0e7640 1 -- 172.21.15.67:0/2955080207 >> 172.21.15.67:0/2955080207 conn(0x7f2324079730 msgr2=0x7f2324079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:37.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.878+0000 7f232a0e7640 1 -- 172.21.15.67:0/2955080207 shutdown_connections 2024-09-21T06:48:37.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.878+0000 7f232a0e7640 1 -- 172.21.15.67:0/2955080207 wait complete. 2024-09-21T06:48:37.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.879+0000 7f232a0e7640 1 Processor -- start 2024-09-21T06:48:37.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.879+0000 7f232a0e7640 1 -- start start 2024-09-21T06:48:37.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.879+0000 7f232a0e7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2324102a60 0x7f232419c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:37.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.879+0000 7f232a0e7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23241083a0 0x7f232419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:37.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.879+0000 7f232a0e7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2324108cf0 0x7f23241a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:37.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.879+0000 7f232a0e7640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2324112860 con 0x7f23241083a0 2024-09-21T06:48:37.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.879+0000 7f232a0e7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f23241126e0 con 0x7f2324102a60 2024-09-21T06:48:37.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.879+0000 7f232a0e7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f23241129e0 con 0x7f2324108cf0 2024-09-21T06:48:37.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.879+0000 7f2322ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23241083a0 0x7f232419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:37.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.880+0000 7f23237fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2324102a60 0x7f232419c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:37.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.880+0000 7f2322ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23241083a0 0x7f232419cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37774/0 (socket says 172.21.15.67:37774) 2024-09-21T06:48:37.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.880+0000 7f2323fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2324108cf0 0x7f23241a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:37.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.880+0000 7f2322ffd640 1 -- 172.21.15.67:0/1836391907 learned_addr learned my addr 172.21.15.67:0/1836391907 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:48:37.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.880+0000 7f2322ffd640 1 -- 172.21.15.67:0/1836391907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2324108cf0 msgr2=0x7f23241a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:37.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.880+0000 7f2322ffd640 1 --2- 172.21.15.67:0/1836391907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2324108cf0 0x7f23241a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:37.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.880+0000 7f2322ffd640 1 -- 172.21.15.67:0/1836391907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2324102a60 msgr2=0x7f232419c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:37.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.880+0000 7f2322ffd640 1 --2- 172.21.15.67:0/1836391907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2324102a60 0x7f232419c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:37.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.880+0000 7f2322ffd640 1 -- 172.21.15.67:0/1836391907 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23241a2560 con 0x7f23241083a0 2024-09-21T06:48:37.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.880+0000 7f23237fe640 1 --2- 172.21.15.67:0/1836391907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2324102a60 0x7f232419c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:48:37.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.880+0000 7f2322ffd640 1 --2- 172.21.15.67:0/1836391907 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23241083a0 0x7f232419cb90 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f231800d770 tx=0x7f231800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:37.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.880+0000 7f2320ff9640 1 -- 172.21.15.67:0/1836391907 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2318017070 con 0x7f23241083a0 2024-09-21T06:48:37.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.880+0000 7f232a0e7640 1 -- 172.21.15.67:0/1836391907 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23241a2850 con 0x7f23241083a0 2024-09-21T06:48:37.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.880+0000 7f2320ff9640 1 -- 172.21.15.67:0/1836391907 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2318009c30 con 0x7f23241083a0 2024-09-21T06:48:37.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.880+0000 7f2320ff9640 1 -- 172.21.15.67:0/1836391907 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231800f620 con 0x7f23241083a0 2024-09-21T06:48:37.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.881+0000 7f232a0e7640 1 -- 172.21.15.67:0/1836391907 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23241a8fb0 con 0x7f23241083a0 2024-09-21T06:48:37.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.882+0000 7f232a0e7640 1 -- 172.21.15.67:0/1836391907 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2324102e70 con 0x7f23241083a0 2024-09-21T06:48:37.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.883+0000 7f2320ff9640 1 -- 172.21.15.67:0/1836391907 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f231801a070 con 0x7f23241083a0 2024-09-21T06:48:37.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.884+0000 7f2320ff9640 1 --2- 172.21.15.67:0/1836391907 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f230c0b6510 0x7f230c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:37.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.884+0000 7f2320ff9640 1 -- 172.21.15.67:0/1836391907 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f23180e7050 con 0x7f23241083a0 2024-09-21T06:48:37.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.884+0000 7f23237fe640 1 --2- 172.21.15.67:0/1836391907 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f230c0b6510 0x7f230c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:37.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.885+0000 7f23237fe640 1 --2- 172.21.15.67:0/1836391907 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f230c0b6510 0x7f230c0b8a20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f231400bfd0 tx=0x7f231400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:37.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:37.888+0000 7f2320ff9640 1 -- 172.21.15.67:0/1836391907 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2318009de0 con 0x7f23241083a0 2024-09-21T06:48:37.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:37 smithi203 ceph-mon[33735]: pgmap v721: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 426 B/s wr, 3 op/s 2024-09-21T06:48:38.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:37 smithi100 ceph-mon[33401]: pgmap v721: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 426 B/s wr, 3 op/s 2024-09-21T06:48:38.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:38.078+0000 7f232a0e7640 1 -- 172.21.15.67:0/1836391907 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23241a9180 con 0x7f23241083a0 2024-09-21T06:48:38.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:38.080+0000 7f2320ff9640 1 -- 172.21.15.67:0/1836391907 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24954 (secure 0 0 0) 0x7f23180cee20 con 0x7f23241083a0 2024-09-21T06:48:38.082 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-21T06:48:38.082 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":85,"fsid":"2a791768-77e2-11ef-baf5-efdc52797490","created":"2024-09-21T06:26:15.222283+0000","modified":"2024-09-21T06:36:05.617384+0000","last_up_change":"2024-09-21T06:33:40.886310+0000","last_in_change":"2024-09-21T06:33:17.339486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:29:50.212825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:34:29.082881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:34:31.301094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:34:37.681356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"18778364-82ed-41c0-92da-ce44cdad7358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6803","nonce":2310759946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6805","nonce":2310759946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6809","nonce":2310759946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2310759946},{"type":"v1","addr":"172.21.15.67:6807","nonce":2310759946}]},"public_addr":"172.21.15.67:6803/2310759946","cluster_addr":"172.21.15.67:6805/2310759946","heartbeat_back_addr":"172.21.15.67:6809/2310759946","heartbeat_front_addr":"172.21.15.67:6807/2310759946","state":["exists","up"]},{"osd":1,"uuid":"84e8d0c2-1c45-4673-ad49-f61df667d7f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6801","nonce":161807364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6803","nonce":161807364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6807","nonce":161807364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":161807364},{"type":"v1","addr":"172.21.15.100:6805","nonce":161807364}]},"public_addr":"172.21.15.100:6801/161807364","cluster_addr":"172.21.15.100:6803/161807364","heartbeat_back_addr":"172.21.15.100:6807/161807364","heartbeat_front_addr":"172.21.15.100:6805/161807364","state":["exists","up"]},{"osd":2,"uuid":"e9a4d919-a779-4a1b-9b5b-d03a65f28ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6801","nonce":2754787169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6803","nonce":2754787169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6807","nonce":2754787169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2754787169},{"type":"v1","addr":"172.21.15.203:6805","nonce":2754787169}]},"public_addr":"172.21.15.203:6801/2754787169","cluster_addr":"172.21.15.203:6803/2754787169","heartbeat_back_addr":"172.21.15.203:6807/2754787169","heartbeat_front_addr":"172.21.15.203:6805/2754787169","state":["exists","up"]},{"osd":3,"uuid":"3f7d9abd-587c-47ad-bf43-11f7dcdf71c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6811","nonce":4007517951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6813","nonce":4007517951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6817","nonce":4007517951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":4007517951},{"type":"v1","addr":"172.21.15.67:6815","nonce":4007517951}]},"public_addr":"172.21.15.67:6811/4007517951","cluster_addr":"172.21.15.67:6813/4007517951","heartbeat_back_addr":"172.21.15.67:6817/4007517951","heartbeat_front_addr":"172.21.15.67:6815/4007517951","state":["exists","up"]},{"osd":4,"uuid":"94f3a075-6d8a-4907-8418-a8513b35779a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6809","nonce":3773646501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6811","nonce":3773646501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6815","nonce":3773646501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3773646501},{"type":"v1","addr":"172.21.15.100:6813","nonce":3773646501}]},"public_addr":"172.21.15.100:6809/3773646501","cluster_addr":"172.21.15.100:6811/3773646501","heartbeat_back_addr":"172.21.15.100:6815/3773646501","heartbeat_front_addr":"172.21.15.100:6813/3773646501","state":["exists","up"]},{"osd":5,"uuid":"5d842edf-06ad-49ae-ac71-863e5657270c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6809","nonce":2096171166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6811","nonce":2096171166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6815","nonce":2096171166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2096171166},{"type":"v1","addr":"172.21.15.203:6813","nonce":2096171166}]},"public_addr":"172.21.15.203:6809/2096171166","cluster_addr":"172.21.15.203:6811/2096171166","heartbeat_back_addr":"172.21.15.203:6815/2096171166","heartbeat_front_addr":"172.21.15.203:6813/2096171166","state":["exists","up"]},{"osd":6,"uuid":"85b9beed-c93d-48ba-bc7a-c4cd816fbdff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6819","nonce":3229936280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6821","nonce":3229936280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6825","nonce":3229936280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3229936280},{"type":"v1","addr":"172.21.15.67:6823","nonce":3229936280}]},"public_addr":"172.21.15.67:6819/3229936280","cluster_addr":"172.21.15.67:6821/3229936280","heartbeat_back_addr":"172.21.15.67:6825/3229936280","heartbeat_front_addr":"172.21.15.67:6823/3229936280","state":["exists","up"]},{"osd":7,"uuid":"86de30c2-b8ee-4c9d-9c17-8a56517934ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6817","nonce":1622917909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6819","nonce":1622917909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6823","nonce":1622917909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1622917909},{"type":"v1","addr":"172.21.15.100:6821","nonce":1622917909}]},"public_addr":"172.21.15.100:6817/1622917909","cluster_addr":"172.21.15.100:6819/1622917909","heartbeat_back_addr":"172.21.15.100:6823/1622917909","heartbeat_front_addr":"172.21.15.100:6821/1622917909","state":["exists","up"]},{"osd":8,"uuid":"0e96cdad-a135-4115-9e2d-9a28306700e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6817","nonce":3452445943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6819","nonce":3452445943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6823","nonce":3452445943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3452445943},{"type":"v1","addr":"172.21.15.203:6821","nonce":3452445943}]},"public_addr":"172.21.15.203:6817/3452445943","cluster_addr":"172.21.15.203:6819/3452445943","heartbeat_back_addr":"172.21.15.203:6823/3452445943","heartbeat_front_addr":"172.21.15.203:6821/3452445943","state":["exists","up"]},{"osd":9,"uuid":"95704ed6-6286-4094-9c6d-23919c7005ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6827","nonce":2998307533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6829","nonce":2998307533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6833","nonce":2998307533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2998307533},{"type":"v1","addr":"172.21.15.67:6831","nonce":2998307533}]},"public_addr":"172.21.15.67:6827/2998307533","cluster_addr":"172.21.15.67:6829/2998307533","heartbeat_back_addr":"172.21.15.67:6833/2998307533","heartbeat_front_addr":"172.21.15.67:6831/2998307533","state":["exists","up"]},{"osd":10,"uuid":"bd2a7d33-b935-46c8-95c3-d76f28242ffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6825","nonce":2819057142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6827","nonce":2819057142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6831","nonce":2819057142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2819057142},{"type":"v1","addr":"172.21.15.100:6829","nonce":2819057142}]},"public_addr":"172.21.15.100:6825/2819057142","cluster_addr":"172.21.15.100:6827/2819057142","heartbeat_back_addr":"172.21.15.100:6831/2819057142","heartbeat_front_addr":"172.21.15.100:6829/2819057142","state":["exists","up"]},{"osd":11,"uuid":"ea591507-10e2-4c08-8d29-d38ba9be96ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6825","nonce":566502600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6827","nonce":566502600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6831","nonce":566502600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":566502600},{"type":"v1","addr":"172.21.15.203:6829","nonce":566502600}]},"public_addr":"172.21.15.203:6825/566502600","cluster_addr":"172.21.15.203:6827/566502600","heartbeat_back_addr":"172.21.15.203:6831/566502600","heartbeat_front_addr":"172.21.15.203:6829/566502600","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:28:58.945733+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:29:23.597118+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:29:48.303563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:30:14.139139+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:30:39.021508+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:04.981241+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:30.853841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:31:56.452967+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:32:22.239036+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:32:47.998803+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:33:13.850759+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:33:39.282150+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6800/1181476774":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/3100321138":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/2309364298":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/3458717076":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/4110206171":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6801/1181476774":"2024-09-22T06:27:03.740475+0000","172.21.15.67:0/2910038982":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6801/2622729103":"2024-09-22T06:26:37.041377+0000","172.21.15.67:6800/2622729103":"2024-09-22T06:26:37.041377+0000","172.21.15.67:0/3346784528":"2024-09-22T06:26:37.041377+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:48:38.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:38.083+0000 7f232a0e7640 1 -- 172.21.15.67:0/1836391907 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f230c0b6510 msgr2=0x7f230c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:38.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:38.083+0000 7f232a0e7640 1 --2- 172.21.15.67:0/1836391907 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f230c0b6510 0x7f230c0b8a20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f231400bfd0 tx=0x7f231400b390 comp rx=0 tx=0).stop 2024-09-21T06:48:38.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:38.083+0000 7f232a0e7640 1 -- 172.21.15.67:0/1836391907 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23241083a0 msgr2=0x7f232419cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:38.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:38.083+0000 7f232a0e7640 1 --2- 172.21.15.67:0/1836391907 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23241083a0 0x7f232419cb90 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f231800d770 tx=0x7f231800dc30 comp rx=0 tx=0).stop 2024-09-21T06:48:38.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:38.083+0000 7f232a0e7640 1 -- 172.21.15.67:0/1836391907 shutdown_connections 2024-09-21T06:48:38.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:38.083+0000 7f232a0e7640 1 --2- 172.21.15.67:0/1836391907 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f230c0b6510 0x7f230c0b8a20 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:38.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:38.083+0000 7f232a0e7640 1 --2- 172.21.15.67:0/1836391907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2324108cf0 0x7f23241a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:38.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:38.083+0000 7f232a0e7640 1 --2- 172.21.15.67:0/1836391907 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23241083a0 0x7f232419cb90 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:38.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:38.083+0000 7f232a0e7640 1 --2- 172.21.15.67:0/1836391907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2324102a60 0x7f232419c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:38.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:38.083+0000 7f232a0e7640 1 -- 172.21.15.67:0/1836391907 >> 172.21.15.67:0/1836391907 conn(0x7f2324079730 msgr2=0x7f232406b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:38.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:38.084+0000 7f232a0e7640 1 -- 172.21.15.67:0/1836391907 shutdown_connections 2024-09-21T06:48:38.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:38.084+0000 7f232a0e7640 1 -- 172.21.15.67:0/1836391907 wait complete. 2024-09-21T06:48:38.805 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs fail cephfs 2024-09-21T06:48:38.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:38 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1836391907' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:48:39.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:38 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1836391907' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:48:39.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:38 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1836391907' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:48:39.066 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:48:39.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.651+0000 7f2f8d109640 1 -- 172.21.15.67:0/287517018 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f88102a90 msgr2=0x7f2f88102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:39.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.651+0000 7f2f8d109640 1 --2- 172.21.15.67:0/287517018 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f88102a90 0x7f2f88102ea0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2f78009900 tx=0x7f2f7802a410 comp rx=0 tx=0).stop 2024-09-21T06:48:39.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.652+0000 7f2f8d109640 1 -- 172.21.15.67:0/287517018 shutdown_connections 2024-09-21T06:48:39.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.652+0000 7f2f8d109640 1 --2- 172.21.15.67:0/287517018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f881083d0 0x7f2f881087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:39.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.652+0000 7f2f8d109640 1 --2- 172.21.15.67:0/287517018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f88108d20 0x7f2f8810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:39.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.652+0000 7f2f8d109640 1 --2- 172.21.15.67:0/287517018 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f88102a90 0x7f2f88102ea0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:39.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.652+0000 7f2f8d109640 1 -- 172.21.15.67:0/287517018 >> 172.21.15.67:0/287517018 conn(0x7f2f88079750 msgr2=0x7f2f88079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:39.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.652+0000 7f2f8d109640 1 -- 172.21.15.67:0/287517018 shutdown_connections 2024-09-21T06:48:39.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.652+0000 7f2f8d109640 1 -- 172.21.15.67:0/287517018 wait complete. 2024-09-21T06:48:39.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.652+0000 7f2f8d109640 1 Processor -- start 2024-09-21T06:48:39.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.653+0000 7f2f8d109640 1 -- start start 2024-09-21T06:48:39.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.653+0000 7f2f8d109640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f88102a90 0x7f2f8819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:39.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.653+0000 7f2f8d109640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f881083d0 0x7f2f8819a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:39.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.653+0000 7f2f8d109640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f88108d20 0x7f2f8819fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:39.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.653+0000 7f2f8d109640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2f88112840 con 0x7f2f88108d20 2024-09-21T06:48:39.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.653+0000 7f2f8d109640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2f881126c0 con 0x7f2f88102a90 2024-09-21T06:48:39.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.653+0000 7f2f8d109640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2f881129c0 con 0x7f2f881083d0 2024-09-21T06:48:39.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.653+0000 7f2f8759e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f88108d20 0x7f2f8819fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:39.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.653+0000 7f2f8759e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f88108d20 0x7f2f8819fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37792/0 (socket says 172.21.15.67:37792) 2024-09-21T06:48:39.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.653+0000 7f2f8759e640 1 -- 172.21.15.67:0/1001569091 learned_addr learned my addr 172.21.15.67:0/1001569091 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:48:39.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.653+0000 7f2f86d9d640 1 --2- 172.21.15.67:0/1001569091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f88102a90 0x7f2f8819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:39.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.653+0000 7f2f8659c640 1 --2- 172.21.15.67:0/1001569091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f881083d0 0x7f2f8819a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:39.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.654+0000 7f2f8759e640 1 -- 172.21.15.67:0/1001569091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f881083d0 msgr2=0x7f2f8819a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:39.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.654+0000 7f2f8759e640 1 --2- 172.21.15.67:0/1001569091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f881083d0 0x7f2f8819a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:39.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.654+0000 7f2f8759e640 1 -- 172.21.15.67:0/1001569091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f88102a90 msgr2=0x7f2f8819a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:39.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.654+0000 7f2f8759e640 1 --2- 172.21.15.67:0/1001569091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f88102a90 0x7f2f8819a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:39.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.654+0000 7f2f8759e640 1 -- 172.21.15.67:0/1001569091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f881a02b0 con 0x7f2f88108d20 2024-09-21T06:48:39.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.654+0000 7f2f8759e640 1 --2- 172.21.15.67:0/1001569091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f88108d20 0x7f2f8819fb80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2f8000eaa0 tx=0x7f2f8000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:39.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.654+0000 7f2f6ffff640 1 -- 172.21.15.67:0/1001569091 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8000cdb0 con 0x7f2f88108d20 2024-09-21T06:48:39.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.654+0000 7f2f6ffff640 1 -- 172.21.15.67:0/1001569091 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f80005020 con 0x7f2f88108d20 2024-09-21T06:48:39.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.654+0000 7f2f8d109640 1 -- 172.21.15.67:0/1001569091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f881a05a0 con 0x7f2f88108d20 2024-09-21T06:48:39.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.654+0000 7f2f6ffff640 1 -- 172.21.15.67:0/1001569091 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f800116d0 con 0x7f2f88108d20 2024-09-21T06:48:39.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.654+0000 7f2f8d109640 1 -- 172.21.15.67:0/1001569091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f881a6d00 con 0x7f2f88108d20 2024-09-21T06:48:39.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.655+0000 7f2f8d109640 1 -- 172.21.15.67:0/1001569091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f88103d10 con 0x7f2f88108d20 2024-09-21T06:48:39.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.657+0000 7f2f6ffff640 1 -- 172.21.15.67:0/1001569091 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f2f800051d0 con 0x7f2f88108d20 2024-09-21T06:48:39.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.658+0000 7f2f6ffff640 1 --2- 172.21.15.67:0/1001569091 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f2f700b6510 0x7f2f700b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:39.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.658+0000 7f2f86d9d640 1 --2- 172.21.15.67:0/1001569091 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f2f700b6510 0x7f2f700b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:39.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.658+0000 7f2f6ffff640 1 -- 172.21.15.67:0/1001569091 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f2f800e7ba0 con 0x7f2f88108d20 2024-09-21T06:48:39.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.658+0000 7f2f86d9d640 1 --2- 172.21.15.67:0/1001569091 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f2f700b6510 0x7f2f700b8a20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2f780057c0 tx=0x7f2f780058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:39.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.661+0000 7f2f6ffff640 1 -- 172.21.15.67:0/1001569091 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2f80016030 con 0x7f2f88108d20 2024-09-21T06:48:39.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.893+0000 7f2f8d109640 1 -- 172.21.15.67:0/1001569091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f2f88102ea0 con 0x7f2f88108d20 2024-09-21T06:48:39.894 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:39 smithi067 ceph-mon[31232]: pgmap v722: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 426 B/s wr, 2 op/s 2024-09-21T06:48:39.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.895+0000 7f2f6ffff640 1 -- 172.21.15.67:0/1001569091 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v22) ==== 161+0+0 (secure 0 0 0) 0x7f2f800b2440 con 0x7f2f88108d20 2024-09-21T06:48:39.897 INFO:teuthology.orchestra.run.smithi067.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T06:48:39.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.898+0000 7f2f8d109640 1 -- 172.21.15.67:0/1001569091 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f2f700b6510 msgr2=0x7f2f700b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:39.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.898+0000 7f2f8d109640 1 --2- 172.21.15.67:0/1001569091 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f2f700b6510 0x7f2f700b8a20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2f780057c0 tx=0x7f2f780058d0 comp rx=0 tx=0).stop 2024-09-21T06:48:39.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.898+0000 7f2f8d109640 1 -- 172.21.15.67:0/1001569091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f88108d20 msgr2=0x7f2f8819fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:39.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.898+0000 7f2f8d109640 1 --2- 172.21.15.67:0/1001569091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f88108d20 0x7f2f8819fb80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2f8000eaa0 tx=0x7f2f8000ef60 comp rx=0 tx=0).stop 2024-09-21T06:48:39.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.899+0000 7f2f8d109640 1 -- 172.21.15.67:0/1001569091 shutdown_connections 2024-09-21T06:48:39.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.899+0000 7f2f8d109640 1 --2- 172.21.15.67:0/1001569091 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f2f700b6510 0x7f2f700b8a20 secure :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2f780057c0 tx=0x7f2f780058d0 comp rx=0 tx=0).stop 2024-09-21T06:48:39.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.899+0000 7f2f8d109640 1 --2- 172.21.15.67:0/1001569091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f88108d20 0x7f2f8819fb80 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:39.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.899+0000 7f2f8d109640 1 --2- 172.21.15.67:0/1001569091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f881083d0 0x7f2f8819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:39.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.899+0000 7f2f8d109640 1 --2- 172.21.15.67:0/1001569091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f88102a90 0x7f2f8819a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:39.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.899+0000 7f2f8d109640 1 -- 172.21.15.67:0/1001569091 >> 172.21.15.67:0/1001569091 conn(0x7f2f88079750 msgr2=0x7f2f88101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:39.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.899+0000 7f2f8d109640 1 -- 172.21.15.67:0/1001569091 shutdown_connections 2024-09-21T06:48:39.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:39.899+0000 7f2f8d109640 1 -- 172.21.15.67:0/1001569091 wait complete. 2024-09-21T06:48:39.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:39 smithi203 ceph-mon[33735]: pgmap v722: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 426 B/s wr, 2 op/s 2024-09-21T06:48:40.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:39 smithi100 ceph-mon[33401]: pgmap v722: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 426 B/s wr, 2 op/s 2024-09-21T06:48:40.054 INFO:journalctl@ceph.mds.b.smithi100.stdout:Sep 21 06:48:39 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-b[61079]: ignoring --setuser ceph since I am not root 2024-09-21T06:48:40.054 INFO:journalctl@ceph.mds.b.smithi100.stdout:Sep 21 06:48:39 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-b[61079]: ignoring --setgroup ceph since I am not root 2024-09-21T06:48:40.054 INFO:journalctl@ceph.mds.b.smithi100.stdout:Sep 21 06:48:39 smithi100 ceph-mds[61083]: -- 172.21.15.100:0/1097987827 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56011910d860 con 0x56011d2b5600 2024-09-21T06:48:40.054 INFO:journalctl@ceph.mds.b.smithi100.stdout:Sep 21 06:48:39 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-b[61079]: starting mds.b at 2024-09-21T06:48:40.054 INFO:journalctl@ceph.mds.h.smithi100.stdout:Sep 21 06:48:39 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-h[60800]: ignoring --setuser ceph since I am not root 2024-09-21T06:48:40.055 INFO:journalctl@ceph.mds.h.smithi100.stdout:Sep 21 06:48:39 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-h[60800]: ignoring --setgroup ceph since I am not root 2024-09-21T06:48:40.055 INFO:journalctl@ceph.mds.h.smithi100.stdout:Sep 21 06:48:39 smithi100 ceph-mds[60804]: -- 172.21.15.100:0/710432016 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5613b1675860 con 0x5613b581da80 2024-09-21T06:48:40.055 INFO:journalctl@ceph.mds.h.smithi100.stdout:Sep 21 06:48:39 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-h[60800]: starting mds.h at 2024-09-21T06:48:40.226 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:39 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mon-a[31228]: 2024-09-21T06:48:39.894+0000 7f2f2bb85640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T06:48:40.575 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T06:48:40.842 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:48:40.963 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:40 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1001569091' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T06:48:40.964 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:40 smithi067 ceph-mon[31232]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T06:48:40.964 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:40 smithi067 ceph-mon[31232]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T06:48:40.964 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:40 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/1001569091' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T06:48:40.964 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:40 smithi067 ceph-mon[31232]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T06:48:40.964 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:40 smithi067 ceph-mon[31232]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-09-21T06:48:40.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:40 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1001569091' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T06:48:40.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:40 smithi203 ceph-mon[33735]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T06:48:40.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:40 smithi203 ceph-mon[33735]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T06:48:40.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:40 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/1001569091' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T06:48:40.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:40 smithi203 ceph-mon[33735]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T06:48:40.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:40 smithi203 ceph-mon[33735]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-09-21T06:48:41.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:40 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1001569091' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T06:48:41.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:40 smithi100 ceph-mon[33401]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T06:48:41.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:40 smithi100 ceph-mon[33401]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T06:48:41.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:40 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/1001569091' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T06:48:41.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:40 smithi100 ceph-mon[33401]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T06:48:41.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:40 smithi100 ceph-mon[33401]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-09-21T06:48:41.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.374+0000 7f7acf7e6640 1 -- 172.21.15.67:0/2056295281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ac8105380 msgr2=0x7f7ac810d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:41.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.374+0000 7f7acf7e6640 1 --2- 172.21.15.67:0/2056295281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ac8105380 0x7f7ac810d8d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4009900 tx=0x7f7ac402a3f0 comp rx=0 tx=0).stop 2024-09-21T06:48:41.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.374+0000 7f7acf7e6640 1 -- 172.21.15.67:0/2056295281 shutdown_connections 2024-09-21T06:48:41.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.374+0000 7f7acf7e6640 1 --2- 172.21.15.67:0/2056295281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ac810de60 0x7f7ac8110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:41.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.374+0000 7f7acf7e6640 1 --2- 172.21.15.67:0/2056295281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ac8105380 0x7f7ac810d8d0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:41.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.374+0000 7f7acf7e6640 1 --2- 172.21.15.67:0/2056295281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ac8104a30 0x7f7ac8104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:41.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.374+0000 7f7acf7e6640 1 -- 172.21.15.67:0/2056295281 >> 172.21.15.67:0/2056295281 conn(0x7f7ac8079760 msgr2=0x7f7ac8079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:41.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.374+0000 7f7acf7e6640 1 -- 172.21.15.67:0/2056295281 shutdown_connections 2024-09-21T06:48:41.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.374+0000 7f7acf7e6640 1 -- 172.21.15.67:0/2056295281 wait complete. 2024-09-21T06:48:41.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.375+0000 7f7acf7e6640 1 Processor -- start 2024-09-21T06:48:41.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.375+0000 7f7acf7e6640 1 -- start start 2024-09-21T06:48:41.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.375+0000 7f7acf7e6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ac8104a30 0x7f7ac819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:41.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.375+0000 7f7acf7e6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ac8105380 0x7f7ac819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:41.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.375+0000 7f7acf7e6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ac810de60 0x7f7ac819fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:41.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.375+0000 7f7acf7e6640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac8113950 con 0x7f7ac8105380 2024-09-21T06:48:41.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.375+0000 7f7acf7e6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81137d0 con 0x7f7ac8104a30 2024-09-21T06:48:41.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.375+0000 7f7acf7e6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac8113ad0 con 0x7f7ac810de60 2024-09-21T06:48:41.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.375+0000 7f7accd81640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ac8105380 0x7f7ac819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:41.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.375+0000 7f7accd81640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ac8105380 0x7f7ac819a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37816/0 (socket says 172.21.15.67:37816) 2024-09-21T06:48:41.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.375+0000 7f7accd81640 1 -- 172.21.15.67:0/2231008173 learned_addr learned my addr 172.21.15.67:0/2231008173 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:48:41.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.375+0000 7f7acd582640 1 --2- 172.21.15.67:0/2231008173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ac8104a30 0x7f7ac819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:41.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.375+0000 7f7acdd83640 1 --2- 172.21.15.67:0/2231008173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ac810de60 0x7f7ac819fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:41.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.375+0000 7f7accd81640 1 -- 172.21.15.67:0/2231008173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ac810de60 msgr2=0x7f7ac819fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:41.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.375+0000 7f7accd81640 1 --2- 172.21.15.67:0/2231008173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ac810de60 0x7f7ac819fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:41.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.376+0000 7f7accd81640 1 -- 172.21.15.67:0/2231008173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ac8104a30 msgr2=0x7f7ac819a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:41.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.376+0000 7f7accd81640 1 --2- 172.21.15.67:0/2231008173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ac8104a30 0x7f7ac819a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:41.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.376+0000 7f7accd81640 1 -- 172.21.15.67:0/2231008173 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac81a02b0 con 0x7f7ac8105380 2024-09-21T06:48:41.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.376+0000 7f7accd81640 1 --2- 172.21.15.67:0/2231008173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ac8105380 0x7f7ac819a9a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4005940 tx=0x7f7ac40040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:41.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.376+0000 7f7aba7fc640 1 -- 172.21.15.67:0/2231008173 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac403f030 con 0x7f7ac8105380 2024-09-21T06:48:41.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.376+0000 7f7acf7e6640 1 -- 172.21.15.67:0/2231008173 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ac81a0540 con 0x7f7ac8105380 2024-09-21T06:48:41.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.376+0000 7f7aba7fc640 1 -- 172.21.15.67:0/2231008173 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ac4003a80 con 0x7f7ac8105380 2024-09-21T06:48:41.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.376+0000 7f7aba7fc640 1 -- 172.21.15.67:0/2231008173 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4035070 con 0x7f7ac8105380 2024-09-21T06:48:41.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.376+0000 7f7acf7e6640 1 -- 172.21.15.67:0/2231008173 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ac81a0a70 con 0x7f7ac8105380 2024-09-21T06:48:41.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.377+0000 7f7acf7e6640 1 -- 172.21.15.67:0/2231008173 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ac806c1f0 con 0x7f7ac8105380 2024-09-21T06:48:41.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.379+0000 7f7aba7fc640 1 -- 172.21.15.67:0/2231008173 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f7ac4042030 con 0x7f7ac8105380 2024-09-21T06:48:41.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.380+0000 7f7aba7fc640 1 --2- 172.21.15.67:0/2231008173 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f7ab40b6540 0x7f7ab40b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:41.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.380+0000 7f7acd582640 1 --2- 172.21.15.67:0/2231008173 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f7ab40b6540 0x7f7ab40b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:41.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.380+0000 7f7aba7fc640 1 -- 172.21.15.67:0/2231008173 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9294+0+0 (secure 0 0 0) 0x7f7ac4108040 con 0x7f7ac8105380 2024-09-21T06:48:41.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.380+0000 7f7acd582640 1 --2- 172.21.15.67:0/2231008173 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f7ab40b6540 0x7f7ab40b8a50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7ac806c8f0 tx=0x7f7abc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:41.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.383+0000 7f7aba7fc640 1 -- 172.21.15.67:0/2231008173 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7ac81a0a70 con 0x7f7ac8105380 2024-09-21T06:48:41.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.611+0000 7f7acf7e6640 1 -- 172.21.15.67:0/2231008173 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f7ac819b530 con 0x7f7ac8105380 2024-09-21T06:48:41.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.903+0000 7f7aba7fc640 1 -- 172.21.15.67:0/2231008173 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v24) ==== 104+0+0 (secure 0 0 0) 0x7f7ac819b530 con 0x7f7ac8105380 2024-09-21T06:48:41.905 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:41 smithi067 ceph-mon[31232]: pgmap v724: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-21T06:48:41.905 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:41 smithi067 ceph-mon[31232]: mds.? [v2:172.21.15.100:6836/2642491904,v1:172.21.15.100:6838/2642491904] up:boot 2024-09-21T06:48:41.905 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:41 smithi067 ceph-mon[31232]: mds.? [v2:172.21.15.100:6837/3452088387,v1:172.21.15.100:6839/3452088387] up:boot 2024-09-21T06:48:41.905 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:41 smithi067 ceph-mon[31232]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-09-21T06:48:41.905 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:41 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T06:48:41.905 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:41 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T06:48:41.905 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:41 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2231008173' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:48:41.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.905+0000 7f7acf7e6640 1 -- 172.21.15.67:0/2231008173 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f7ab40b6540 msgr2=0x7f7ab40b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:41.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.906+0000 7f7acf7e6640 1 --2- 172.21.15.67:0/2231008173 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f7ab40b6540 0x7f7ab40b8a50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7ac806c8f0 tx=0x7f7abc005f90 comp rx=0 tx=0).stop 2024-09-21T06:48:41.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.906+0000 7f7acf7e6640 1 -- 172.21.15.67:0/2231008173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ac8105380 msgr2=0x7f7ac819a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:41.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.906+0000 7f7acf7e6640 1 --2- 172.21.15.67:0/2231008173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ac8105380 0x7f7ac819a9a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4005940 tx=0x7f7ac40040e0 comp rx=0 tx=0).stop 2024-09-21T06:48:41.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.906+0000 7f7acf7e6640 1 -- 172.21.15.67:0/2231008173 shutdown_connections 2024-09-21T06:48:41.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.906+0000 7f7acf7e6640 1 --2- 172.21.15.67:0/2231008173 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f7ab40b6540 0x7f7ab40b8a50 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:41.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.906+0000 7f7acf7e6640 1 --2- 172.21.15.67:0/2231008173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ac810de60 0x7f7ac819fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:41.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.906+0000 7f7acf7e6640 1 --2- 172.21.15.67:0/2231008173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ac8105380 0x7f7ac819a9a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:41.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.906+0000 7f7acf7e6640 1 --2- 172.21.15.67:0/2231008173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ac8104a30 0x7f7ac819a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:41.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.906+0000 7f7acf7e6640 1 -- 172.21.15.67:0/2231008173 >> 172.21.15.67:0/2231008173 conn(0x7f7ac8079760 msgr2=0x7f7ac81009c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:41.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.906+0000 7f7acf7e6640 1 -- 172.21.15.67:0/2231008173 shutdown_connections 2024-09-21T06:48:41.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:41.906+0000 7f7acf7e6640 1 -- 172.21.15.67:0/2231008173 wait complete. 2024-09-21T06:48:41.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:41 smithi203 ceph-mon[33735]: pgmap v724: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-21T06:48:41.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:41 smithi203 ceph-mon[33735]: mds.? [v2:172.21.15.100:6836/2642491904,v1:172.21.15.100:6838/2642491904] up:boot 2024-09-21T06:48:41.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:41 smithi203 ceph-mon[33735]: mds.? [v2:172.21.15.100:6837/3452088387,v1:172.21.15.100:6839/3452088387] up:boot 2024-09-21T06:48:41.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:41 smithi203 ceph-mon[33735]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-09-21T06:48:41.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:41 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T06:48:41.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:41 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T06:48:41.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:41 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2231008173' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:48:42.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:41 smithi100 ceph-mon[33401]: pgmap v724: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-21T06:48:42.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:41 smithi100 ceph-mon[33401]: mds.? [v2:172.21.15.100:6836/2642491904,v1:172.21.15.100:6838/2642491904] up:boot 2024-09-21T06:48:42.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:41 smithi100 ceph-mon[33401]: mds.? [v2:172.21.15.100:6837/3452088387,v1:172.21.15.100:6839/3452088387] up:boot 2024-09-21T06:48:42.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:41 smithi100 ceph-mon[33401]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-09-21T06:48:42.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:41 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T06:48:42.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:41 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T06:48:42.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:41 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2231008173' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:48:42.550 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T06:48:42.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:42 smithi067 ceph-mon[31232]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T06:48:42.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:42 smithi067 ceph-mon[31232]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T06:48:42.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:42 smithi067 ceph-mon[31232]: Cluster is now healthy 2024-09-21T06:48:42.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:42 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/2231008173' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T06:48:42.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:42 smithi067 ceph-mon[31232]: fsmap 12 up:standby 2024-09-21T06:48:42.811 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:48:42.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:42 smithi203 ceph-mon[33735]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T06:48:42.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:42 smithi203 ceph-mon[33735]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T06:48:42.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:42 smithi203 ceph-mon[33735]: Cluster is now healthy 2024-09-21T06:48:42.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:42 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/2231008173' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T06:48:42.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:42 smithi203 ceph-mon[33735]: fsmap 12 up:standby 2024-09-21T06:48:43.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:42 smithi100 ceph-mon[33401]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T06:48:43.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:42 smithi100 ceph-mon[33401]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T06:48:43.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:42 smithi100 ceph-mon[33401]: Cluster is now healthy 2024-09-21T06:48:43.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:42 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/2231008173' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T06:48:43.055 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:42 smithi100 ceph-mon[33401]: fsmap 12 up:standby 2024-09-21T06:48:43.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.792+0000 7f05c179a640 1 -- 172.21.15.67:0/4045030502 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05bc109e40 msgr2=0x7f05bc110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:43.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.792+0000 7f05c179a640 1 --2- 172.21.15.67:0/4045030502 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05bc109e40 0x7f05bc110670 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f05b4009ca0 tx=0x7f05b402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:48:43.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.792+0000 7f05c179a640 1 -- 172.21.15.67:0/4045030502 shutdown_connections 2024-09-21T06:48:43.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.792+0000 7f05c179a640 1 --2- 172.21.15.67:0/4045030502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05bc1094f0 0x7f05bc109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:43.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.792+0000 7f05c179a640 1 --2- 172.21.15.67:0/4045030502 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05bc109e40 0x7f05bc110670 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:43.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.792+0000 7f05c179a640 1 --2- 172.21.15.67:0/4045030502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05bc103bb0 0x7f05bc103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:43.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.792+0000 7f05c179a640 1 -- 172.21.15.67:0/4045030502 >> 172.21.15.67:0/4045030502 conn(0x7f05bc0ff600 msgr2=0x7f05bc101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:43.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.792+0000 7f05c179a640 1 -- 172.21.15.67:0/4045030502 shutdown_connections 2024-09-21T06:48:43.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.792+0000 7f05c179a640 1 -- 172.21.15.67:0/4045030502 wait complete. 2024-09-21T06:48:43.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.793+0000 7f05c179a640 1 Processor -- start 2024-09-21T06:48:43.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.793+0000 7f05c179a640 1 -- start start 2024-09-21T06:48:43.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.793+0000 7f05c179a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05bc103bb0 0x7f05bc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:43.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.793+0000 7f05c179a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05bc1094f0 0x7f05bc075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:43.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.793+0000 7f05c179a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05bc109e40 0x7f05bc076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:43.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.793+0000 7f05c179a640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc113a00 con 0x7f05bc1094f0 2024-09-21T06:48:43.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.793+0000 7f05c179a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc113880 con 0x7f05bc109e40 2024-09-21T06:48:43.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.793+0000 7f05c179a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc113b80 con 0x7f05bc103bb0 2024-09-21T06:48:43.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.793+0000 7f05b3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05bc1094f0 0x7f05bc075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:43.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.794+0000 7f05b3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05bc1094f0 0x7f05bc075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37846/0 (socket says 172.21.15.67:37846) 2024-09-21T06:48:43.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.794+0000 7f05b3fff640 1 -- 172.21.15.67:0/3618903227 learned_addr learned my addr 172.21.15.67:0/3618903227 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:48:43.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.794+0000 7f05baffd640 1 --2- 172.21.15.67:0/3618903227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05bc103bb0 0x7f05bc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:43.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.794+0000 7f05bb7fe640 1 --2- 172.21.15.67:0/3618903227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05bc109e40 0x7f05bc076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:43.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.794+0000 7f05b3fff640 1 -- 172.21.15.67:0/3618903227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05bc103bb0 msgr2=0x7f05bc075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:43.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.794+0000 7f05b3fff640 1 --2- 172.21.15.67:0/3618903227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05bc103bb0 0x7f05bc075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:43.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.794+0000 7f05b3fff640 1 -- 172.21.15.67:0/3618903227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05bc109e40 msgr2=0x7f05bc076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:43.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.794+0000 7f05b3fff640 1 --2- 172.21.15.67:0/3618903227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05bc109e40 0x7f05bc076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:43.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.794+0000 7f05b3fff640 1 -- 172.21.15.67:0/3618903227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05bc1a8690 con 0x7f05bc1094f0 2024-09-21T06:48:43.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.794+0000 7f05baffd640 1 --2- 172.21.15.67:0/3618903227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05bc103bb0 0x7f05bc075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:48:43.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.794+0000 7f05b3fff640 1 --2- 172.21.15.67:0/3618903227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05bc1094f0 0x7f05bc075c50 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f05a400d740 tx=0x7f05a400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:43.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.794+0000 7f05b8ff9640 1 -- 172.21.15.67:0/3618903227 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a4017070 con 0x7f05bc1094f0 2024-09-21T06:48:43.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.794+0000 7f05c179a640 1 -- 172.21.15.67:0/3618903227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05bc1a8980 con 0x7f05bc1094f0 2024-09-21T06:48:43.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.795+0000 7f05b8ff9640 1 -- 172.21.15.67:0/3618903227 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f05a400de50 con 0x7f05bc1094f0 2024-09-21T06:48:43.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.795+0000 7f05c179a640 1 -- 172.21.15.67:0/3618903227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05bc1a8f10 con 0x7f05bc1094f0 2024-09-21T06:48:43.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.795+0000 7f05b8ff9640 1 -- 172.21.15.67:0/3618903227 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a400f620 con 0x7f05bc1094f0 2024-09-21T06:48:43.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.796+0000 7f05c179a640 1 -- 172.21.15.67:0/3618903227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05bc103fc0 con 0x7f05bc1094f0 2024-09-21T06:48:43.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.798+0000 7f05b8ff9640 1 -- 172.21.15.67:0/3618903227 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f05a401a070 con 0x7f05bc1094f0 2024-09-21T06:48:43.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.798+0000 7f05b8ff9640 1 --2- 172.21.15.67:0/3618903227 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f05a80b6590 0x7f05a80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:43.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.799+0000 7f05baffd640 1 --2- 172.21.15.67:0/3618903227 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f05a80b6590 0x7f05a80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:43.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.799+0000 7f05b8ff9640 1 -- 172.21.15.67:0/3618903227 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9294+0+0 (secure 0 0 0) 0x7f05a40e6fc0 con 0x7f05bc1094f0 2024-09-21T06:48:43.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.799+0000 7f05baffd640 1 --2- 172.21.15.67:0/3618903227 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f05a80b6590 0x7f05a80b8aa0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f05bc077160 tx=0x7f05ac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:43.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.801+0000 7f05b8ff9640 1 -- 172.21.15.67:0/3618903227 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f05a40b1890 con 0x7f05bc1094f0 2024-09-21T06:48:43.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:43 smithi203 ceph-mon[33735]: pgmap v725: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-09-21T06:48:43.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:43.992+0000 7f05c179a640 1 -- 172.21.15.67:0/3618903227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f05bc076a10 con 0x7f05bc1094f0 2024-09-21T06:48:43.993 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:43 smithi067 ceph-mon[31232]: pgmap v725: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-09-21T06:48:44.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:43 smithi100 ceph-mon[33401]: pgmap v725: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-09-21T06:48:44.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:44.732+0000 7f05b8ff9640 1 -- 172.21.15.67:0/3618903227 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v87) ==== 181+0+0 (secure 0 0 0) 0x7f05bc076a10 con 0x7f05bc1094f0 2024-09-21T06:48:44.734 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' removed 2024-09-21T06:48:44.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:44.735+0000 7f05c179a640 1 -- 172.21.15.67:0/3618903227 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f05a80b6590 msgr2=0x7f05a80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:44.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:44.735+0000 7f05c179a640 1 --2- 172.21.15.67:0/3618903227 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f05a80b6590 0x7f05a80b8aa0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f05bc077160 tx=0x7f05ac005f90 comp rx=0 tx=0).stop 2024-09-21T06:48:44.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:44.735+0000 7f05c179a640 1 -- 172.21.15.67:0/3618903227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05bc1094f0 msgr2=0x7f05bc075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:44.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:44.735+0000 7f05c179a640 1 --2- 172.21.15.67:0/3618903227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05bc1094f0 0x7f05bc075c50 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f05a400d740 tx=0x7f05a400dc00 comp rx=0 tx=0).stop 2024-09-21T06:48:44.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:44.735+0000 7f05c179a640 1 -- 172.21.15.67:0/3618903227 shutdown_connections 2024-09-21T06:48:44.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:44.735+0000 7f05c179a640 1 --2- 172.21.15.67:0/3618903227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05bc109e40 0x7f05bc076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:44.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:44.735+0000 7f05c179a640 1 --2- 172.21.15.67:0/3618903227 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f05a80b6590 0x7f05a80b8aa0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:44.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:44.735+0000 7f05c179a640 1 --2- 172.21.15.67:0/3618903227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05bc1094f0 0x7f05bc075c50 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:44.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:44.735+0000 7f05c179a640 1 --2- 172.21.15.67:0/3618903227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05bc103bb0 0x7f05bc075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:44.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:44.735+0000 7f05c179a640 1 -- 172.21.15.67:0/3618903227 >> 172.21.15.67:0/3618903227 conn(0x7f05bc0ff600 msgr2=0x7f05bc101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:44.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:44.735+0000 7f05c179a640 1 -- 172.21.15.67:0/3618903227 shutdown_connections 2024-09-21T06:48:44.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:44.735+0000 7f05c179a640 1 -- 172.21.15.67:0/3618903227 wait complete. 2024-09-21T06:48:45.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:44 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3618903227' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T06:48:45.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:44 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3618903227' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T06:48:45.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:44 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3618903227' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T06:48:45.484 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T06:48:45.758 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:48:46.017 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:45 smithi067 ceph-mon[31232]: pgmap v726: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-09-21T06:48:46.018 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:45 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3618903227' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T06:48:46.018 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:45 smithi067 ceph-mon[31232]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T06:48:46.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:45 smithi100 ceph-mon[33401]: pgmap v726: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-09-21T06:48:46.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:45 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3618903227' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T06:48:46.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:45 smithi100 ceph-mon[33401]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T06:48:46.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:45 smithi203 ceph-mon[33735]: pgmap v726: 129 pgs: 129 active+clean; 1.4 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-09-21T06:48:46.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:45 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3618903227' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T06:48:46.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:45 smithi203 ceph-mon[33735]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T06:48:46.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.264+0000 7f0b865de640 1 -- 172.21.15.67:0/3442681819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b801094a0 msgr2=0x7f0b801098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:46.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.264+0000 7f0b865de640 1 --2- 172.21.15.67:0/3442681819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b801094a0 0x7f0b801098b0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0b68009900 tx=0x7f0b6802a410 comp rx=0 tx=0).stop 2024-09-21T06:48:46.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.264+0000 7f0b865de640 1 -- 172.21.15.67:0/3442681819 shutdown_connections 2024-09-21T06:48:46.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.264+0000 7f0b865de640 1 --2- 172.21.15.67:0/3442681819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b80109df0 0x7f0b80110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:46.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.264+0000 7f0b865de640 1 --2- 172.21.15.67:0/3442681819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b801094a0 0x7f0b801098b0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:46.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.264+0000 7f0b865de640 1 --2- 172.21.15.67:0/3442681819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b80103b60 0x7f0b80103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:46.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.264+0000 7f0b865de640 1 -- 172.21.15.67:0/3442681819 >> 172.21.15.67:0/3442681819 conn(0x7f0b8007b850 msgr2=0x7f0b8007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:46.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.264+0000 7f0b865de640 1 -- 172.21.15.67:0/3442681819 shutdown_connections 2024-09-21T06:48:46.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.264+0000 7f0b865de640 1 -- 172.21.15.67:0/3442681819 wait complete. 2024-09-21T06:48:46.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.265+0000 7f0b865de640 1 Processor -- start 2024-09-21T06:48:46.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.265+0000 7f0b865de640 1 -- start start 2024-09-21T06:48:46.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.265+0000 7f0b865de640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b80103b60 0x7f0b80075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:46.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.265+0000 7f0b865de640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b801094a0 0x7f0b80075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:46.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.265+0000 7f0b865de640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b80109df0 0x7f0b80076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:46.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.265+0000 7f0b865de640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0b80113960 con 0x7f0b801094a0 2024-09-21T06:48:46.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.265+0000 7f0b865de640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0b801137e0 con 0x7f0b80103b60 2024-09-21T06:48:46.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.265+0000 7f0b865de640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0b80113ae0 con 0x7f0b80109df0 2024-09-21T06:48:46.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.265+0000 7f0b77fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b801094a0 0x7f0b80075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:46.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.265+0000 7f0b77fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b801094a0 0x7f0b80075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37850/0 (socket says 172.21.15.67:37850) 2024-09-21T06:48:46.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.265+0000 7f0b77fff640 1 -- 172.21.15.67:0/3707411980 learned_addr learned my addr 172.21.15.67:0/3707411980 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:48:46.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.266+0000 7f0b84b7b640 1 --2- 172.21.15.67:0/3707411980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b80109df0 0x7f0b80076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:46.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.266+0000 7f0b7ffff640 1 --2- 172.21.15.67:0/3707411980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b80103b60 0x7f0b80075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:46.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.266+0000 7f0b77fff640 1 -- 172.21.15.67:0/3707411980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b80109df0 msgr2=0x7f0b80076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:46.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.267+0000 7f0b77fff640 1 --2- 172.21.15.67:0/3707411980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b80109df0 0x7f0b80076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:46.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.267+0000 7f0b77fff640 1 -- 172.21.15.67:0/3707411980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b80103b60 msgr2=0x7f0b80075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:46.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.267+0000 7f0b77fff640 1 --2- 172.21.15.67:0/3707411980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b80103b60 0x7f0b80075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:46.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.267+0000 7f0b77fff640 1 -- 172.21.15.67:0/3707411980 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b801a86f0 con 0x7f0b801094a0 2024-09-21T06:48:46.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.267+0000 7f0b84b7b640 1 --2- 172.21.15.67:0/3707411980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b80109df0 0x7f0b80076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:48:46.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.267+0000 7f0b7ffff640 1 --2- 172.21.15.67:0/3707411980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b80103b60 0x7f0b80075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:48:46.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.267+0000 7f0b77fff640 1 --2- 172.21.15.67:0/3707411980 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b801094a0 0x7f0b80075c50 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0b68004770 tx=0x7f0b68004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:46.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.268+0000 7f0b7dffb640 1 -- 172.21.15.67:0/3707411980 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6803f030 con 0x7f0b801094a0 2024-09-21T06:48:46.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.268+0000 7f0b7dffb640 1 -- 172.21.15.67:0/3707411980 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0b68004440 con 0x7f0b801094a0 2024-09-21T06:48:46.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.268+0000 7f0b865de640 1 -- 172.21.15.67:0/3707411980 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b801a8980 con 0x7f0b801094a0 2024-09-21T06:48:46.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.268+0000 7f0b7dffb640 1 -- 172.21.15.67:0/3707411980 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b68035070 con 0x7f0b801094a0 2024-09-21T06:48:46.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.268+0000 7f0b865de640 1 -- 172.21.15.67:0/3707411980 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b801a8e60 con 0x7f0b801094a0 2024-09-21T06:48:46.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.269+0000 7f0b865de640 1 -- 172.21.15.67:0/3707411980 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b80103f70 con 0x7f0b801094a0 2024-09-21T06:48:46.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.271+0000 7f0b7dffb640 1 -- 172.21.15.67:0/3707411980 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0b68042030 con 0x7f0b801094a0 2024-09-21T06:48:46.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.271+0000 7f0b7dffb640 1 --2- 172.21.15.67:0/3707411980 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f0b6c0b6510 0x7f0b6c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:46.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.272+0000 7f0b7ffff640 1 --2- 172.21.15.67:0/3707411980 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f0b6c0b6510 0x7f0b6c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:46.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.272+0000 7f0b7dffb640 1 -- 172.21.15.67:0/3707411980 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8740+0+0 (secure 0 0 0) 0x7f0b68107da0 con 0x7f0b801094a0 2024-09-21T06:48:46.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.272+0000 7f0b7ffff640 1 --2- 172.21.15.67:0/3707411980 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f0b6c0b6510 0x7f0b6c0b8a20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0b70005fd0 tx=0x7f0b70005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:46.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.275+0000 7f0b7dffb640 1 -- 172.21.15.67:0/3707411980 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0b680ce430 con 0x7f0b801094a0 2024-09-21T06:48:46.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.465+0000 7f0b865de640 1 -- 172.21.15.67:0/3707411980 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0b8010b650 con 0x7f0b801094a0 2024-09-21T06:48:46.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.742+0000 7f0b7dffb640 1 -- 172.21.15.67:0/3707411980 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v88) ==== 169+0+0 (secure 0 0 0) 0x7f0b680d2810 con 0x7f0b801094a0 2024-09-21T06:48:46.743 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' removed 2024-09-21T06:48:46.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.744+0000 7f0b865de640 1 -- 172.21.15.67:0/3707411980 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f0b6c0b6510 msgr2=0x7f0b6c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:46.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.744+0000 7f0b865de640 1 --2- 172.21.15.67:0/3707411980 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f0b6c0b6510 0x7f0b6c0b8a20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0b70005fd0 tx=0x7f0b70005850 comp rx=0 tx=0).stop 2024-09-21T06:48:46.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.744+0000 7f0b865de640 1 -- 172.21.15.67:0/3707411980 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b801094a0 msgr2=0x7f0b80075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:46.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.744+0000 7f0b865de640 1 --2- 172.21.15.67:0/3707411980 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b801094a0 0x7f0b80075c50 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0b68004770 tx=0x7f0b68004040 comp rx=0 tx=0).stop 2024-09-21T06:48:46.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.744+0000 7f0b865de640 1 -- 172.21.15.67:0/3707411980 shutdown_connections 2024-09-21T06:48:46.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.744+0000 7f0b865de640 1 --2- 172.21.15.67:0/3707411980 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f0b6c0b6510 0x7f0b6c0b8a20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:46.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.744+0000 7f0b865de640 1 --2- 172.21.15.67:0/3707411980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b80109df0 0x7f0b80076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:46.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.744+0000 7f0b865de640 1 --2- 172.21.15.67:0/3707411980 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b801094a0 0x7f0b80075c50 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:46.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.744+0000 7f0b865de640 1 --2- 172.21.15.67:0/3707411980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b80103b60 0x7f0b80075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:46.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.744+0000 7f0b865de640 1 -- 172.21.15.67:0/3707411980 >> 172.21.15.67:0/3707411980 conn(0x7f0b8007b850 msgr2=0x7f0b801076a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:46.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.745+0000 7f0b865de640 1 -- 172.21.15.67:0/3707411980 shutdown_connections 2024-09-21T06:48:46.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:46.745+0000 7f0b865de640 1 -- 172.21.15.67:0/3707411980 wait complete. 2024-09-21T06:48:47.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:46 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3707411980' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T06:48:47.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:46 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3707411980' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T06:48:47.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:46 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3707411980' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T06:48:47.477 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2a791768-77e2-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-21T06:48:47.738 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:48:48.020 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:47 smithi067 ceph-mon[31232]: pgmap v728: 97 pgs: 97 active+clean; 577 KiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:48:48.020 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:47 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/3707411980' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T06:48:48.020 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:47 smithi067 ceph-mon[31232]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T06:48:48.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:47 smithi100 ceph-mon[33401]: pgmap v728: 97 pgs: 97 active+clean; 577 KiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:48:48.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:47 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/3707411980' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T06:48:48.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:47 smithi100 ceph-mon[33401]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T06:48:48.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:47 smithi203 ceph-mon[33735]: pgmap v728: 97 pgs: 97 active+clean; 577 KiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:48:48.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:47 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/3707411980' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T06:48:48.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:47 smithi203 ceph-mon[33735]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T06:48:48.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.250+0000 7f16fe12e640 1 -- 172.21.15.67:0/4275486416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16f8108cf0 msgr2=0x7f16f810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:48.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.250+0000 7f16fe12e640 1 --2- 172.21.15.67:0/4275486416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16f8108cf0 0x7f16f810f520 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f16f0009ca0 tx=0x7f16f002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:48:48.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.250+0000 7f16fe12e640 1 -- 172.21.15.67:0/4275486416 shutdown_connections 2024-09-21T06:48:48.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.250+0000 7f16fe12e640 1 --2- 172.21.15.67:0/4275486416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16f81083a0 0x7f16f81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:48.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.250+0000 7f16fe12e640 1 --2- 172.21.15.67:0/4275486416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16f8108cf0 0x7f16f810f520 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:48.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.250+0000 7f16fe12e640 1 --2- 172.21.15.67:0/4275486416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16f8102a60 0x7f16f8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:48.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.250+0000 7f16fe12e640 1 -- 172.21.15.67:0/4275486416 >> 172.21.15.67:0/4275486416 conn(0x7f16f8079730 msgr2=0x7f16f8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:48.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.250+0000 7f16fe12e640 1 -- 172.21.15.67:0/4275486416 shutdown_connections 2024-09-21T06:48:48.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.250+0000 7f16fe12e640 1 -- 172.21.15.67:0/4275486416 wait complete. 2024-09-21T06:48:48.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.251+0000 7f16fe12e640 1 Processor -- start 2024-09-21T06:48:48.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.251+0000 7f16fe12e640 1 -- start start 2024-09-21T06:48:48.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.251+0000 7f16fe12e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16f8102a60 0x7f16f819a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:48.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.251+0000 7f16fe12e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16f81083a0 0x7f16f819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:48.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.251+0000 7f16fe12e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16f8108cf0 0x7f16f819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:48.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.251+0000 7f16fe12e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f16f8112860 con 0x7f16f8108cf0 2024-09-21T06:48:48.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.251+0000 7f16fe12e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f16f81126e0 con 0x7f16f8102a60 2024-09-21T06:48:48.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.251+0000 7f16fe12e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f16f81129e0 con 0x7f16f81083a0 2024-09-21T06:48:48.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.252+0000 7f16f7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16f8108cf0 0x7f16f819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:48.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.252+0000 7f16f77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16f8102a60 0x7f16f819a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:48.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.252+0000 7f16f7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16f8108cf0 0x7f16f819fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57850/0 (socket says 172.21.15.67:57850) 2024-09-21T06:48:48.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.252+0000 7f16f7fff640 1 -- 172.21.15.67:0/4036725049 learned_addr learned my addr 172.21.15.67:0/4036725049 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-21T06:48:48.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.252+0000 7f16f77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16f8102a60 0x7f16f819a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.67:52360/0 (socket says 172.21.15.67:52360) 2024-09-21T06:48:48.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.252+0000 7f16f6ffd640 1 --2- 172.21.15.67:0/4036725049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16f81083a0 0x7f16f819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:48.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.252+0000 7f16f77fe640 1 -- 172.21.15.67:0/4036725049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16f81083a0 msgr2=0x7f16f819a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:48.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.252+0000 7f16f77fe640 1 --2- 172.21.15.67:0/4036725049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16f81083a0 0x7f16f819a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:48.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.252+0000 7f16f77fe640 1 -- 172.21.15.67:0/4036725049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16f8108cf0 msgr2=0x7f16f819fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:48.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.252+0000 7f16f77fe640 1 --2- 172.21.15.67:0/4036725049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16f8108cf0 0x7f16f819fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:48.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.252+0000 7f16f77fe640 1 -- 172.21.15.67:0/4036725049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f81a0310 con 0x7f16f8102a60 2024-09-21T06:48:48.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.252+0000 7f16f77fe640 1 --2- 172.21.15.67:0/4036725049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16f8102a60 0x7f16f819a3a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f16e800e9f0 tx=0x7f16e800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:48.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.252+0000 7f16f4ff9640 1 -- 172.21.15.67:0/4036725049 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e800ccc0 con 0x7f16f8102a60 2024-09-21T06:48:48.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.252+0000 7f16fe12e640 1 -- 172.21.15.67:0/4036725049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16f81a0600 con 0x7f16f8102a60 2024-09-21T06:48:48.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.252+0000 7f16f4ff9640 1 -- 172.21.15.67:0/4036725049 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16e8005020 con 0x7f16f8102a60 2024-09-21T06:48:48.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.252+0000 7f16f4ff9640 1 -- 172.21.15.67:0/4036725049 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e800ce60 con 0x7f16f8102a60 2024-09-21T06:48:48.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.252+0000 7f16fe12e640 1 -- 172.21.15.67:0/4036725049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16f81a6d60 con 0x7f16f8102a60 2024-09-21T06:48:48.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.253+0000 7f16fe12e640 1 -- 172.21.15.67:0/4036725049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16f8102e70 con 0x7f16f8102a60 2024-09-21T06:48:48.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.255+0000 7f16f4ff9640 1 -- 172.21.15.67:0/4036725049 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f16e8004a20 con 0x7f16f8102a60 2024-09-21T06:48:48.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.256+0000 7f16f4ff9640 1 --2- 172.21.15.67:0/4036725049 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f16e40b6510 0x7f16e40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:48.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.256+0000 7f16f6ffd640 1 --2- 172.21.15.67:0/4036725049 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f16e40b6510 0x7f16e40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:48.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.256+0000 7f16f4ff9640 1 -- 172.21.15.67:0/4036725049 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8152+0+0 (secure 0 0 0) 0x7f16e80e7380 con 0x7f16f8102a60 2024-09-21T06:48:48.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.256+0000 7f16f6ffd640 1 --2- 172.21.15.67:0/4036725049 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f16e40b6510 0x7f16e40b8a20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f16f8079c50 tx=0x7f16e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:48.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.259+0000 7f16f4ff9640 1 -- 172.21.15.67:0/4036725049 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f16e80adca0 con 0x7f16f8102a60 2024-09-21T06:48:48.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:48.453+0000 7f16fe12e640 1 -- 172.21.15.67:0/4036725049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f16f819b4d0 con 0x7f16f8102a60 2024-09-21T06:48:49.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:48 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:48:49.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:48 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:48:49.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:48 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:48:49.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:48 smithi067 ceph-mon[31232]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:48:49.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:48 smithi067 ceph-mon[31232]: from='client.? 172.21.15.67:0/4036725049' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T06:48:49.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:48 smithi067 ceph-mon[31232]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T06:48:49.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:48 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:48:49.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:48 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:48:49.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:48 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:48:49.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:48 smithi100 ceph-mon[33401]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:48:49.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:48 smithi100 ceph-mon[33401]: from='client.? 172.21.15.67:0/4036725049' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T06:48:49.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:48 smithi100 ceph-mon[33401]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T06:48:49.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:48 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:48:49.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:48 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:48:49.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:48 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:48:49.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:48 smithi203 ceph-mon[33735]: from='mgr.14152 172.21.15.67:0/2012851699' entity='mgr.x' 2024-09-21T06:48:49.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:48 smithi203 ceph-mon[33735]: from='client.? 172.21.15.67:0/4036725049' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T06:48:49.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:48 smithi203 ceph-mon[33735]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T06:48:49.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:49.420+0000 7f16f4ff9640 1 -- 172.21.15.67:0/4036725049 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v89) ==== 178+0+0 (secure 0 0 0) 0x7f16e80b2080 con 0x7f16f8102a60 2024-09-21T06:48:49.422 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data_ec' removed 2024-09-21T06:48:49.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:49.424+0000 7f16fe12e640 1 -- 172.21.15.67:0/4036725049 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f16e40b6510 msgr2=0x7f16e40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:49.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:49.424+0000 7f16fe12e640 1 --2- 172.21.15.67:0/4036725049 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f16e40b6510 0x7f16e40b8a20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f16f8079c50 tx=0x7f16e0005f90 comp rx=0 tx=0).stop 2024-09-21T06:48:49.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:49.424+0000 7f16fe12e640 1 -- 172.21.15.67:0/4036725049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16f8102a60 msgr2=0x7f16f819a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:49.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:49.424+0000 7f16fe12e640 1 --2- 172.21.15.67:0/4036725049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16f8102a60 0x7f16f819a3a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f16e800e9f0 tx=0x7f16e800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:48:49.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:49.424+0000 7f16fe12e640 1 -- 172.21.15.67:0/4036725049 shutdown_connections 2024-09-21T06:48:49.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:49.424+0000 7f16fe12e640 1 --2- 172.21.15.67:0/4036725049 >> [v2:172.21.15.67:6800/1134893458,v1:172.21.15.67:6801/1134893458] conn(0x7f16e40b6510 0x7f16e40b8a20 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:49.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:49.424+0000 7f16fe12e640 1 --2- 172.21.15.67:0/4036725049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16f8108cf0 0x7f16f819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:49.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:49.424+0000 7f16fe12e640 1 --2- 172.21.15.67:0/4036725049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16f81083a0 0x7f16f819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:49.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:49.424+0000 7f16fe12e640 1 --2- 172.21.15.67:0/4036725049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16f8102a60 0x7f16f819a3a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:49.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:49.424+0000 7f16fe12e640 1 -- 172.21.15.67:0/4036725049 >> 172.21.15.67:0/4036725049 conn(0x7f16f8079730 msgr2=0x7f16f806b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:49.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:49.425+0000 7f16fe12e640 1 -- 172.21.15.67:0/4036725049 shutdown_connections 2024-09-21T06:48:49.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-21T06:48:49.425+0000 7f16fe12e640 1 -- 172.21.15.67:0/4036725049 wait complete. 2024-09-21T06:48:50.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:49 smithi067 ceph-mon[31232]: pgmap v730: 33 pgs: 33 active+clean; 577 KiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:48:50.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:49 smithi067 ceph-mon[31232]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T06:48:50.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:49 smithi067 ceph-mon[31232]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T06:48:50.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:49 smithi100 ceph-mon[33401]: pgmap v730: 33 pgs: 33 active+clean; 577 KiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:48:50.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:49 smithi100 ceph-mon[33401]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T06:48:50.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:49 smithi100 ceph-mon[33401]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T06:48:50.105 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:48:50.133 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:49 smithi203 ceph-mon[33735]: pgmap v730: 33 pgs: 33 active+clean; 577 KiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:48:50.133 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:49 smithi203 ceph-mon[33735]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T06:48:50.133 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:49 smithi203 ceph-mon[33735]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T06:48:50.149 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:48:50.177 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:48:50.205 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-21T06:48:50.205 DEBUG:teuthology.orchestra.run.smithi067:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T06:48:50.222 DEBUG:teuthology.orchestra.run.smithi100:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T06:48:50.237 DEBUG:teuthology.orchestra.run.smithi203:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T06:48:50.261 INFO:tasks.cephadm:Stopping all daemons... 2024-09-21T06:48:50.261 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-21T06:48:50.261 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-2a791768-77e2-11ef-baf5-efdc52797490@mon.a 2024-09-21T06:48:50.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:50 smithi067 systemd[1]: Stopping Ceph mon.a for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:48:51.288 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:50 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mon-a[31228]: 2024-09-21T06:48:50.844+0000 7f2f31390640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:48:51.288 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:50 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mon-a[31228]: 2024-09-21T06:48:50.844+0000 7f2f31390640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-21T06:48:51.288 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:50 smithi067 podman[99529]: 2024-09-21 06:48:50.988146349 +0000 UTC m=+0.283265372 container died 853c94eacb0a03af1e73ff829802dde383540b0848e22c2197cbdf6e7c29de18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-21T06:48:51.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:51 smithi067 podman[99529]: 2024-09-21 06:48:51.394668409 +0000 UTC m=+0.689787421 container cleanup 853c94eacb0a03af1e73ff829802dde383540b0848e22c2197cbdf6e7c29de18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-21T06:48:51.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 21 06:48:51 smithi067 bash[99529]: ceph-2a791768-77e2-11ef-baf5-efdc52797490-mon-a 2024-09-21T06:48:51.797 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mon.a.service' 2024-09-21T06:48:51.836 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:48:51.836 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-21T06:48:51.836 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-21T06:48:51.836 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-2a791768-77e2-11ef-baf5-efdc52797490@mon.b 2024-09-21T06:48:52.264 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:51 smithi100 systemd[1]: Stopping Ceph mon.b for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:48:52.553 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:52 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mon-b[33397]: 2024-09-21T06:48:52.261+0000 7f46f8bbd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:48:52.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:52 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mon-b[33397]: 2024-09-21T06:48:52.261+0000 7f46f8bbd640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-21T06:48:52.554 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:52 smithi100 podman[64168]: 2024-09-21 06:48:52.486365288 +0000 UTC m=+0.460153434 container died f935de8304a8af0afce1d81ccae64dc73572b8d68734732dbbba3c6b4132ece2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mon-b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-21T06:48:53.053 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:52 smithi100 podman[64168]: 2024-09-21 06:48:52.986969079 +0000 UTC m=+0.960757257 container cleanup f935de8304a8af0afce1d81ccae64dc73572b8d68734732dbbba3c6b4132ece2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mon-b, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=) 2024-09-21T06:48:53.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Sep 21 06:48:52 smithi100 bash[64168]: ceph-2a791768-77e2-11ef-baf5-efdc52797490-mon-b 2024-09-21T06:48:53.658 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mon.b.service' 2024-09-21T06:48:53.702 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:48:53.702 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-21T06:48:53.702 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-21T06:48:53.702 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-2a791768-77e2-11ef-baf5-efdc52797490@mon.c 2024-09-21T06:48:53.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:53 smithi203 systemd[1]: Stopping Ceph mon.c for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:48:54.268 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:54 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mon-c[33728]: 2024-09-21T06:48:54.001+0000 7f19e0994640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:48:54.268 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:54 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mon-c[33728]: 2024-09-21T06:48:54.001+0000 7f19e0994640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-21T06:48:54.595 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:54 smithi203 podman[64909]: 2024-09-21 06:48:54.26705411 +0000 UTC m=+0.363991800 container died 258ab452f69ce6510b5dcb94712584830e64bebb4ed68c12b243a14d7d297a69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mon-c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-21T06:48:54.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:54 smithi203 podman[64909]: 2024-09-21 06:48:54.785894269 +0000 UTC m=+0.882831951 container cleanup 258ab452f69ce6510b5dcb94712584830e64bebb4ed68c12b243a14d7d297a69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mon-c, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-21T06:48:54.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:54 smithi203 bash[64909]: ceph-2a791768-77e2-11ef-baf5-efdc52797490-mon-c 2024-09-21T06:48:55.265 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 06:48:55 smithi203 podman[64921]: 2024-09-21 06:48:55.012612391 +0000 UTC m=+0.739078979 container remove 258ab452f69ce6510b5dcb94712584830e64bebb4ed68c12b243a14d7d297a69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mon-c, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:48:55.270 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mon.c.service' 2024-09-21T06:48:55.353 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:48:55.353 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-21T06:48:55.353 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-21T06:48:55.353 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-2a791768-77e2-11ef-baf5-efdc52797490@mgr.x 2024-09-21T06:48:55.676 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:48:55 smithi067 systemd[1]: Stopping Ceph mgr.x for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:48:56.019 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:48:55 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: 2024-09-21T06:48:55.674+0000 7fc02b56b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:48:56.019 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:48:55 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x[31459]: 2024-09-21T06:48:55.674+0000 7fc02b56b640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-21T06:48:56.019 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:48:55 smithi067 podman[99647]: 2024-09-21 06:48:55.716216705 +0000 UTC m=+0.162620200 container died 5f64d4ee2103e6d32e6dd163eeae1e1419ecdebddd8f127444218d71ebf57a55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:48:56.293 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:48:56 smithi067 podman[99647]: 2024-09-21 06:48:56.133979448 +0000 UTC m=+0.580382933 container cleanup 5f64d4ee2103e6d32e6dd163eeae1e1419ecdebddd8f127444218d71ebf57a55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-21T06:48:56.293 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 21 06:48:56 smithi067 bash[99647]: ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-x 2024-09-21T06:48:56.580 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mgr.x.service' 2024-09-21T06:48:56.619 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:48:56.619 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-21T06:48:56.619 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-21T06:48:56.619 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-2a791768-77e2-11ef-baf5-efdc52797490@mgr.y 2024-09-21T06:48:56.957 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 21 06:48:56 smithi100 systemd[1]: Stopping Ceph mgr.y for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:48:57.295 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 21 06:48:56 smithi100 podman[64295]: 2024-09-21 06:48:56.955756389 +0000 UTC m=+0.133442877 container died ffd7841c3d54a6e9e30b7b88b1bd859478875ef9bb4aa407e4cdc244e1ad039f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS) 2024-09-21T06:48:57.553 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 21 06:48:57 smithi100 podman[64295]: 2024-09-21 06:48:57.426458571 +0000 UTC m=+0.604145057 container cleanup ffd7841c3d54a6e9e30b7b88b1bd859478875ef9bb4aa407e4cdc244e1ad039f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-y, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-21T06:48:57.554 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 21 06:48:57 smithi100 bash[64295]: ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-y 2024-09-21T06:48:57.858 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mgr.y.service' 2024-09-21T06:48:57.892 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:48:57.892 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-21T06:48:57.892 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-21T06:48:57.893 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-2a791768-77e2-11ef-baf5-efdc52797490@mgr.z 2024-09-21T06:48:58.232 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 21 06:48:57 smithi203 systemd[1]: Stopping Ceph mgr.z for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:48:58.618 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 21 06:48:58 smithi203 podman[65032]: 2024-09-21 06:48:58.330394985 +0000 UTC m=+0.243039634 container died 37bfc5ac89ca220b0a0bea9bca5c76a57ca7ded7a0a53b4c2887c993da5d0253 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-z, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-21T06:48:58.982 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 21 06:48:58 smithi203 podman[65032]: 2024-09-21 06:48:58.708158078 +0000 UTC m=+0.620802719 container cleanup 37bfc5ac89ca220b0a0bea9bca5c76a57ca7ded7a0a53b4c2887c993da5d0253 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-z, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-21T06:48:58.982 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 21 06:48:58 smithi203 bash[65032]: ceph-2a791768-77e2-11ef-baf5-efdc52797490-mgr-z 2024-09-21T06:48:59.133 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mgr.z.service' 2024-09-21T06:48:59.168 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:48:59.168 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-21T06:48:59.168 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-21T06:48:59.168 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.0 2024-09-21T06:48:59.473 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 21 06:48:59 smithi067 systemd[1]: Stopping Ceph osd.0 for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:48:59.792 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 21 06:48:59 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-0[46350]: 2024-09-21T06:48:59.472+0000 7f17d0593640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:48:59.792 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 21 06:48:59 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-0[46350]: 2024-09-21T06:48:59.472+0000 7f17d0593640 -1 osd.0 89 *** Got signal Terminated *** 2024-09-21T06:48:59.792 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 21 06:48:59 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-0[46350]: 2024-09-21T06:48:59.472+0000 7f17d0593640 -1 osd.0 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:49:05.358 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 21 06:49:05 smithi067 podman[99771]: 2024-09-21 06:49:05.055095303 +0000 UTC m=+5.695675892 container died ac4a3c1547e9b712ee9bc3ce226c1c4b764502143f4f8b36cfb42b8a9ddbb41d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T06:49:05.717 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 21 06:49:05 smithi067 podman[99771]: 2024-09-21 06:49:05.456349036 +0000 UTC m=+6.096929622 container cleanup ac4a3c1547e9b712ee9bc3ce226c1c4b764502143f4f8b36cfb42b8a9ddbb41d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-0, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-21T06:49:05.717 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 21 06:49:05 smithi067 bash[99771]: ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-0 2024-09-21T06:49:06.042 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 21 06:49:05 smithi067 podman[99786]: 2024-09-21 06:49:05.716076392 +0000 UTC m=+0.655240876 container remove ac4a3c1547e9b712ee9bc3ce226c1c4b764502143f4f8b36cfb42b8a9ddbb41d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-21T06:49:06.346 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 21 06:49:06 smithi067 podman[99875]: 2024-09-21 06:49:05.983960904 +0000 UTC m=+0.015968984 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:49:06.346 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 21 06:49:06 smithi067 podman[99875]: 2024-09-21 06:49:06.100719606 +0000 UTC m=+0.132727721 container create 56afe0514551da720f2988aba1d2bdccb71c0a9961f2023a2e0281ae3a2e72fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True) 2024-09-21T06:49:06.792 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 21 06:49:06 smithi067 podman[99875]: 2024-09-21 06:49:06.345270506 +0000 UTC m=+0.377278586 container init 56afe0514551da720f2988aba1d2bdccb71c0a9961f2023a2e0281ae3a2e72fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-21T06:49:06.792 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 21 06:49:06 smithi067 podman[99875]: 2024-09-21 06:49:06.34885984 +0000 UTC m=+0.380867913 container start 56afe0514551da720f2988aba1d2bdccb71c0a9961f2023a2e0281ae3a2e72fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-21T06:49:06.792 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 21 06:49:06 smithi067 podman[99875]: 2024-09-21 06:49:06.453581507 +0000 UTC m=+0.485589597 container attach 56afe0514551da720f2988aba1d2bdccb71c0a9961f2023a2e0281ae3a2e72fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-0-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:49:07.292 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 21 06:49:06 smithi067 podman[99875]: 2024-09-21 06:49:06.875346445 +0000 UTC m=+0.907354518 container died 56afe0514551da720f2988aba1d2bdccb71c0a9961f2023a2e0281ae3a2e72fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-0-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:49:07.798 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.0.service' 2024-09-21T06:49:07.846 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:49:07.846 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-21T06:49:07.846 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-21T06:49:07.847 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.1 2024-09-21T06:49:08.249 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 21 06:49:07 smithi100 systemd[1]: Stopping Ceph osd.1 for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:49:08.553 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 21 06:49:08 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-1[37274]: 2024-09-21T06:49:08.247+0000 7f0b14cb7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:49:08.553 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 21 06:49:08 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-1[37274]: 2024-09-21T06:49:08.247+0000 7f0b14cb7640 -1 osd.1 89 *** Got signal Terminated *** 2024-09-21T06:49:08.554 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 21 06:49:08 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-1[37274]: 2024-09-21T06:49:08.247+0000 7f0b14cb7640 -1 osd.1 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:49:14.157 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 21 06:49:13 smithi100 podman[64419]: 2024-09-21 06:49:13.868778088 +0000 UTC m=+5.830965506 container died 4930c2512656ee2b57b72d748fe0974dc03f5b63665f97bddf15641c4ce003f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-1, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:49:14.508 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 21 06:49:14 smithi100 podman[64419]: 2024-09-21 06:49:14.25607929 +0000 UTC m=+6.218266710 container cleanup 4930c2512656ee2b57b72d748fe0974dc03f5b63665f97bddf15641c4ce003f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-1, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-21T06:49:14.508 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 21 06:49:14 smithi100 bash[64419]: ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-1 2024-09-21T06:49:14.803 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 21 06:49:14 smithi100 podman[64435]: 2024-09-21 06:49:14.507321117 +0000 UTC m=+0.632301490 container remove 4930c2512656ee2b57b72d748fe0974dc03f5b63665f97bddf15641c4ce003f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-09-21T06:49:15.070 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 21 06:49:14 smithi100 podman[64526]: 2024-09-21 06:49:14.808278658 +0000 UTC m=+0.091744935 container create 0f1246c39f1f34b788db9d06e2bc633203ff939bf2922cd145138c2aa194d375 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:49:15.070 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 21 06:49:14 smithi100 podman[64526]: 2024-09-21 06:49:14.732588442 +0000 UTC m=+0.016054667 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:49:15.070 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 21 06:49:15 smithi100 podman[64526]: 2024-09-21 06:49:15.069245373 +0000 UTC m=+0.352711601 container init 0f1246c39f1f34b788db9d06e2bc633203ff939bf2922cd145138c2aa194d375 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-21T06:49:15.554 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 21 06:49:15 smithi100 podman[64526]: 2024-09-21 06:49:15.073041745 +0000 UTC m=+0.356507973 container start 0f1246c39f1f34b788db9d06e2bc633203ff939bf2922cd145138c2aa194d375 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-1-deactivate, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:49:15.554 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 21 06:49:15 smithi100 podman[64526]: 2024-09-21 06:49:15.169411689 +0000 UTC m=+0.452877908 container attach 0f1246c39f1f34b788db9d06e2bc633203ff939bf2922cd145138c2aa194d375 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-21T06:49:15.554 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 21 06:49:15 smithi100 podman[64526]: 2024-09-21 06:49:15.262090045 +0000 UTC m=+0.545556266 container died 0f1246c39f1f34b788db9d06e2bc633203ff939bf2922cd145138c2aa194d375 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T06:49:15.950 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.1.service' 2024-09-21T06:49:15.989 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:49:15.989 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-21T06:49:15.989 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-21T06:49:15.990 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.2 2024-09-21T06:49:16.305 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 06:49:16 smithi203 systemd[1]: Stopping Ceph osd.2 for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:49:16.732 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 06:49:16 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-2[38108]: 2024-09-21T06:49:16.303+0000 7f0b74e46640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:49:16.732 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 06:49:16 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-2[38108]: 2024-09-21T06:49:16.303+0000 7f0b74e46640 -1 osd.2 89 *** Got signal Terminated *** 2024-09-21T06:49:16.732 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 06:49:16 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-2[38108]: 2024-09-21T06:49:16.303+0000 7f0b74e46640 -1 osd.2 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:49:22.040 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 06:49:21 smithi203 podman[65156]: 2024-09-21 06:49:21.750105525 +0000 UTC m=+5.562946487 container died 0419b3bdb938472e13bd38c3ae6f844f37ce74df29a88097754b5c7fb52a7f9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-21T06:49:22.307 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 06:49:22 smithi203 podman[65156]: 2024-09-21 06:49:22.12952045 +0000 UTC m=+5.942361413 container cleanup 0419b3bdb938472e13bd38c3ae6f844f37ce74df29a88097754b5c7fb52a7f9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:49:22.307 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 06:49:22 smithi203 bash[65156]: ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-2 2024-09-21T06:49:22.725 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 06:49:22 smithi203 podman[65171]: 2024-09-21 06:49:22.305962425 +0000 UTC m=+0.551747151 container remove 0419b3bdb938472e13bd38c3ae6f844f37ce74df29a88097754b5c7fb52a7f9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-2, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T06:49:22.982 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 06:49:22 smithi203 podman[65257]: 2024-09-21 06:49:22.627164625 +0000 UTC m=+0.015899229 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:49:22.982 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 06:49:22 smithi203 podman[65257]: 2024-09-21 06:49:22.740467584 +0000 UTC m=+0.129202175 container create a25ada2fdd89c5908667a940dd3bf512af8a46c6db84a7e211b15b974a8db087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-2-deactivate, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:49:23.474 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 06:49:22 smithi203 podman[65257]: 2024-09-21 06:49:22.99321069 +0000 UTC m=+0.381945288 container init a25ada2fdd89c5908667a940dd3bf512af8a46c6db84a7e211b15b974a8db087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-21T06:49:23.474 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 06:49:22 smithi203 podman[65257]: 2024-09-21 06:49:22.996656293 +0000 UTC m=+0.385390884 container start a25ada2fdd89c5908667a940dd3bf512af8a46c6db84a7e211b15b974a8db087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-2-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-21T06:49:23.475 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 06:49:23 smithi203 podman[65257]: 2024-09-21 06:49:23.084939873 +0000 UTC m=+0.473674486 container attach a25ada2fdd89c5908667a940dd3bf512af8a46c6db84a7e211b15b974a8db087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2) 2024-09-21T06:49:23.475 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 06:49:23 smithi203 podman[65257]: 2024-09-21 06:49:23.200406635 +0000 UTC m=+0.589141239 container died a25ada2fdd89c5908667a940dd3bf512af8a46c6db84a7e211b15b974a8db087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-2-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-21T06:49:23.892 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.2.service' 2024-09-21T06:49:23.930 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:49:23.931 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-21T06:49:23.931 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-21T06:49:23.931 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.3 2024-09-21T06:49:24.257 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 21 06:49:23 smithi067 systemd[1]: Stopping Ceph osd.3 for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:49:24.542 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 21 06:49:24 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-3[52794]: 2024-09-21T06:49:24.253+0000 7efd7c9b2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:49:24.542 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 21 06:49:24 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-3[52794]: 2024-09-21T06:49:24.253+0000 7efd7c9b2640 -1 osd.3 89 *** Got signal Terminated *** 2024-09-21T06:49:24.542 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 21 06:49:24 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-3[52794]: 2024-09-21T06:49:24.253+0000 7efd7c9b2640 -1 osd.3 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:49:26.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:25 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:25.974+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:26.304 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:25 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:25.874+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:27.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:26 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:26.927+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:27.303 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:26 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:26.864+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:28.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:27 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:27.724+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:28.232 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:27 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:27.988+0000 7f6f257ba640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.839476+0000 front 2024-09-21T06:49:01.839654+0000 (oldest deadline 2024-09-21T06:49:27.139464+0000) 2024-09-21T06:49:28.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:27 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:27.944+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:28.303 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:27 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:27.849+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:28.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:28 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:28.318+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:29.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:28 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:28.716+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:29.042 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 21 06:49:28 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-3[52794]: 2024-09-21T06:49:28.894+0000 7efd74f95640 -1 osd.3 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.298702+0000 front 2024-09-21T06:49:04.298738+0000 (oldest deadline 2024-09-21T06:49:28.398913+0000) 2024-09-21T06:49:29.232 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:29 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:29.003+0000 7f6f257ba640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.839476+0000 front 2024-09-21T06:49:01.839654+0000 (oldest deadline 2024-09-21T06:49:27.139464+0000) 2024-09-21T06:49:29.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:28 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:28.910+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:29.303 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:28 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:28.855+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:29.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:29 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:29.349+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:30.041 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:29 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:29.681+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:30.042 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 21 06:49:29 smithi067 podman[99988]: 2024-09-21 06:49:29.806736104 +0000 UTC m=+5.667995283 container died 5f55c49290dbd3302906a9481e469df22a7ff4b52ab8a97f63def43b161942ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-3, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-21T06:49:30.232 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:30 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:30.003+0000 7f6f257ba640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.839476+0000 front 2024-09-21T06:49:01.839654+0000 (oldest deadline 2024-09-21T06:49:27.139464+0000) 2024-09-21T06:49:30.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:29 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:29.954+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:30.303 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:29 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:29.834+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:30.542 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 21 06:49:30 smithi067 podman[99988]: 2024-09-21 06:49:30.328735664 +0000 UTC m=+6.189994882 container cleanup 5f55c49290dbd3302906a9481e469df22a7ff4b52ab8a97f63def43b161942ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:49:30.542 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 21 06:49:30 smithi067 bash[99988]: ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-3 2024-09-21T06:49:30.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:30 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:30.370+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:30.961 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:30 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:30.709+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:30.961 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 21 06:49:30 smithi067 podman[100006]: 2024-09-21 06:49:30.597062405 +0000 UTC m=+0.770027406 container remove 5f55c49290dbd3302906a9481e469df22a7ff4b52ab8a97f63def43b161942ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:49:31.226 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 21 06:49:30 smithi067 podman[100092]: 2024-09-21 06:49:30.862498866 +0000 UTC m=+0.016032124 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:49:31.227 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 21 06:49:30 smithi067 podman[100092]: 2024-09-21 06:49:30.96446415 +0000 UTC m=+0.117997390 container create 7812753a56b76e2fba257429aa34d318dd85167d5967aaf596d226cda8b80767 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True) 2024-09-21T06:49:31.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:30 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:30.908+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:31.232 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:30 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:30.994+0000 7f6f257ba640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.839476+0000 front 2024-09-21T06:49:01.839654+0000 (oldest deadline 2024-09-21T06:49:27.139464+0000) 2024-09-21T06:49:31.304 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:30 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:30.806+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:31.542 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 21 06:49:31 smithi067 podman[100092]: 2024-09-21 06:49:31.225755163 +0000 UTC m=+0.379288416 container init 7812753a56b76e2fba257429aa34d318dd85167d5967aaf596d226cda8b80767 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-21T06:49:31.542 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 21 06:49:31 smithi067 podman[100092]: 2024-09-21 06:49:31.229401595 +0000 UTC m=+0.382934837 container start 7812753a56b76e2fba257429aa34d318dd85167d5967aaf596d226cda8b80767 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-3-deactivate, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-21T06:49:31.542 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 21 06:49:31 smithi067 podman[100092]: 2024-09-21 06:49:31.342627054 +0000 UTC m=+0.496160296 container attach 7812753a56b76e2fba257429aa34d318dd85167d5967aaf596d226cda8b80767 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-3-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-21T06:49:31.542 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 21 06:49:31 smithi067 podman[100092]: 2024-09-21 06:49:31.425600829 +0000 UTC m=+0.579134091 container died 7812753a56b76e2fba257429aa34d318dd85167d5967aaf596d226cda8b80767 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-3-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=) 2024-09-21T06:49:31.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:31 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:31.366+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:32.039 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:31 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:31.681+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:32.053 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:31 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:31.782+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:32.064 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.3.service' 2024-09-21T06:49:32.106 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:49:32.106 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-21T06:49:32.106 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-21T06:49:32.106 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.4 2024-09-21T06:49:32.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:31 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:31.860+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:32.232 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:32 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:32.021+0000 7f6f257ba640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.839476+0000 front 2024-09-21T06:49:01.839654+0000 (oldest deadline 2024-09-21T06:49:27.139464+0000) 2024-09-21T06:49:32.427 INFO:journalctl@ceph.osd.4.smithi100.stdout:Sep 21 06:49:32 smithi100 systemd[1]: Stopping Ceph osd.4 for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:49:32.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:32 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:32.322+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:32.803 INFO:journalctl@ceph.osd.4.smithi100.stdout:Sep 21 06:49:32 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-4[43550]: 2024-09-21T06:49:32.425+0000 7f2248873640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:49:32.803 INFO:journalctl@ceph.osd.4.smithi100.stdout:Sep 21 06:49:32 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-4[43550]: 2024-09-21T06:49:32.425+0000 7f2248873640 -1 osd.4 89 *** Got signal Terminated *** 2024-09-21T06:49:32.804 INFO:journalctl@ceph.osd.4.smithi100.stdout:Sep 21 06:49:32 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-4[43550]: 2024-09-21T06:49:32.425+0000 7f2248873640 -1 osd.4 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:49:33.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:32 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:32.698+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:33.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:32 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:32.823+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:33.232 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:33 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:33.016+0000 7f6f257ba640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.839476+0000 front 2024-09-21T06:49:01.839654+0000 (oldest deadline 2024-09-21T06:49:27.139464+0000) 2024-09-21T06:49:33.303 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:32 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:32.826+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:33.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:33 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:33.320+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:34.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:33 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:33.743+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:34.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:33 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:33.831+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:34.232 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:33 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:33.998+0000 7f6f257ba640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.839476+0000 front 2024-09-21T06:49:01.839654+0000 (oldest deadline 2024-09-21T06:49:27.139464+0000) 2024-09-21T06:49:34.303 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:33 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:33.865+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:34.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:34 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:34.360+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:35.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:34 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:34.748+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:35.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:34 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:34.748+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:49:35.042 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:34 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:34.550+0000 7f85c1e72640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:08.364522+0000 front 2024-09-21T06:49:08.364491+0000 (oldest deadline 2024-09-21T06:49:34.264689+0000) 2024-09-21T06:49:35.232 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:35 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:35.022+0000 7f6f257ba640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.839476+0000 front 2024-09-21T06:49:01.839654+0000 (oldest deadline 2024-09-21T06:49:27.139464+0000) 2024-09-21T06:49:35.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:34 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:34.865+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:35.303 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:34 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:34.815+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:35.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:35 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:35.371+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:36.042 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:35 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:35.599+0000 7f85c1e72640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:08.364522+0000 front 2024-09-21T06:49:08.364491+0000 (oldest deadline 2024-09-21T06:49:34.264689+0000) 2024-09-21T06:49:36.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:35 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:35.739+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:36.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:35 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:35.739+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:49:36.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:35 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:35.843+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:36.232 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:36 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:36.007+0000 7f6f257ba640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.839476+0000 front 2024-09-21T06:49:01.839654+0000 (oldest deadline 2024-09-21T06:49:27.139464+0000) 2024-09-21T06:49:36.232 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:36 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:36.007+0000 7f6f257ba640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.040985+0000 front 2024-09-21T06:49:10.040876+0000 (oldest deadline 2024-09-21T06:49:35.940819+0000) 2024-09-21T06:49:36.303 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:35 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:35.824+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:36.304 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:35 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:35.824+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:49:36.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:36 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:36.391+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:37.042 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:36 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:36.613+0000 7f85c1e72640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:08.364522+0000 front 2024-09-21T06:49:08.364491+0000 (oldest deadline 2024-09-21T06:49:34.264689+0000) 2024-09-21T06:49:37.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:36 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:36.692+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:37.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:36 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:36.692+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:49:37.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:36 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:36.850+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:37.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:36 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:36.850+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:49:37.232 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:36 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:36.970+0000 7f6f257ba640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.839476+0000 front 2024-09-21T06:49:01.839654+0000 (oldest deadline 2024-09-21T06:49:27.139464+0000) 2024-09-21T06:49:37.233 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:36 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:36.970+0000 7f6f257ba640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.040985+0000 front 2024-09-21T06:49:10.040876+0000 (oldest deadline 2024-09-21T06:49:35.940819+0000) 2024-09-21T06:49:37.303 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:36 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:36.814+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:37.303 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:36 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:36.814+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:49:37.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:37 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:37.415+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:37.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:37 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:37.415+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:49:37.855 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:37 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:37.566+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:11.492448+0000 front 2024-09-21T06:49:11.492364+0000 (oldest deadline 2024-09-21T06:49:37.392788+0000) 2024-09-21T06:49:37.943 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:37 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:37.617+0000 7f85c1e72640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:08.364522+0000 front 2024-09-21T06:49:08.364491+0000 (oldest deadline 2024-09-21T06:49:34.264689+0000) 2024-09-21T06:49:37.943 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:37 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:37.661+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:37.943 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:37 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:37.661+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:49:38.203 INFO:journalctl@ceph.osd.4.smithi100.stdout:Sep 21 06:49:37 smithi100 podman[64639]: 2024-09-21 06:49:37.943709134 +0000 UTC m=+5.641875846 container died f422eeeb192cacfd7cc8db7a4cd6fe1852542d7b2a722a9e7c2a774ae5e2c84d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-4, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-21T06:49:38.203 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:37 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:37.853+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:38.203 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:37 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:37.853+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:49:38.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:37 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:37.826+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:38.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:37 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:37.826+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:49:38.233 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:37 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:37.954+0000 7f6f257ba640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.839476+0000 front 2024-09-21T06:49:01.839654+0000 (oldest deadline 2024-09-21T06:49:27.139464+0000) 2024-09-21T06:49:38.233 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:37 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:37.954+0000 7f6f257ba640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.040985+0000 front 2024-09-21T06:49:10.040876+0000 (oldest deadline 2024-09-21T06:49:35.940819+0000) 2024-09-21T06:49:38.553 INFO:journalctl@ceph.osd.4.smithi100.stdout:Sep 21 06:49:38 smithi100 podman[64639]: 2024-09-21 06:49:38.291137466 +0000 UTC m=+5.989304175 container cleanup f422eeeb192cacfd7cc8db7a4cd6fe1852542d7b2a722a9e7c2a774ae5e2c84d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-4, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:49:38.553 INFO:journalctl@ceph.osd.4.smithi100.stdout:Sep 21 06:49:38 smithi100 bash[64639]: ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-4 2024-09-21T06:49:38.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:38 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:38.411+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:38.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:38 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:38.411+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:49:38.888 INFO:journalctl@ceph.osd.4.smithi100.stdout:Sep 21 06:49:38 smithi100 podman[64654]: 2024-09-21 06:49:38.577585506 +0000 UTC m=+0.635843951 container remove f422eeeb192cacfd7cc8db7a4cd6fe1852542d7b2a722a9e7c2a774ae5e2c84d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True) 2024-09-21T06:49:38.888 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:38 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:38.601+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:11.492448+0000 front 2024-09-21T06:49:11.492364+0000 (oldest deadline 2024-09-21T06:49:37.392788+0000) 2024-09-21T06:49:38.889 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:38 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:38.803+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:38.889 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:38 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:38.803+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:49:38.955 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:38 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:38.591+0000 7f85c1e72640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:08.364522+0000 front 2024-09-21T06:49:08.364491+0000 (oldest deadline 2024-09-21T06:49:34.264689+0000) 2024-09-21T06:49:38.955 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:38 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:38.677+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:38.955 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:38 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:38.677+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:49:39.224 INFO:journalctl@ceph.osd.4.smithi100.stdout:Sep 21 06:49:38 smithi100 podman[64740]: 2024-09-21 06:49:38.789816473 +0000 UTC m=+0.015774585 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:49:39.224 INFO:journalctl@ceph.osd.4.smithi100.stdout:Sep 21 06:49:38 smithi100 podman[64740]: 2024-09-21 06:49:38.911751921 +0000 UTC m=+0.137710014 container create 8d013925002270bbc5b2a5d969433d2abd629dc9d51c78205fe1161152511b8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T06:49:39.224 INFO:journalctl@ceph.osd.4.smithi100.stdout:Sep 21 06:49:39 smithi100 podman[64740]: 2024-09-21 06:49:39.122884765 +0000 UTC m=+0.348842869 container init 8d013925002270bbc5b2a5d969433d2abd629dc9d51c78205fe1161152511b8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-4-deactivate, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD) 2024-09-21T06:49:39.224 INFO:journalctl@ceph.osd.4.smithi100.stdout:Sep 21 06:49:39 smithi100 podman[64740]: 2024-09-21 06:49:39.126165009 +0000 UTC m=+0.352123104 container start 8d013925002270bbc5b2a5d969433d2abd629dc9d51c78205fe1161152511b8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-21T06:49:39.224 INFO:journalctl@ceph.osd.4.smithi100.stdout:Sep 21 06:49:39 smithi100 podman[64740]: 2024-09-21 06:49:39.222873619 +0000 UTC m=+0.448831723 container attach 8d013925002270bbc5b2a5d969433d2abd629dc9d51c78205fe1161152511b8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:49:39.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:38 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:38.791+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:39.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:38 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:38.791+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:49:39.232 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:38 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:38.918+0000 7f6f257ba640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.839476+0000 front 2024-09-21T06:49:01.839654+0000 (oldest deadline 2024-09-21T06:49:27.139464+0000) 2024-09-21T06:49:39.232 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:38 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:38.918+0000 7f6f257ba640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.040985+0000 front 2024-09-21T06:49:10.040876+0000 (oldest deadline 2024-09-21T06:49:35.940819+0000) 2024-09-21T06:49:39.553 INFO:journalctl@ceph.osd.4.smithi100.stdout:Sep 21 06:49:39 smithi100 podman[64740]: 2024-09-21 06:49:39.313798743 +0000 UTC m=+0.539756851 container died 8d013925002270bbc5b2a5d969433d2abd629dc9d51c78205fe1161152511b8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-21T06:49:39.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:39 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:39.380+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:39.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:39 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:39.380+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:49:39.919 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:39 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:39.571+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:11.492448+0000 front 2024-09-21T06:49:11.492364+0000 (oldest deadline 2024-09-21T06:49:37.392788+0000) 2024-09-21T06:49:39.920 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:39 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:39.813+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:39.920 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:39 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:39.813+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:49:39.939 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.4.service' 2024-09-21T06:49:39.979 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:49:39.979 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-21T06:49:39.979 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-21T06:49:39.979 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.5 2024-09-21T06:49:40.006 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:39 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:39.928+0000 7f6f257ba640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.839476+0000 front 2024-09-21T06:49:01.839654+0000 (oldest deadline 2024-09-21T06:49:27.139464+0000) 2024-09-21T06:49:40.006 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:39 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:39.928+0000 7f6f257ba640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.040985+0000 front 2024-09-21T06:49:10.040876+0000 (oldest deadline 2024-09-21T06:49:35.940819+0000) 2024-09-21T06:49:40.006 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:39 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:39.755+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:40.006 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:39 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:39.755+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:49:40.042 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:39 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:39.555+0000 7f85c1e72640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:08.364522+0000 front 2024-09-21T06:49:08.364491+0000 (oldest deadline 2024-09-21T06:49:34.264689+0000) 2024-09-21T06:49:40.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:39 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:39.695+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:40.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:39 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:39.695+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:49:40.294 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:40 smithi203 systemd[1]: Stopping Ceph osd.5 for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:49:40.294 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:40 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:40.292+0000 7f6f291cf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:49:40.294 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:40 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:40.292+0000 7f6f291cf640 -1 osd.5 89 *** Got signal Terminated *** 2024-09-21T06:49:40.294 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:40 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:40.292+0000 7f6f291cf640 -1 osd.5 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:49:40.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:40 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:40.411+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:40.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:40 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:40.411+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:49:41.042 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:40 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:40.598+0000 7f85c1e72640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:08.364522+0000 front 2024-09-21T06:49:08.364491+0000 (oldest deadline 2024-09-21T06:49:34.264689+0000) 2024-09-21T06:49:41.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:40 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:40.712+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:41.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:40 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:40.712+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:49:41.053 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:40 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:40.554+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:11.492448+0000 front 2024-09-21T06:49:11.492364+0000 (oldest deadline 2024-09-21T06:49:37.392788+0000) 2024-09-21T06:49:41.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:40 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:40.782+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:41.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:40 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:40.782+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:49:41.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:40 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:40.761+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:41.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:40 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:40.761+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:49:41.233 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:40 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:40.953+0000 7f6f257ba640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.839476+0000 front 2024-09-21T06:49:01.839654+0000 (oldest deadline 2024-09-21T06:49:27.139464+0000) 2024-09-21T06:49:41.233 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:40 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:40.953+0000 7f6f257ba640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.040985+0000 front 2024-09-21T06:49:10.040876+0000 (oldest deadline 2024-09-21T06:49:35.940819+0000) 2024-09-21T06:49:41.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:41 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:41.459+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:41.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:41 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:41.459+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:49:42.042 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:41 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:41.626+0000 7f85c1e72640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:08.364522+0000 front 2024-09-21T06:49:08.364491+0000 (oldest deadline 2024-09-21T06:49:34.264689+0000) 2024-09-21T06:49:42.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:41 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:41.725+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:42.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:41 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:41.725+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:49:42.053 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:41 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:41.591+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:11.492448+0000 front 2024-09-21T06:49:11.492364+0000 (oldest deadline 2024-09-21T06:49:37.392788+0000) 2024-09-21T06:49:42.053 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:41 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:41.776+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:42.053 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:41 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:41.776+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:49:42.232 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:41 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:41.905+0000 7f6f257ba640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.839476+0000 front 2024-09-21T06:49:01.839654+0000 (oldest deadline 2024-09-21T06:49:27.139464+0000) 2024-09-21T06:49:42.232 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:41 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:41.905+0000 7f6f257ba640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.040985+0000 front 2024-09-21T06:49:10.040876+0000 (oldest deadline 2024-09-21T06:49:35.940819+0000) 2024-09-21T06:49:42.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:41 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:41.806+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:42.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:41 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:41.806+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:49:42.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:42 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:42.433+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:42.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:42 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:42.433+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:49:43.041 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:42 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:42.671+0000 7f85c1e72640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:08.364522+0000 front 2024-09-21T06:49:08.364491+0000 (oldest deadline 2024-09-21T06:49:34.264689+0000) 2024-09-21T06:49:43.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:42 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:42.726+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:43.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:42 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:42.726+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:49:43.053 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:42 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:42.633+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:11.492448+0000 front 2024-09-21T06:49:11.492364+0000 (oldest deadline 2024-09-21T06:49:37.392788+0000) 2024-09-21T06:49:43.053 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:42 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:42.772+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:43.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:42 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:42.772+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:49:43.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:42 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:42.772+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:49:43.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:42 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:42.817+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:43.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:42 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:42.817+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:49:43.233 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:42 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:42.886+0000 7f6f257ba640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.839476+0000 front 2024-09-21T06:49:01.839654+0000 (oldest deadline 2024-09-21T06:49:27.139464+0000) 2024-09-21T06:49:43.233 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:42 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:42.886+0000 7f6f257ba640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.040985+0000 front 2024-09-21T06:49:10.040876+0000 (oldest deadline 2024-09-21T06:49:35.940819+0000) 2024-09-21T06:49:43.233 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:42 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:42.886+0000 7f6f257ba640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:20.542641+0000 front 2024-09-21T06:49:20.542689+0000 (oldest deadline 2024-09-21T06:49:42.842822+0000) 2024-09-21T06:49:43.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:43 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:43.446+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:43.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:43 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:43.446+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:49:43.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:43 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:43.446+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:49:44.042 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:43 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:43.695+0000 7f85c1e72640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:08.364522+0000 front 2024-09-21T06:49:08.364491+0000 (oldest deadline 2024-09-21T06:49:34.264689+0000) 2024-09-21T06:49:44.042 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:43 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:43.695+0000 7f85c1e72640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.167065+0000 front 2024-09-21T06:49:21.167209+0000 (oldest deadline 2024-09-21T06:49:43.367775+0000) 2024-09-21T06:49:44.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:43 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:43.735+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:44.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:43 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:43.735+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:49:44.053 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:43 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:43.590+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:11.492448+0000 front 2024-09-21T06:49:11.492364+0000 (oldest deadline 2024-09-21T06:49:37.392788+0000) 2024-09-21T06:49:44.054 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:43 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:43.590+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:17.393208+0000 front 2024-09-21T06:49:17.393358+0000 (oldest deadline 2024-09-21T06:49:43.293249+0000) 2024-09-21T06:49:44.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:43 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:43.751+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:44.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:43 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:43.751+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:49:44.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:43 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:43.751+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:49:44.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:43 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:43.771+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:44.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:43 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:43.771+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:49:44.232 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:43 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:43.894+0000 7f6f257ba640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.839476+0000 front 2024-09-21T06:49:01.839654+0000 (oldest deadline 2024-09-21T06:49:27.139464+0000) 2024-09-21T06:49:44.233 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:43 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:43.894+0000 7f6f257ba640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.040985+0000 front 2024-09-21T06:49:10.040876+0000 (oldest deadline 2024-09-21T06:49:35.940819+0000) 2024-09-21T06:49:44.233 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:43 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:43.894+0000 7f6f257ba640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:20.542641+0000 front 2024-09-21T06:49:20.542689+0000 (oldest deadline 2024-09-21T06:49:42.842822+0000) 2024-09-21T06:49:44.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:44 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:44.468+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:44.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:44 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:44.468+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:49:44.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:44 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:44.468+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:49:45.042 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:44 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:44.697+0000 7f85c1e72640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:08.364522+0000 front 2024-09-21T06:49:08.364491+0000 (oldest deadline 2024-09-21T06:49:34.264689+0000) 2024-09-21T06:49:45.042 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:44 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:44.697+0000 7f85c1e72640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.167065+0000 front 2024-09-21T06:49:21.167209+0000 (oldest deadline 2024-09-21T06:49:43.367775+0000) 2024-09-21T06:49:45.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:44 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:44.759+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:45.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:44 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:44.759+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:49:45.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:44 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:44.759+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:49:45.053 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:44 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:44.579+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:11.492448+0000 front 2024-09-21T06:49:11.492364+0000 (oldest deadline 2024-09-21T06:49:37.392788+0000) 2024-09-21T06:49:45.054 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:44 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:44.579+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:17.393208+0000 front 2024-09-21T06:49:17.393358+0000 (oldest deadline 2024-09-21T06:49:43.293249+0000) 2024-09-21T06:49:45.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:44 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:44.744+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:45.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:44 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:44.744+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:49:45.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:44 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:44.744+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:49:45.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:44 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:44.805+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:45.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:44 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:44.805+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:49:45.232 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:44 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:44.930+0000 7f6f257ba640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.839476+0000 front 2024-09-21T06:49:01.839654+0000 (oldest deadline 2024-09-21T06:49:27.139464+0000) 2024-09-21T06:49:45.233 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:44 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:44.930+0000 7f6f257ba640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.040985+0000 front 2024-09-21T06:49:10.040876+0000 (oldest deadline 2024-09-21T06:49:35.940819+0000) 2024-09-21T06:49:45.233 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:44 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5[44213]: 2024-09-21T06:49:44.930+0000 7f6f257ba640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:20.542641+0000 front 2024-09-21T06:49:20.542689+0000 (oldest deadline 2024-09-21T06:49:42.842822+0000) 2024-09-21T06:49:45.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:45 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:45.433+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:45.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:45 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:45.433+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:49:45.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:45 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:45.433+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:49:46.042 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:45 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:45.688+0000 7f85c1e72640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:08.364522+0000 front 2024-09-21T06:49:08.364491+0000 (oldest deadline 2024-09-21T06:49:34.264689+0000) 2024-09-21T06:49:46.042 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:45 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:45.688+0000 7f85c1e72640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.167065+0000 front 2024-09-21T06:49:21.167209+0000 (oldest deadline 2024-09-21T06:49:43.367775+0000) 2024-09-21T06:49:46.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:45 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:45.729+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:46.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:45 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:45.729+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:49:46.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:45 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:45.729+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:49:46.053 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:45 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:45.556+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:11.492448+0000 front 2024-09-21T06:49:11.492364+0000 (oldest deadline 2024-09-21T06:49:37.392788+0000) 2024-09-21T06:49:46.053 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:45 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:45.556+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:17.393208+0000 front 2024-09-21T06:49:17.393358+0000 (oldest deadline 2024-09-21T06:49:43.293249+0000) 2024-09-21T06:49:46.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:45 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:45.716+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:46.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:45 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:45.716+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:49:46.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:45 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:45.716+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:49:46.086 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:45 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:45.795+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:46.086 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:45 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:45.795+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:49:46.086 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:45 smithi203 podman[65369]: 2024-09-21 06:49:45.743724205 +0000 UTC m=+5.562363647 container died efc20c9e6e70730e8600a87f5bfb90b0d02ae6b983814c27263b2e7995b5414e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:49:46.387 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:46 smithi203 podman[65369]: 2024-09-21 06:49:46.184245404 +0000 UTC m=+6.002884845 container cleanup efc20c9e6e70730e8600a87f5bfb90b0d02ae6b983814c27263b2e7995b5414e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0) 2024-09-21T06:49:46.387 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:46 smithi203 bash[65369]: ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5 2024-09-21T06:49:46.732 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:46 smithi203 podman[65386]: 2024-09-21 06:49:46.385612411 +0000 UTC m=+0.643798250 container remove efc20c9e6e70730e8600a87f5bfb90b0d02ae6b983814c27263b2e7995b5414e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-21T06:49:46.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:46 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:46.402+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:46.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:46 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:46.402+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:49:46.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:46 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:46.402+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:49:46.803 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:46 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:46.541+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:11.492448+0000 front 2024-09-21T06:49:11.492364+0000 (oldest deadline 2024-09-21T06:49:37.392788+0000) 2024-09-21T06:49:46.804 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:46 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:46.541+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:17.393208+0000 front 2024-09-21T06:49:17.393358+0000 (oldest deadline 2024-09-21T06:49:43.293249+0000) 2024-09-21T06:49:46.804 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:46 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:46.698+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:46.804 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:46 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:46.698+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:49:46.804 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:46 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:46.698+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:49:47.042 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:46 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:46.704+0000 7f85c1e72640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:08.364522+0000 front 2024-09-21T06:49:08.364491+0000 (oldest deadline 2024-09-21T06:49:34.264689+0000) 2024-09-21T06:49:47.042 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:46 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:46.704+0000 7f85c1e72640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.167065+0000 front 2024-09-21T06:49:21.167209+0000 (oldest deadline 2024-09-21T06:49:43.367775+0000) 2024-09-21T06:49:47.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:46 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:46.698+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:47.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:46 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:46.698+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:49:47.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:46 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:46.698+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:49:47.073 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:46 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:46.789+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:47.073 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:46 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:46.789+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:49:47.074 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:46 smithi203 podman[65478]: 2024-09-21 06:49:46.699793506 +0000 UTC m=+0.015842344 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:49:47.074 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:46 smithi203 podman[65478]: 2024-09-21 06:49:46.82842979 +0000 UTC m=+0.144478667 container create 3df695e9bde708b3c1929bf26e4008bb3d12be747098b618986ce22101b4947f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-21T06:49:47.074 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:47 smithi203 podman[65478]: 2024-09-21 06:49:47.072616559 +0000 UTC m=+0.388665404 container init 3df695e9bde708b3c1929bf26e4008bb3d12be747098b618986ce22101b4947f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:49:47.373 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:47 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:47.370+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:47.373 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:47 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:47.370+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:49:47.373 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:47 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:47.370+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:49:47.373 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:47 smithi203 podman[65478]: 2024-09-21 06:49:47.076041451 +0000 UTC m=+0.392090289 container start 3df695e9bde708b3c1929bf26e4008bb3d12be747098b618986ce22101b4947f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-21T06:49:47.373 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:47 smithi203 podman[65478]: 2024-09-21 06:49:47.172592683 +0000 UTC m=+0.488641519 container attach 3df695e9bde708b3c1929bf26e4008bb3d12be747098b618986ce22101b4947f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:49:47.373 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 06:49:47 smithi203 podman[65478]: 2024-09-21 06:49:47.27208923 +0000 UTC m=+0.588138068 container died 3df695e9bde708b3c1929bf26e4008bb3d12be747098b618986ce22101b4947f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-21T06:49:47.803 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:47 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:47.523+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:11.492448+0000 front 2024-09-21T06:49:11.492364+0000 (oldest deadline 2024-09-21T06:49:37.392788+0000) 2024-09-21T06:49:47.804 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:47 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:47.523+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:17.393208+0000 front 2024-09-21T06:49:17.393358+0000 (oldest deadline 2024-09-21T06:49:43.293249+0000) 2024-09-21T06:49:47.804 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:47 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:47.702+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:47.804 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:47 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:47.702+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:49:47.804 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:47 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:47.702+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:49:47.935 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:47 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:47.816+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:47.936 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:47 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:47.816+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:49:47.956 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.5.service' 2024-09-21T06:49:48.036 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:49:48.036 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-21T06:49:48.036 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-21T06:49:48.037 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.6 2024-09-21T06:49:48.041 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:47 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:47.663+0000 7f85c1e72640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:08.364522+0000 front 2024-09-21T06:49:08.364491+0000 (oldest deadline 2024-09-21T06:49:34.264689+0000) 2024-09-21T06:49:48.042 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:47 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:47.663+0000 7f85c1e72640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.167065+0000 front 2024-09-21T06:49:21.167209+0000 (oldest deadline 2024-09-21T06:49:43.367775+0000) 2024-09-21T06:49:48.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:47 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:47.661+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:48.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:47 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:47.661+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:49:48.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:47 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:47.661+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:49:48.476 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:48 smithi067 systemd[1]: Stopping Ceph osd.6 for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:49:48.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:48 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:48.405+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:48.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:48 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:48.405+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:49:48.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:48 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:48.405+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:49:48.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:48 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:48.706+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:48.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:48 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:48.706+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:49:48.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:48 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:48.706+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:49:48.792 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:48 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:48.474+0000 7f85c5887640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:49:48.792 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:48 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:48.474+0000 7f85c5887640 -1 osd.6 89 *** Got signal Terminated *** 2024-09-21T06:49:48.793 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:48 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:48.474+0000 7f85c5887640 -1 osd.6 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:49:48.793 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:48 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:48.656+0000 7f85c1e72640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:08.364522+0000 front 2024-09-21T06:49:08.364491+0000 (oldest deadline 2024-09-21T06:49:34.264689+0000) 2024-09-21T06:49:48.793 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:48 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:48.656+0000 7f85c1e72640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.167065+0000 front 2024-09-21T06:49:21.167209+0000 (oldest deadline 2024-09-21T06:49:43.367775+0000) 2024-09-21T06:49:48.804 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:48 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:48.491+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:11.492448+0000 front 2024-09-21T06:49:11.492364+0000 (oldest deadline 2024-09-21T06:49:37.392788+0000) 2024-09-21T06:49:48.804 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:48 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:48.491+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:17.393208+0000 front 2024-09-21T06:49:17.393358+0000 (oldest deadline 2024-09-21T06:49:43.293249+0000) 2024-09-21T06:49:48.804 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:48 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:48.674+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:48.804 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:48 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:48.674+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:49:48.804 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:48 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:48.674+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:49:49.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:48 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:48.769+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:49.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:48 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:48.769+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:49:49.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:49 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:49.448+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:49.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:49 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:49.448+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:49:49.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:49 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:49.448+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:49:49.803 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:49 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:49.532+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:11.492448+0000 front 2024-09-21T06:49:11.492364+0000 (oldest deadline 2024-09-21T06:49:37.392788+0000) 2024-09-21T06:49:49.804 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:49 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:49.532+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:17.393208+0000 front 2024-09-21T06:49:17.393358+0000 (oldest deadline 2024-09-21T06:49:43.293249+0000) 2024-09-21T06:49:49.804 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:49 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:49.668+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:49.804 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:49 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:49.668+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:49:49.804 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:49 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:49.668+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:49:50.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:49 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:49.690+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:50.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:49 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:49.690+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:49:50.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:49 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:49.690+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:49:50.042 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:49 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:49.676+0000 7f85c1e72640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:08.364522+0000 front 2024-09-21T06:49:08.364491+0000 (oldest deadline 2024-09-21T06:49:34.264689+0000) 2024-09-21T06:49:50.043 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:49 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:49.676+0000 7f85c1e72640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.167065+0000 front 2024-09-21T06:49:21.167209+0000 (oldest deadline 2024-09-21T06:49:43.367775+0000) 2024-09-21T06:49:50.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:49 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:49.742+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:50.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:49 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:49.742+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:49:50.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:50 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:50.415+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:50.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:50 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:50.415+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:49:50.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:50 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:50.415+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:49:50.803 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:50 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:50.516+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:11.492448+0000 front 2024-09-21T06:49:11.492364+0000 (oldest deadline 2024-09-21T06:49:37.392788+0000) 2024-09-21T06:49:50.804 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:50 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:50.516+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:17.393208+0000 front 2024-09-21T06:49:17.393358+0000 (oldest deadline 2024-09-21T06:49:43.293249+0000) 2024-09-21T06:49:50.804 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:50 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:50.655+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:50.804 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:50 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:50.655+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:49:50.804 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:50 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:50.655+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:49:51.042 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:50 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:50.702+0000 7f85c1e72640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:08.364522+0000 front 2024-09-21T06:49:08.364491+0000 (oldest deadline 2024-09-21T06:49:34.264689+0000) 2024-09-21T06:49:51.042 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:50 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:50.702+0000 7f85c1e72640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.167065+0000 front 2024-09-21T06:49:21.167209+0000 (oldest deadline 2024-09-21T06:49:43.367775+0000) 2024-09-21T06:49:51.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:50 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:50.675+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:51.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:50 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:50.675+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:49:51.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:50 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:50.675+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:49:51.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:50 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:50.730+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:51.232 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:50 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:50.730+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:49:51.701 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:51 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:51.377+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:51.701 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:51 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:51.377+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:49:51.701 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:51 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:51.377+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:49:51.803 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:51 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:51.472+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:11.492448+0000 front 2024-09-21T06:49:11.492364+0000 (oldest deadline 2024-09-21T06:49:37.392788+0000) 2024-09-21T06:49:51.803 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:51 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:51.472+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:17.393208+0000 front 2024-09-21T06:49:17.393358+0000 (oldest deadline 2024-09-21T06:49:43.293249+0000) 2024-09-21T06:49:51.804 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:51 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:51.472+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:25.494635+0000 front 2024-09-21T06:49:25.494522+0000 (oldest deadline 2024-09-21T06:49:50.794558+0000) 2024-09-21T06:49:51.804 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:51 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:51.626+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:51.804 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:51 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:51.626+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:49:51.804 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:51 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:51.626+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:49:51.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:51 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:51.699+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:51.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:51 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:51.699+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:49:51.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:51 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:51.699+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.489943+0000 front 2024-09-21T06:49:27.490040+0000 (oldest deadline 2024-09-21T06:49:50.989997+0000) 2024-09-21T06:49:52.042 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:51 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:51.741+0000 7f85c1e72640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:08.364522+0000 front 2024-09-21T06:49:08.364491+0000 (oldest deadline 2024-09-21T06:49:34.264689+0000) 2024-09-21T06:49:52.042 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:51 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:51.741+0000 7f85c1e72640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.167065+0000 front 2024-09-21T06:49:21.167209+0000 (oldest deadline 2024-09-21T06:49:43.367775+0000) 2024-09-21T06:49:52.042 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:51 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:51.741+0000 7f85c1e72640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.468740+0000 front 2024-09-21T06:49:27.468718+0000 (oldest deadline 2024-09-21T06:49:51.568679+0000) 2024-09-21T06:49:52.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:51 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:51.636+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:52.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:51 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:51.636+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:49:52.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:51 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:51.636+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:49:52.686 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:52 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:52.412+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:52.686 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:52 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:52.412+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:49:52.686 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:52 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:52.412+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:49:52.804 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:52 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:52.477+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:11.492448+0000 front 2024-09-21T06:49:11.492364+0000 (oldest deadline 2024-09-21T06:49:37.392788+0000) 2024-09-21T06:49:52.804 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:52 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:52.477+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:17.393208+0000 front 2024-09-21T06:49:17.393358+0000 (oldest deadline 2024-09-21T06:49:43.293249+0000) 2024-09-21T06:49:52.804 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:52 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:52.477+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:25.494635+0000 front 2024-09-21T06:49:25.494522+0000 (oldest deadline 2024-09-21T06:49:50.794558+0000) 2024-09-21T06:49:52.804 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:52 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:52.662+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:52.804 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:52 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:52.662+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:49:52.804 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:52 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:52.662+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:49:52.804 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:52 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:52.662+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:49:52.963 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:52 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:52.684+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:52.963 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:52 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:52.684+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:49:52.963 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:52 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:52.684+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.489943+0000 front 2024-09-21T06:49:27.490040+0000 (oldest deadline 2024-09-21T06:49:50.989997+0000) 2024-09-21T06:49:53.042 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:52 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:52.716+0000 7f85c1e72640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:08.364522+0000 front 2024-09-21T06:49:08.364491+0000 (oldest deadline 2024-09-21T06:49:34.264689+0000) 2024-09-21T06:49:53.042 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:52 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:52.716+0000 7f85c1e72640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.167065+0000 front 2024-09-21T06:49:21.167209+0000 (oldest deadline 2024-09-21T06:49:43.367775+0000) 2024-09-21T06:49:53.042 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:52 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6[59393]: 2024-09-21T06:49:52.716+0000 7f85c1e72640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.468740+0000 front 2024-09-21T06:49:27.468718+0000 (oldest deadline 2024-09-21T06:49:51.568679+0000) 2024-09-21T06:49:53.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:52 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:52.677+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:53.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:52 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:52.677+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:49:53.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:52 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:52.677+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:49:53.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:52 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:52.677+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.478941+0000 front 2024-09-21T06:49:27.478897+0000 (oldest deadline 2024-09-21T06:49:52.179050+0000) 2024-09-21T06:49:53.732 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:53 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:53.686+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:53.732 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:53 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:53.686+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:49:53.732 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:53 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:53.686+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.489943+0000 front 2024-09-21T06:49:27.490040+0000 (oldest deadline 2024-09-21T06:49:50.989997+0000) 2024-09-21T06:49:53.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:53 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:53.452+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:53.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:53 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:53.452+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:49:53.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:53 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:53.452+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:49:53.803 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:53 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:53.481+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:11.492448+0000 front 2024-09-21T06:49:11.492364+0000 (oldest deadline 2024-09-21T06:49:37.392788+0000) 2024-09-21T06:49:53.804 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:53 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:53.481+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:17.393208+0000 front 2024-09-21T06:49:17.393358+0000 (oldest deadline 2024-09-21T06:49:43.293249+0000) 2024-09-21T06:49:53.804 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:53 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:53.481+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:25.494635+0000 front 2024-09-21T06:49:25.494522+0000 (oldest deadline 2024-09-21T06:49:50.794558+0000) 2024-09-21T06:49:53.804 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:53 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:53.684+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:53.804 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:53 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:53.684+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:49:53.804 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:53 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:53.684+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:49:53.804 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:53 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:53.684+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:49:54.042 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:53 smithi067 podman[100206]: 2024-09-21 06:49:53.902539797 +0000 UTC m=+5.669025987 container died 328db6064d002440e83a1e46b70f3b964cac9ddb4ef80c1309acdb524a6bae1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-09-21T06:49:54.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:53 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:53.720+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:54.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:53 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:53.720+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:49:54.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:53 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:53.720+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:49:54.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:53 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:53.720+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.478941+0000 front 2024-09-21T06:49:27.478897+0000 (oldest deadline 2024-09-21T06:49:52.179050+0000) 2024-09-21T06:49:54.732 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:54 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:54.654+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:54.732 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:54 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:54.654+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:49:54.732 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:54 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:54.654+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.489943+0000 front 2024-09-21T06:49:27.490040+0000 (oldest deadline 2024-09-21T06:49:50.989997+0000) 2024-09-21T06:49:54.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:54 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:54.420+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:54.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:54 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:54.420+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:49:54.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:54 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:54.420+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:49:54.761 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:54 smithi067 podman[100206]: 2024-09-21 06:49:54.508973524 +0000 UTC m=+6.275459705 container cleanup 328db6064d002440e83a1e46b70f3b964cac9ddb4ef80c1309acdb524a6bae1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:49:54.761 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:54 smithi067 bash[100206]: ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6 2024-09-21T06:49:54.803 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:54 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:54.519+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:11.492448+0000 front 2024-09-21T06:49:11.492364+0000 (oldest deadline 2024-09-21T06:49:37.392788+0000) 2024-09-21T06:49:54.804 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:54 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:54.519+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:17.393208+0000 front 2024-09-21T06:49:17.393358+0000 (oldest deadline 2024-09-21T06:49:43.293249+0000) 2024-09-21T06:49:54.804 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:54 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:54.519+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:25.494635+0000 front 2024-09-21T06:49:25.494522+0000 (oldest deadline 2024-09-21T06:49:50.794558+0000) 2024-09-21T06:49:54.804 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:54 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:54.721+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:54.804 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:54 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:54.721+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:49:54.804 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:54 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:54.721+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:49:54.804 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:54 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:54.721+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:49:55.042 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:54 smithi067 podman[100222]: 2024-09-21 06:49:54.768803669 +0000 UTC m=+0.862550953 container remove 328db6064d002440e83a1e46b70f3b964cac9ddb4ef80c1309acdb524a6bae1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T06:49:55.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:54 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:54.759+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:55.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:54 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:54.759+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:49:55.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:54 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:54.759+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:49:55.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:54 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:54.759+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.478941+0000 front 2024-09-21T06:49:27.478897+0000 (oldest deadline 2024-09-21T06:49:52.179050+0000) 2024-09-21T06:49:55.406 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:55 smithi067 podman[100308]: 2024-09-21 06:49:54.988443785 +0000 UTC m=+0.016309415 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:49:55.406 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:55 smithi067 podman[100308]: 2024-09-21 06:49:55.086228129 +0000 UTC m=+0.114093744 container create 2b4eeb6a797d1d6ce56ebd4cd80187804adc4bd62ac6e145dfe689fdca68147d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-21T06:49:55.406 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:55 smithi067 podman[100308]: 2024-09-21 06:49:55.313804146 +0000 UTC m=+0.341669762 container init 2b4eeb6a797d1d6ce56ebd4cd80187804adc4bd62ac6e145dfe689fdca68147d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, io.buildah.version=1.37.2) 2024-09-21T06:49:55.406 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:55 smithi067 podman[100308]: 2024-09-21 06:49:55.317753552 +0000 UTC m=+0.345619174 container start 2b4eeb6a797d1d6ce56ebd4cd80187804adc4bd62ac6e145dfe689fdca68147d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:49:55.406 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:55 smithi067 podman[100308]: 2024-09-21 06:49:55.40527645 +0000 UTC m=+0.433142074 container attach 2b4eeb6a797d1d6ce56ebd4cd80187804adc4bd62ac6e145dfe689fdca68147d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-21T06:49:55.655 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:55 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:55.393+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:55.655 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:55 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:55.393+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:49:55.655 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:55 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:55.393+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:49:55.775 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 21 06:49:55 smithi067 podman[100308]: 2024-09-21 06:49:55.513759492 +0000 UTC m=+0.541625115 container died 2b4eeb6a797d1d6ce56ebd4cd80187804adc4bd62ac6e145dfe689fdca68147d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-21T06:49:55.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:55 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:55.653+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:55.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:55 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:55.653+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:49:55.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:55 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:55.653+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.489943+0000 front 2024-09-21T06:49:27.490040+0000 (oldest deadline 2024-09-21T06:49:50.989997+0000) 2024-09-21T06:49:56.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:55 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:55.773+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:56.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:55 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:55.773+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:49:56.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:55 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:55.773+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:49:56.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:55 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:55.773+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.478941+0000 front 2024-09-21T06:49:27.478897+0000 (oldest deadline 2024-09-21T06:49:52.179050+0000) 2024-09-21T06:49:56.053 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:55 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:55.555+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:11.492448+0000 front 2024-09-21T06:49:11.492364+0000 (oldest deadline 2024-09-21T06:49:37.392788+0000) 2024-09-21T06:49:56.054 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:55 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:55.555+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:17.393208+0000 front 2024-09-21T06:49:17.393358+0000 (oldest deadline 2024-09-21T06:49:43.293249+0000) 2024-09-21T06:49:56.054 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:55 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:55.555+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:25.494635+0000 front 2024-09-21T06:49:25.494522+0000 (oldest deadline 2024-09-21T06:49:50.794558+0000) 2024-09-21T06:49:56.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:55 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:55.713+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:56.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:55 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:55.713+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:49:56.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:55 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:55.713+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:49:56.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:55 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:55.713+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:49:56.290 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.6.service' 2024-09-21T06:49:56.375 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:49:56.376 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-21T06:49:56.376 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-21T06:49:56.376 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.7 2024-09-21T06:49:56.639 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:56 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:56.371+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:56.639 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:56 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:56.371+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:49:56.639 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:56 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:56.371+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:49:56.722 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:56 smithi100 systemd[1]: Stopping Ceph osd.7 for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:49:56.722 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:56 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:56.551+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:11.492448+0000 front 2024-09-21T06:49:11.492364+0000 (oldest deadline 2024-09-21T06:49:37.392788+0000) 2024-09-21T06:49:56.722 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:56 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:56.551+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:17.393208+0000 front 2024-09-21T06:49:17.393358+0000 (oldest deadline 2024-09-21T06:49:43.293249+0000) 2024-09-21T06:49:56.722 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:56 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:56.551+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:25.494635+0000 front 2024-09-21T06:49:25.494522+0000 (oldest deadline 2024-09-21T06:49:50.794558+0000) 2024-09-21T06:49:56.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:56 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:56.637+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:56.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:56 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:56.637+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:49:56.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:56 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:56.637+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.489943+0000 front 2024-09-21T06:49:27.490040+0000 (oldest deadline 2024-09-21T06:49:50.989997+0000) 2024-09-21T06:49:57.053 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:56 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:56.720+0000 7fb24ec2b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:49:57.053 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:56 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:56.720+0000 7fb24ec2b640 -1 osd.7 89 *** Got signal Terminated *** 2024-09-21T06:49:57.054 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:56 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:56.720+0000 7fb24ec2b640 -1 osd.7 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:49:57.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:56 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:56.750+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:57.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:56 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:56.750+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:49:57.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:56 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:56.750+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:49:57.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:56 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:56.750+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:49:57.292 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:56 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:56.798+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:57.292 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:56 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:56.798+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:49:57.292 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:56 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:56.798+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:49:57.292 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:56 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:56.798+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.478941+0000 front 2024-09-21T06:49:27.478897+0000 (oldest deadline 2024-09-21T06:49:52.179050+0000) 2024-09-21T06:49:57.677 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:57 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:57.411+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:57.677 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:57 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:57.411+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:49:57.678 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:57 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:57.411+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:49:57.776 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:57 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:57.514+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:11.492448+0000 front 2024-09-21T06:49:11.492364+0000 (oldest deadline 2024-09-21T06:49:37.392788+0000) 2024-09-21T06:49:57.776 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:57 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:57.514+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:17.393208+0000 front 2024-09-21T06:49:17.393358+0000 (oldest deadline 2024-09-21T06:49:43.293249+0000) 2024-09-21T06:49:57.776 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:57 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:57.514+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:25.494635+0000 front 2024-09-21T06:49:25.494522+0000 (oldest deadline 2024-09-21T06:49:50.794558+0000) 2024-09-21T06:49:57.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:57 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:57.675+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:57.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:57 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:57.675+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:49:57.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:57 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:57.675+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.489943+0000 front 2024-09-21T06:49:27.490040+0000 (oldest deadline 2024-09-21T06:49:50.989997+0000) 2024-09-21T06:49:58.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:57 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:57.788+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:58.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:57 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:57.788+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:49:58.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:57 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:57.788+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:49:58.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:57 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:57.788+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.478941+0000 front 2024-09-21T06:49:27.478897+0000 (oldest deadline 2024-09-21T06:49:52.179050+0000) 2024-09-21T06:49:58.053 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:57 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:57.775+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:58.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:57 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:57.775+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:49:58.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:57 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:57.775+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:49:58.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:57 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:57.775+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:49:58.666 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:58 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:58.397+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:58.666 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:58 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:58.397+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:49:58.666 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:58 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:58.397+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:49:58.735 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:58 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:58.488+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:11.492448+0000 front 2024-09-21T06:49:11.492364+0000 (oldest deadline 2024-09-21T06:49:37.392788+0000) 2024-09-21T06:49:58.735 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:58 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:58.488+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:17.393208+0000 front 2024-09-21T06:49:17.393358+0000 (oldest deadline 2024-09-21T06:49:43.293249+0000) 2024-09-21T06:49:58.736 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:58 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:58.488+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:25.494635+0000 front 2024-09-21T06:49:25.494522+0000 (oldest deadline 2024-09-21T06:49:50.794558+0000) 2024-09-21T06:49:58.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:58 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:58.663+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:58.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:58 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:58.663+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:49:58.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:58 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:58.663+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.489943+0000 front 2024-09-21T06:49:27.490040+0000 (oldest deadline 2024-09-21T06:49:50.989997+0000) 2024-09-21T06:49:58.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:58 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:58.663+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.090903+0000 front 2024-09-21T06:49:35.091108+0000 (oldest deadline 2024-09-21T06:49:57.990863+0000) 2024-09-21T06:49:58.992 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:58 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:58.733+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:49:58.992 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:58 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:58.733+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:49:58.992 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:58 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:58.733+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:49:58.992 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:58 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:58.733+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:49:59.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:58 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:58.739+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:49:59.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:58 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:58.739+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:49:59.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:58 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:58.739+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:49:59.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:58 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:58.739+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.478941+0000 front 2024-09-21T06:49:27.478897+0000 (oldest deadline 2024-09-21T06:49:52.179050+0000) 2024-09-21T06:49:59.711 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:59 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:59.439+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:49:59.711 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:59 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:59.439+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:49:59.711 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:59 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:59.439+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:49:59.711 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:49:59 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:49:59.439+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:49:59.747 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:59 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:59.445+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:11.492448+0000 front 2024-09-21T06:49:11.492364+0000 (oldest deadline 2024-09-21T06:49:37.392788+0000) 2024-09-21T06:49:59.747 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:59 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:59.445+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:17.393208+0000 front 2024-09-21T06:49:17.393358+0000 (oldest deadline 2024-09-21T06:49:43.293249+0000) 2024-09-21T06:49:59.747 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:59 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:59.445+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:25.494635+0000 front 2024-09-21T06:49:25.494522+0000 (oldest deadline 2024-09-21T06:49:50.794558+0000) 2024-09-21T06:49:59.747 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:49:59 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:49:59.445+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:34.895495+0000 front 2024-09-21T06:49:34.895522+0000 (oldest deadline 2024-09-21T06:49:58.995619+0000) 2024-09-21T06:49:59.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:59 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:59.709+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:49:59.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:59 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:59.709+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:49:59.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:59 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:59.709+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.489943+0000 front 2024-09-21T06:49:27.490040+0000 (oldest deadline 2024-09-21T06:49:50.989997+0000) 2024-09-21T06:49:59.983 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:49:59 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:49:59.709+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.090903+0000 front 2024-09-21T06:49:35.091108+0000 (oldest deadline 2024-09-21T06:49:57.990863+0000) 2024-09-21T06:50:00.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:59 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:59.721+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:50:00.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:59 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:59.721+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:50:00.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:59 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:59.721+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:50:00.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:59 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:59.721+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.478941+0000 front 2024-09-21T06:49:27.478897+0000 (oldest deadline 2024-09-21T06:49:52.179050+0000) 2024-09-21T06:50:00.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:49:59 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:49:59.721+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:36.680995+0000 front 2024-09-21T06:49:36.681080+0000 (oldest deadline 2024-09-21T06:49:59.580953+0000) 2024-09-21T06:50:00.053 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:59 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:59.745+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:50:00.053 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:59 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:59.745+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:50:00.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:59 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:59.745+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:50:00.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:49:59 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:49:59.745+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:50:00.732 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:00 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:00.671+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:50:00.732 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:00 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:00.671+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:50:00.732 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:00 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:00.671+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.489943+0000 front 2024-09-21T06:49:27.490040+0000 (oldest deadline 2024-09-21T06:49:50.989997+0000) 2024-09-21T06:50:00.732 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:00 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:00.671+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.090903+0000 front 2024-09-21T06:49:35.091108+0000 (oldest deadline 2024-09-21T06:49:57.990863+0000) 2024-09-21T06:50:00.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:00 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:00.437+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:00.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:00 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:00.437+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:00.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:00 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:00.437+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:00.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:00 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:00.437+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:00.774 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:50:00 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:50:00.405+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:11.492448+0000 front 2024-09-21T06:49:11.492364+0000 (oldest deadline 2024-09-21T06:49:37.392788+0000) 2024-09-21T06:50:00.774 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:50:00 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:50:00.405+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:17.393208+0000 front 2024-09-21T06:49:17.393358+0000 (oldest deadline 2024-09-21T06:49:43.293249+0000) 2024-09-21T06:50:00.774 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:50:00 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:50:00.405+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:25.494635+0000 front 2024-09-21T06:49:25.494522+0000 (oldest deadline 2024-09-21T06:49:50.794558+0000) 2024-09-21T06:50:00.774 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:50:00 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:50:00.405+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:34.895495+0000 front 2024-09-21T06:49:34.895522+0000 (oldest deadline 2024-09-21T06:49:58.995619+0000) 2024-09-21T06:50:01.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:00 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:00.704+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:50:01.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:00 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:00.704+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:50:01.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:00 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:00.704+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:50:01.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:00 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:00.704+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.478941+0000 front 2024-09-21T06:49:27.478897+0000 (oldest deadline 2024-09-21T06:49:52.179050+0000) 2024-09-21T06:50:01.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:00 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:00.704+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:36.680995+0000 front 2024-09-21T06:49:36.681080+0000 (oldest deadline 2024-09-21T06:49:59.580953+0000) 2024-09-21T06:50:01.053 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:00 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:00.772+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:50:01.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:00 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:00.772+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:50:01.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:00 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:00.772+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:50:01.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:00 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:00.772+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:50:01.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:00 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:00.772+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:34.667785+0000 front 2024-09-21T06:49:34.667749+0000 (oldest deadline 2024-09-21T06:50:00.567875+0000) 2024-09-21T06:50:01.732 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:01 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:01.678+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:50:01.732 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:01 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:01.678+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:50:01.732 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:01 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:01.678+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.489943+0000 front 2024-09-21T06:49:27.490040+0000 (oldest deadline 2024-09-21T06:49:50.989997+0000) 2024-09-21T06:50:01.732 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:01 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:01.678+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.090903+0000 front 2024-09-21T06:49:35.091108+0000 (oldest deadline 2024-09-21T06:49:57.990863+0000) 2024-09-21T06:50:01.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:01 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:01.461+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:01.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:01 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:01.461+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:01.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:01 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:01.461+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:01.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:01 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:01.461+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:01.771 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:50:01 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:50:01.424+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:11.492448+0000 front 2024-09-21T06:49:11.492364+0000 (oldest deadline 2024-09-21T06:49:37.392788+0000) 2024-09-21T06:50:01.772 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:50:01 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:50:01.424+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:17.393208+0000 front 2024-09-21T06:49:17.393358+0000 (oldest deadline 2024-09-21T06:49:43.293249+0000) 2024-09-21T06:50:01.772 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:50:01 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:50:01.424+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:25.494635+0000 front 2024-09-21T06:49:25.494522+0000 (oldest deadline 2024-09-21T06:49:50.794558+0000) 2024-09-21T06:50:01.772 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:50:01 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7[50002]: 2024-09-21T06:50:01.424+0000 7fb24720e640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:34.895495+0000 front 2024-09-21T06:49:34.895522+0000 (oldest deadline 2024-09-21T06:49:58.995619+0000) 2024-09-21T06:50:02.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:01 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:01.666+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:50:02.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:01 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:01.666+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:50:02.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:01 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:01.666+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:50:02.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:01 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:01.666+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.478941+0000 front 2024-09-21T06:49:27.478897+0000 (oldest deadline 2024-09-21T06:49:52.179050+0000) 2024-09-21T06:50:02.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:01 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:01.666+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:36.680995+0000 front 2024-09-21T06:49:36.681080+0000 (oldest deadline 2024-09-21T06:49:59.580953+0000) 2024-09-21T06:50:02.053 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:01 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:01.769+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:50:02.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:01 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:01.769+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:50:02.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:01 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:01.769+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:50:02.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:01 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:01.769+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:50:02.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:01 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:01.769+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:34.667785+0000 front 2024-09-21T06:49:34.667749+0000 (oldest deadline 2024-09-21T06:50:00.567875+0000) 2024-09-21T06:50:02.481 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:50:02 smithi100 podman[64854]: 2024-09-21 06:50:02.203899546 +0000 UTC m=+5.634978574 container died 0f9b5b2533c38e18dd10a7dae25ea21d9c97d49fa57efcb1656025666c33e3b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-21T06:50:02.732 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:02 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:02.633+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:50:02.732 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:02 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:02.633+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:50:02.732 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:02 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:02.633+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.489943+0000 front 2024-09-21T06:49:27.490040+0000 (oldest deadline 2024-09-21T06:49:50.989997+0000) 2024-09-21T06:50:02.732 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:02 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:02.633+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.090903+0000 front 2024-09-21T06:49:35.091108+0000 (oldest deadline 2024-09-21T06:49:57.990863+0000) 2024-09-21T06:50:02.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:02 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:02.440+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:02.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:02 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:02.440+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:02.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:02 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:02.440+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:02.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:02 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:02.440+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:02.798 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:50:02 smithi100 podman[64854]: 2024-09-21 06:50:02.570649721 +0000 UTC m=+6.001728743 container cleanup 0f9b5b2533c38e18dd10a7dae25ea21d9c97d49fa57efcb1656025666c33e3b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:50:02.798 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:50:02 smithi100 bash[64854]: ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7 2024-09-21T06:50:03.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:02 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:02.668+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:50:03.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:02 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:02.668+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:50:03.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:02 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:02.668+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:50:03.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:02 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:02.668+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.478941+0000 front 2024-09-21T06:49:27.478897+0000 (oldest deadline 2024-09-21T06:49:52.179050+0000) 2024-09-21T06:50:03.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:02 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:02.668+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:36.680995+0000 front 2024-09-21T06:49:36.681080+0000 (oldest deadline 2024-09-21T06:49:59.580953+0000) 2024-09-21T06:50:03.053 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:50:02 smithi100 podman[64873]: 2024-09-21 06:50:02.813449236 +0000 UTC m=+0.611924875 container remove 0f9b5b2533c38e18dd10a7dae25ea21d9c97d49fa57efcb1656025666c33e3b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:50:03.053 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:02 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:02.796+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:50:03.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:02 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:02.796+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:50:03.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:02 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:02.796+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:50:03.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:02 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:02.796+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:50:03.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:02 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:02.796+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:34.667785+0000 front 2024-09-21T06:49:34.667749+0000 (oldest deadline 2024-09-21T06:50:00.567875+0000) 2024-09-21T06:50:03.410 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:50:03 smithi100 podman[64964]: 2024-09-21 06:50:03.031953304 +0000 UTC m=+0.016013594 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:50:03.410 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:50:03 smithi100 podman[64964]: 2024-09-21 06:50:03.156399601 +0000 UTC m=+0.140459882 container create 4ecfa9da670ce60dec83cfa37e011fb5dde531a50a91b8a7c0a8fb5d7d4ed08f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-21T06:50:03.410 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:50:03 smithi100 podman[64964]: 2024-09-21 06:50:03.408879867 +0000 UTC m=+0.392940152 container init 4ecfa9da670ce60dec83cfa37e011fb5dde531a50a91b8a7c0a8fb5d7d4ed08f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True) 2024-09-21T06:50:03.732 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:03 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:03.635+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:50:03.732 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:03 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:03.635+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:50:03.732 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:03 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:03.635+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.489943+0000 front 2024-09-21T06:49:27.490040+0000 (oldest deadline 2024-09-21T06:49:50.989997+0000) 2024-09-21T06:50:03.732 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:03 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:03.635+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.090903+0000 front 2024-09-21T06:49:35.091108+0000 (oldest deadline 2024-09-21T06:49:57.990863+0000) 2024-09-21T06:50:03.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:03 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:03.438+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:03.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:03 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:03.438+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:03.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:03 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:03.438+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:03.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:03 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:03.438+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:03.804 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:50:03 smithi100 podman[64964]: 2024-09-21 06:50:03.412275626 +0000 UTC m=+0.396335911 container start 4ecfa9da670ce60dec83cfa37e011fb5dde531a50a91b8a7c0a8fb5d7d4ed08f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:50:03.804 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:50:03 smithi100 podman[64964]: 2024-09-21 06:50:03.508960283 +0000 UTC m=+0.493020585 container attach 4ecfa9da670ce60dec83cfa37e011fb5dde531a50a91b8a7c0a8fb5d7d4ed08f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-21T06:50:03.804 INFO:journalctl@ceph.osd.7.smithi100.stdout:Sep 21 06:50:03 smithi100 podman[64964]: 2024-09-21 06:50:03.609844057 +0000 UTC m=+0.593904349 container died 4ecfa9da670ce60dec83cfa37e011fb5dde531a50a91b8a7c0a8fb5d7d4ed08f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2) 2024-09-21T06:50:04.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:03 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:03.646+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:50:04.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:03 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:03.646+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:50:04.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:03 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:03.646+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:50:04.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:03 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:03.646+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.478941+0000 front 2024-09-21T06:49:27.478897+0000 (oldest deadline 2024-09-21T06:49:52.179050+0000) 2024-09-21T06:50:04.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:03 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:03.646+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:36.680995+0000 front 2024-09-21T06:49:36.681080+0000 (oldest deadline 2024-09-21T06:49:59.580953+0000) 2024-09-21T06:50:04.197 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:03 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:03.805+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:50:04.197 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:03 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:03.805+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:50:04.197 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:03 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:03.805+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:50:04.197 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:03 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:03.805+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:50:04.197 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:03 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:03.805+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:34.667785+0000 front 2024-09-21T06:49:34.667749+0000 (oldest deadline 2024-09-21T06:50:00.567875+0000) 2024-09-21T06:50:04.221 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.7.service' 2024-09-21T06:50:04.262 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:50:04.262 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-21T06:50:04.262 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-21T06:50:04.262 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.8 2024-09-21T06:50:04.589 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:04 smithi203 systemd[1]: Stopping Ceph osd.8 for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:50:04.589 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:04 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:04.410+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:04.589 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:04 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:04.410+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:04.589 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:04 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:04.410+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:04.589 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:04 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:04.410+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:04.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:04 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:04.587+0000 7f4c1a436640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:50:04.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:04 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:04.587+0000 7f4c1a436640 -1 osd.8 89 *** Got signal Terminated *** 2024-09-21T06:50:04.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:04 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:04.587+0000 7f4c1a436640 -1 osd.8 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:50:04.983 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:04 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:04.591+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:50:04.983 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:04 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:04.591+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:50:04.983 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:04 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:04.591+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.489943+0000 front 2024-09-21T06:49:27.490040+0000 (oldest deadline 2024-09-21T06:49:50.989997+0000) 2024-09-21T06:50:04.983 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:04 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:04.591+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.090903+0000 front 2024-09-21T06:49:35.091108+0000 (oldest deadline 2024-09-21T06:49:57.990863+0000) 2024-09-21T06:50:05.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:04 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:04.693+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:50:05.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:04 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:04.693+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:50:05.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:04 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:04.693+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:50:05.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:04 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:04.693+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.478941+0000 front 2024-09-21T06:49:27.478897+0000 (oldest deadline 2024-09-21T06:49:52.179050+0000) 2024-09-21T06:50:05.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:04 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:04.693+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:36.680995+0000 front 2024-09-21T06:49:36.681080+0000 (oldest deadline 2024-09-21T06:49:59.580953+0000) 2024-09-21T06:50:05.053 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:04 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:04.756+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:50:05.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:04 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:04.756+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:50:05.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:04 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:04.756+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:50:05.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:04 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:04.756+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:50:05.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:04 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:04.756+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:34.667785+0000 front 2024-09-21T06:49:34.667749+0000 (oldest deadline 2024-09-21T06:50:00.567875+0000) 2024-09-21T06:50:05.640 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:05 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:05.385+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:05.640 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:05 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:05.385+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:05.640 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:05 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:05.385+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:05.640 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:05 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:05.385+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:05.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:05 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:05.638+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:50:05.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:05 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:05.638+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:50:05.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:05 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:05.638+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.489943+0000 front 2024-09-21T06:49:27.490040+0000 (oldest deadline 2024-09-21T06:49:50.989997+0000) 2024-09-21T06:50:05.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:05 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:05.638+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.090903+0000 front 2024-09-21T06:49:35.091108+0000 (oldest deadline 2024-09-21T06:49:57.990863+0000) 2024-09-21T06:50:06.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:05 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:05.656+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:50:06.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:05 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:05.656+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:50:06.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:05 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:05.656+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:50:06.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:05 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:05.656+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.478941+0000 front 2024-09-21T06:49:27.478897+0000 (oldest deadline 2024-09-21T06:49:52.179050+0000) 2024-09-21T06:50:06.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:05 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:05.656+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:36.680995+0000 front 2024-09-21T06:49:36.681080+0000 (oldest deadline 2024-09-21T06:49:59.580953+0000) 2024-09-21T06:50:06.053 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:05 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:05.771+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:50:06.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:05 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:05.771+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:50:06.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:05 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:05.771+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:50:06.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:05 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:05.771+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:50:06.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:05 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:05.771+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:34.667785+0000 front 2024-09-21T06:49:34.667749+0000 (oldest deadline 2024-09-21T06:50:00.567875+0000) 2024-09-21T06:50:06.679 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:06 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:06.415+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:06.679 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:06 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:06.415+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:06.680 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:06 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:06.415+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:06.680 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:06 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:06.415+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:06.680 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:06 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:06.415+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.409872+0000 front 2024-09-21T06:49:42.409992+0000 (oldest deadline 2024-09-21T06:50:05.910048+0000) 2024-09-21T06:50:06.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:06 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:06.678+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:50:06.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:06 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:06.678+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:50:06.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:06 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:06.678+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.489943+0000 front 2024-09-21T06:49:27.490040+0000 (oldest deadline 2024-09-21T06:49:50.989997+0000) 2024-09-21T06:50:06.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:06 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:06.678+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.090903+0000 front 2024-09-21T06:49:35.091108+0000 (oldest deadline 2024-09-21T06:49:57.990863+0000) 2024-09-21T06:50:07.041 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:06 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:06.649+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:50:07.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:06 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:06.649+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:50:07.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:06 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:06.649+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:50:07.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:06 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:06.649+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.478941+0000 front 2024-09-21T06:49:27.478897+0000 (oldest deadline 2024-09-21T06:49:52.179050+0000) 2024-09-21T06:50:07.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:06 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:06.649+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:36.680995+0000 front 2024-09-21T06:49:36.681080+0000 (oldest deadline 2024-09-21T06:49:59.580953+0000) 2024-09-21T06:50:07.053 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:06 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:06.794+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:50:07.053 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:06 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:06.794+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:50:07.053 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:06 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:06.794+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:50:07.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:06 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:06.794+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:50:07.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:06 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:06.794+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:34.667785+0000 front 2024-09-21T06:49:34.667749+0000 (oldest deadline 2024-09-21T06:50:00.567875+0000) 2024-09-21T06:50:07.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:07 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:07.428+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:07.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:07 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:07.428+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:07.710 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:07 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:07.428+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:07.710 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:07 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:07.428+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:07.710 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:07 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:07.428+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.409872+0000 front 2024-09-21T06:49:42.409992+0000 (oldest deadline 2024-09-21T06:50:05.910048+0000) 2024-09-21T06:50:07.943 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:07 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:07.614+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:50:07.943 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:07 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:07.614+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:50:07.943 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:07 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:07.614+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:50:07.943 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:07 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:07.614+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.478941+0000 front 2024-09-21T06:49:27.478897+0000 (oldest deadline 2024-09-21T06:49:52.179050+0000) 2024-09-21T06:50:07.944 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:07 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:07.614+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:36.680995+0000 front 2024-09-21T06:49:36.681080+0000 (oldest deadline 2024-09-21T06:49:59.580953+0000) 2024-09-21T06:50:07.944 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:07 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:07.614+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:41.782471+0000 front 2024-09-21T06:49:41.782449+0000 (oldest deadline 2024-09-21T06:50:07.082285+0000) 2024-09-21T06:50:07.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:07 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:07.708+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:50:07.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:07 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:07.708+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:50:07.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:07 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:07.708+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.489943+0000 front 2024-09-21T06:49:27.490040+0000 (oldest deadline 2024-09-21T06:49:50.989997+0000) 2024-09-21T06:50:07.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:07 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:07.708+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.090903+0000 front 2024-09-21T06:49:35.091108+0000 (oldest deadline 2024-09-21T06:49:57.990863+0000) 2024-09-21T06:50:08.053 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:07 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:07.775+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:50:08.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:07 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:07.775+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:50:08.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:07 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:07.775+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:50:08.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:07 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:07.775+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:50:08.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:07 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:07.775+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:34.667785+0000 front 2024-09-21T06:49:34.667749+0000 (oldest deadline 2024-09-21T06:50:00.567875+0000) 2024-09-21T06:50:08.720 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:08 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:08.435+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:08.720 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:08 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:08.435+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:08.720 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:08 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:08.435+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:08.721 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:08 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:08.435+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:08.721 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:08 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:08.435+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.409872+0000 front 2024-09-21T06:49:42.409992+0000 (oldest deadline 2024-09-21T06:50:05.910048+0000) 2024-09-21T06:50:08.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:08 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:08.718+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:01.786135+0000 front 2024-09-21T06:49:01.786236+0000 (oldest deadline 2024-09-21T06:49:25.786917+0000) 2024-09-21T06:50:08.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:08 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:08.718+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.488180+0000 front 2024-09-21T06:49:10.487998+0000 (oldest deadline 2024-09-21T06:49:36.387910+0000) 2024-09-21T06:50:08.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:08 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:08.718+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.489943+0000 front 2024-09-21T06:49:27.490040+0000 (oldest deadline 2024-09-21T06:49:50.989997+0000) 2024-09-21T06:50:08.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:08 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:08.718+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.090903+0000 front 2024-09-21T06:49:35.091108+0000 (oldest deadline 2024-09-21T06:49:57.990863+0000) 2024-09-21T06:50:08.983 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:08 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8[50328]: 2024-09-21T06:50:08.718+0000 7f4c12a19640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.592109+0000 front 2024-09-21T06:49:42.592147+0000 (oldest deadline 2024-09-21T06:50:08.492292+0000) 2024-09-21T06:50:09.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:08 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:08.659+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:50:09.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:08 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:08.659+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:50:09.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:08 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:08.659+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:50:09.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:08 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:08.659+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.478941+0000 front 2024-09-21T06:49:27.478897+0000 (oldest deadline 2024-09-21T06:49:52.179050+0000) 2024-09-21T06:50:09.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:08 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:08.659+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:36.680995+0000 front 2024-09-21T06:49:36.681080+0000 (oldest deadline 2024-09-21T06:49:59.580953+0000) 2024-09-21T06:50:09.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:08 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:08.659+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:41.782471+0000 front 2024-09-21T06:49:41.782449+0000 (oldest deadline 2024-09-21T06:50:07.082285+0000) 2024-09-21T06:50:09.053 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:08 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:08.801+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:50:09.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:08 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:08.801+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:50:09.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:08 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:08.801+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:50:09.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:08 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:08.801+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:50:09.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:08 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:08.801+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:34.667785+0000 front 2024-09-21T06:49:34.667749+0000 (oldest deadline 2024-09-21T06:50:00.567875+0000) 2024-09-21T06:50:09.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:08 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:08.801+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.868894+0000 front 2024-09-21T06:49:42.868778+0000 (oldest deadline 2024-09-21T06:50:08.168688+0000) 2024-09-21T06:50:09.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:09 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:09.410+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:09.734 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:09 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:09.410+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:09.734 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:09 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:09.410+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:09.734 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:09 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:09.410+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:09.734 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:09 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:09.410+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.409872+0000 front 2024-09-21T06:49:42.409992+0000 (oldest deadline 2024-09-21T06:50:05.910048+0000) 2024-09-21T06:50:10.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:09 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:09.636+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:50:10.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:09 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:09.636+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:50:10.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:09 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:09.636+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:50:10.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:09 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:09.636+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.478941+0000 front 2024-09-21T06:49:27.478897+0000 (oldest deadline 2024-09-21T06:49:52.179050+0000) 2024-09-21T06:50:10.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:09 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:09.636+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:36.680995+0000 front 2024-09-21T06:49:36.681080+0000 (oldest deadline 2024-09-21T06:49:59.580953+0000) 2024-09-21T06:50:10.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:09 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:09.636+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:41.782471+0000 front 2024-09-21T06:49:41.782449+0000 (oldest deadline 2024-09-21T06:50:07.082285+0000) 2024-09-21T06:50:10.053 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:09 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:09.762+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:50:10.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:09 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:09.762+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:50:10.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:09 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:09.762+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:50:10.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:09 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:09.762+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:50:10.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:09 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:09.762+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:34.667785+0000 front 2024-09-21T06:49:34.667749+0000 (oldest deadline 2024-09-21T06:50:00.567875+0000) 2024-09-21T06:50:10.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:09 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:09.762+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.868894+0000 front 2024-09-21T06:49:42.868778+0000 (oldest deadline 2024-09-21T06:50:08.168688+0000) 2024-09-21T06:50:10.439 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:10 smithi203 podman[65594]: 2024-09-21 06:50:10.171067585 +0000 UTC m=+5.704923340 container died c24b9ef02b98fb7cab4cd232016e3b33129594ccab3752df8250efc94931c9fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-21T06:50:10.440 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:10 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:10.385+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:10.440 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:10 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:10.385+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:10.440 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:10 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:10.385+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:10.440 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:10 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:10.385+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:10.440 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:10 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:10.385+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.409872+0000 front 2024-09-21T06:49:42.409992+0000 (oldest deadline 2024-09-21T06:50:05.910048+0000) 2024-09-21T06:50:10.732 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:10 smithi203 podman[65594]: 2024-09-21 06:50:10.537227059 +0000 UTC m=+6.071082858 container cleanup c24b9ef02b98fb7cab4cd232016e3b33129594ccab3752df8250efc94931c9fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:50:10.732 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:10 smithi203 bash[65594]: ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8 2024-09-21T06:50:11.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:10 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:10.594+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:50:11.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:10 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:10.594+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:50:11.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:10 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:10.594+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:50:11.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:10 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:10.594+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.478941+0000 front 2024-09-21T06:49:27.478897+0000 (oldest deadline 2024-09-21T06:49:52.179050+0000) 2024-09-21T06:50:11.043 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:10 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:10.594+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:36.680995+0000 front 2024-09-21T06:49:36.681080+0000 (oldest deadline 2024-09-21T06:49:59.580953+0000) 2024-09-21T06:50:11.043 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:10 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:10.594+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:41.782471+0000 front 2024-09-21T06:49:41.782449+0000 (oldest deadline 2024-09-21T06:50:07.082285+0000) 2024-09-21T06:50:11.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:10 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:10.789+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:50:11.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:10 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:10.789+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:50:11.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:10 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:10.789+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:50:11.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:10 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:10.789+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:50:11.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:10 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:10.789+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:34.667785+0000 front 2024-09-21T06:49:34.667749+0000 (oldest deadline 2024-09-21T06:50:00.567875+0000) 2024-09-21T06:50:11.055 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:10 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:10.789+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.868894+0000 front 2024-09-21T06:49:42.868778+0000 (oldest deadline 2024-09-21T06:50:08.168688+0000) 2024-09-21T06:50:11.144 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:10 smithi203 podman[65609]: 2024-09-21 06:50:10.804881349 +0000 UTC m=+0.635158944 container remove c24b9ef02b98fb7cab4cd232016e3b33129594ccab3752df8250efc94931c9fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-09-21T06:50:11.417 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:11 smithi203 podman[65695]: 2024-09-21 06:50:11.046108407 +0000 UTC m=+0.015803401 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:50:11.418 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:11 smithi203 podman[65695]: 2024-09-21 06:50:11.147266293 +0000 UTC m=+0.116961278 container create de890502fc37d86d5596a005fec53d2a02f1b134fccf7096e7fe27e0a15d100d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-21T06:50:11.418 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:11 smithi203 podman[65695]: 2024-09-21 06:50:11.41663094 +0000 UTC m=+0.386325936 container init de890502fc37d86d5596a005fec53d2a02f1b134fccf7096e7fe27e0a15d100d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-21T06:50:11.418 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:11 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:11.374+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:11.418 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:11 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:11.374+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:11.418 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:11 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:11.374+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:11.418 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:11 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:11.374+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:11.418 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:11 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:11.374+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.409872+0000 front 2024-09-21T06:49:42.409992+0000 (oldest deadline 2024-09-21T06:50:05.910048+0000) 2024-09-21T06:50:11.732 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:11 smithi203 podman[65695]: 2024-09-21 06:50:11.420370047 +0000 UTC m=+0.390065036 container start de890502fc37d86d5596a005fec53d2a02f1b134fccf7096e7fe27e0a15d100d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8-deactivate, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-21T06:50:11.732 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:11 smithi203 podman[65695]: 2024-09-21 06:50:11.49172666 +0000 UTC m=+0.461421665 container attach de890502fc37d86d5596a005fec53d2a02f1b134fccf7096e7fe27e0a15d100d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:50:11.733 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 06:50:11 smithi203 podman[65695]: 2024-09-21 06:50:11.610551701 +0000 UTC m=+0.580246694 container died de890502fc37d86d5596a005fec53d2a02f1b134fccf7096e7fe27e0a15d100d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:50:12.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:11 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:11.551+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:50:12.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:11 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:11.551+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:50:12.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:11 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:11.551+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:50:12.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:11 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:11.551+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.478941+0000 front 2024-09-21T06:49:27.478897+0000 (oldest deadline 2024-09-21T06:49:52.179050+0000) 2024-09-21T06:50:12.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:11 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:11.551+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:36.680995+0000 front 2024-09-21T06:49:36.681080+0000 (oldest deadline 2024-09-21T06:49:59.580953+0000) 2024-09-21T06:50:12.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:11 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:11.551+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:41.782471+0000 front 2024-09-21T06:49:41.782449+0000 (oldest deadline 2024-09-21T06:50:07.082285+0000) 2024-09-21T06:50:12.053 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:11 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:11.765+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:50:12.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:11 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:11.765+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:50:12.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:11 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:11.765+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:50:12.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:11 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:11.765+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:50:12.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:11 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:11.765+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:34.667785+0000 front 2024-09-21T06:49:34.667749+0000 (oldest deadline 2024-09-21T06:50:00.567875+0000) 2024-09-21T06:50:12.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:11 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:11.765+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.868894+0000 front 2024-09-21T06:49:42.868778+0000 (oldest deadline 2024-09-21T06:50:08.168688+0000) 2024-09-21T06:50:12.228 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.8.service' 2024-09-21T06:50:12.266 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:50:12.267 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-21T06:50:12.267 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-21T06:50:12.267 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.9 2024-09-21T06:50:12.478 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:12 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:12.370+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:12.478 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:12 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:12.370+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:12.478 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:12 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:12.370+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:12.478 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:12 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:12.370+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:12.478 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:12 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:12.370+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.409872+0000 front 2024-09-21T06:49:42.409992+0000 (oldest deadline 2024-09-21T06:50:05.910048+0000) 2024-09-21T06:50:12.552 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:12 smithi067 systemd[1]: Stopping Ceph osd.9 for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:50:13.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:12 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:12.550+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:50:13.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:12 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:12.550+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:50:13.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:12 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:12.550+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:50:13.043 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:12 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:12.550+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.478941+0000 front 2024-09-21T06:49:27.478897+0000 (oldest deadline 2024-09-21T06:49:52.179050+0000) 2024-09-21T06:50:13.043 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:12 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:12.550+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:36.680995+0000 front 2024-09-21T06:49:36.681080+0000 (oldest deadline 2024-09-21T06:49:59.580953+0000) 2024-09-21T06:50:13.043 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:12 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:12.550+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:41.782471+0000 front 2024-09-21T06:49:41.782449+0000 (oldest deadline 2024-09-21T06:50:07.082285+0000) 2024-09-21T06:50:13.043 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:12 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:12.687+0000 7f3cad971640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:50:13.043 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:12 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:12.687+0000 7f3cad971640 -1 osd.9 89 *** Got signal Terminated *** 2024-09-21T06:50:13.043 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:12 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:12.687+0000 7f3cad971640 -1 osd.9 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:50:13.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:12 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:12.779+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:50:13.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:12 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:12.779+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:50:13.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:12 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:12.779+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:50:13.055 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:12 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:12.779+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:50:13.055 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:12 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:12.779+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:34.667785+0000 front 2024-09-21T06:49:34.667749+0000 (oldest deadline 2024-09-21T06:50:00.567875+0000) 2024-09-21T06:50:13.055 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:12 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:12.779+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.868894+0000 front 2024-09-21T06:49:42.868778+0000 (oldest deadline 2024-09-21T06:50:08.168688+0000) 2024-09-21T06:50:13.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:13 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:13.386+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:13.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:13 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:13.386+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:13.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:13 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:13.386+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:13.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:13 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:13.386+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:13.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:13 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:13.386+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.409872+0000 front 2024-09-21T06:49:42.409992+0000 (oldest deadline 2024-09-21T06:50:05.910048+0000) 2024-09-21T06:50:13.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:13 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:13.505+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:50:13.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:13 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:13.505+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:50:13.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:13 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:13.505+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:50:13.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:13 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:13.505+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.478941+0000 front 2024-09-21T06:49:27.478897+0000 (oldest deadline 2024-09-21T06:49:52.179050+0000) 2024-09-21T06:50:13.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:13 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:13.505+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:36.680995+0000 front 2024-09-21T06:49:36.681080+0000 (oldest deadline 2024-09-21T06:49:59.580953+0000) 2024-09-21T06:50:13.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:13 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:13.505+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:41.782471+0000 front 2024-09-21T06:49:41.782449+0000 (oldest deadline 2024-09-21T06:50:07.082285+0000) 2024-09-21T06:50:14.053 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:13 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:13.772+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:50:14.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:13 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:13.772+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:50:14.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:13 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:13.772+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:50:14.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:13 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:13.772+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:50:14.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:13 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:13.772+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:34.667785+0000 front 2024-09-21T06:49:34.667749+0000 (oldest deadline 2024-09-21T06:50:00.567875+0000) 2024-09-21T06:50:14.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:13 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:13.772+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.868894+0000 front 2024-09-21T06:49:42.868778+0000 (oldest deadline 2024-09-21T06:50:08.168688+0000) 2024-09-21T06:50:14.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:14 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:14.385+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:14.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:14 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:14.385+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:14.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:14 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:14.385+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:14.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:14 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:14.385+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:14.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:14 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:14.385+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.409872+0000 front 2024-09-21T06:49:42.409992+0000 (oldest deadline 2024-09-21T06:50:05.910048+0000) 2024-09-21T06:50:14.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:14 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:14.385+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-09-21T06:49:51.811098+0000 front 2024-09-21T06:49:51.811251+0000 (oldest deadline 2024-09-21T06:50:14.111041+0000) 2024-09-21T06:50:14.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:14 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:14.463+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:50:14.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:14 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:14.463+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:50:14.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:14 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:14.463+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:50:14.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:14 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:14.463+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.478941+0000 front 2024-09-21T06:49:27.478897+0000 (oldest deadline 2024-09-21T06:49:52.179050+0000) 2024-09-21T06:50:14.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:14 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:14.463+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:36.680995+0000 front 2024-09-21T06:49:36.681080+0000 (oldest deadline 2024-09-21T06:49:59.580953+0000) 2024-09-21T06:50:14.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:14 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:14.463+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:41.782471+0000 front 2024-09-21T06:49:41.782449+0000 (oldest deadline 2024-09-21T06:50:07.082285+0000) 2024-09-21T06:50:15.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:14 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:14.771+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:50:15.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:14 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:14.771+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:50:15.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:14 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:14.771+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:50:15.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:14 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:14.771+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:50:15.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:14 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:14.771+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:34.667785+0000 front 2024-09-21T06:49:34.667749+0000 (oldest deadline 2024-09-21T06:50:00.567875+0000) 2024-09-21T06:50:15.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:14 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:14.771+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.868894+0000 front 2024-09-21T06:49:42.868778+0000 (oldest deadline 2024-09-21T06:50:08.168688+0000) 2024-09-21T06:50:15.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:15 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:15.383+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:15.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:15 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:15.383+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:15.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:15 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:15.383+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:15.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:15 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:15.383+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:15.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:15 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:15.383+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.409872+0000 front 2024-09-21T06:49:42.409992+0000 (oldest deadline 2024-09-21T06:50:05.910048+0000) 2024-09-21T06:50:15.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:15 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:15.383+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-09-21T06:49:51.811098+0000 front 2024-09-21T06:49:51.811251+0000 (oldest deadline 2024-09-21T06:50:14.111041+0000) 2024-09-21T06:50:15.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:15 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:15.462+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:50:15.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:15 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:15.462+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:50:15.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:15 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:15.462+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:50:15.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:15 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:15.462+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.478941+0000 front 2024-09-21T06:49:27.478897+0000 (oldest deadline 2024-09-21T06:49:52.179050+0000) 2024-09-21T06:50:15.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:15 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:15.462+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:36.680995+0000 front 2024-09-21T06:49:36.681080+0000 (oldest deadline 2024-09-21T06:49:59.580953+0000) 2024-09-21T06:50:15.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:15 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:15.462+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:41.782471+0000 front 2024-09-21T06:49:41.782449+0000 (oldest deadline 2024-09-21T06:50:07.082285+0000) 2024-09-21T06:50:16.053 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:15 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:15.772+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:50:16.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:15 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:15.772+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:50:16.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:15 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:15.772+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:50:16.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:15 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:15.772+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:50:16.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:15 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:15.772+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:34.667785+0000 front 2024-09-21T06:49:34.667749+0000 (oldest deadline 2024-09-21T06:50:00.567875+0000) 2024-09-21T06:50:16.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:15 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:15.772+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.868894+0000 front 2024-09-21T06:49:42.868778+0000 (oldest deadline 2024-09-21T06:50:08.168688+0000) 2024-09-21T06:50:16.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:16 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:16.377+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:16.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:16 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:16.377+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:16.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:16 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:16.377+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:16.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:16 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:16.377+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:16.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:16 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:16.377+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.409872+0000 front 2024-09-21T06:49:42.409992+0000 (oldest deadline 2024-09-21T06:50:05.910048+0000) 2024-09-21T06:50:16.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:16 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:16.377+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-09-21T06:49:51.811098+0000 front 2024-09-21T06:49:51.811251+0000 (oldest deadline 2024-09-21T06:50:14.111041+0000) 2024-09-21T06:50:16.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:16 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:16.497+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:50:16.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:16 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:16.497+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:50:16.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:16 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:16.497+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:50:16.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:16 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:16.497+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.478941+0000 front 2024-09-21T06:49:27.478897+0000 (oldest deadline 2024-09-21T06:49:52.179050+0000) 2024-09-21T06:50:16.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:16 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:16.497+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:36.680995+0000 front 2024-09-21T06:49:36.681080+0000 (oldest deadline 2024-09-21T06:49:59.580953+0000) 2024-09-21T06:50:16.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:16 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:16.497+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:41.782471+0000 front 2024-09-21T06:49:41.782449+0000 (oldest deadline 2024-09-21T06:50:07.082285+0000) 2024-09-21T06:50:17.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:16 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:16.758+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:50:17.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:16 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:16.758+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:50:17.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:16 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:16.758+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:50:17.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:16 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:16.758+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:50:17.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:16 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:16.758+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:34.667785+0000 front 2024-09-21T06:49:34.667749+0000 (oldest deadline 2024-09-21T06:50:00.567875+0000) 2024-09-21T06:50:17.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:16 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:16.758+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.868894+0000 front 2024-09-21T06:49:42.868778+0000 (oldest deadline 2024-09-21T06:50:08.168688+0000) 2024-09-21T06:50:17.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:17 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:17.389+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:17.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:17 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:17.389+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:17.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:17 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:17.389+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:17.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:17 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:17.389+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:17.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:17 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:17.389+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.409872+0000 front 2024-09-21T06:49:42.409992+0000 (oldest deadline 2024-09-21T06:50:05.910048+0000) 2024-09-21T06:50:17.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:17 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:17.389+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-09-21T06:49:51.811098+0000 front 2024-09-21T06:49:51.811251+0000 (oldest deadline 2024-09-21T06:50:14.111041+0000) 2024-09-21T06:50:17.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:17 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:17.468+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:04.075856+0000 front 2024-09-21T06:49:04.075774+0000 (oldest deadline 2024-09-21T06:49:27.575960+0000) 2024-09-21T06:50:17.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:17 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:17.468+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.277035+0000 front 2024-09-21T06:49:12.277252+0000 (oldest deadline 2024-09-21T06:49:33.977079+0000) 2024-09-21T06:50:17.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:17 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:17.468+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.878129+0000 front 2024-09-21T06:49:19.878346+0000 (oldest deadline 2024-09-21T06:49:44.578083+0000) 2024-09-21T06:50:17.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:17 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:17.468+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:27.478941+0000 front 2024-09-21T06:49:27.478897+0000 (oldest deadline 2024-09-21T06:49:52.179050+0000) 2024-09-21T06:50:17.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:17 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:17.468+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:36.680995+0000 front 2024-09-21T06:49:36.681080+0000 (oldest deadline 2024-09-21T06:49:59.580953+0000) 2024-09-21T06:50:17.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:17 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9[65761]: 2024-09-21T06:50:17.468+0000 7f3ca9f5c640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:41.782471+0000 front 2024-09-21T06:49:41.782449+0000 (oldest deadline 2024-09-21T06:50:07.082285+0000) 2024-09-21T06:50:18.053 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:17 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:17.794+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:50:18.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:17 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:17.794+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:50:18.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:17 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:17.794+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:50:18.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:17 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:17.794+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:50:18.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:17 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:17.794+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:34.667785+0000 front 2024-09-21T06:49:34.667749+0000 (oldest deadline 2024-09-21T06:50:00.567875+0000) 2024-09-21T06:50:18.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:17 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:17.794+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.868894+0000 front 2024-09-21T06:49:42.868778+0000 (oldest deadline 2024-09-21T06:50:08.168688+0000) 2024-09-21T06:50:18.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:17 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:17.794+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-09-21T06:49:52.170471+0000 front 2024-09-21T06:49:52.170349+0000 (oldest deadline 2024-09-21T06:50:16.770511+0000) 2024-09-21T06:50:18.597 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:18 smithi067 podman[100423]: 2024-09-21 06:50:18.335649259 +0000 UTC m=+5.860237321 container died 908e132948bb467eda2b5df7a59be4e1360d49d249153350efe3ab947b76cd25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-21T06:50:18.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:18 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:18.367+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:18.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:18 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:18.367+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:18.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:18 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:18.367+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:18.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:18 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:18.367+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:18.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:18 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:18.367+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.409872+0000 front 2024-09-21T06:49:42.409992+0000 (oldest deadline 2024-09-21T06:50:05.910048+0000) 2024-09-21T06:50:18.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:18 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:18.367+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-09-21T06:49:51.811098+0000 front 2024-09-21T06:49:51.811251+0000 (oldest deadline 2024-09-21T06:50:14.111041+0000) 2024-09-21T06:50:19.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:18 smithi067 podman[100423]: 2024-09-21 06:50:18.795626255 +0000 UTC m=+6.320214316 container cleanup 908e132948bb467eda2b5df7a59be4e1360d49d249153350efe3ab947b76cd25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:50:19.042 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:18 smithi067 bash[100423]: ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9 2024-09-21T06:50:19.053 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:18 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:18.763+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:50:19.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:18 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:18.763+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:50:19.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:18 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:18.763+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:50:19.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:18 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:18.763+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:50:19.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:18 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:18.763+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:34.667785+0000 front 2024-09-21T06:49:34.667749+0000 (oldest deadline 2024-09-21T06:50:00.567875+0000) 2024-09-21T06:50:19.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:18 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:18.763+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.868894+0000 front 2024-09-21T06:49:42.868778+0000 (oldest deadline 2024-09-21T06:50:08.168688+0000) 2024-09-21T06:50:19.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:18 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:18.763+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-09-21T06:49:52.170471+0000 front 2024-09-21T06:49:52.170349+0000 (oldest deadline 2024-09-21T06:50:16.770511+0000) 2024-09-21T06:50:19.507 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:19 smithi067 podman[100441]: 2024-09-21 06:50:19.188585489 +0000 UTC m=+0.849320058 container remove 908e132948bb467eda2b5df7a59be4e1360d49d249153350efe3ab947b76cd25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:50:19.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:19 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:19.318+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:19.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:19 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:19.318+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:19.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:19 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:19.318+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:19.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:19 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:19.318+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:19.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:19 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:19.318+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.409872+0000 front 2024-09-21T06:49:42.409992+0000 (oldest deadline 2024-09-21T06:50:05.910048+0000) 2024-09-21T06:50:19.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:19 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:19.318+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-09-21T06:49:51.811098+0000 front 2024-09-21T06:49:51.811251+0000 (oldest deadline 2024-09-21T06:50:14.111041+0000) 2024-09-21T06:50:19.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:19 smithi067 podman[100532]: 2024-09-21 06:50:19.409024625 +0000 UTC m=+0.016721575 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:50:19.792 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:19 smithi067 podman[100532]: 2024-09-21 06:50:19.53119934 +0000 UTC m=+0.138896323 container create d3026b246a002fc12122c37a79314876c253cf19339f8567459bc85ae4adf7a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-21T06:50:20.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:19 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:19.731+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:50:20.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:19 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:19.731+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:50:20.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:19 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:19.732+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:50:20.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:19 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:19.732+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:50:20.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:19 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:19.732+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:34.667785+0000 front 2024-09-21T06:49:34.667749+0000 (oldest deadline 2024-09-21T06:50:00.567875+0000) 2024-09-21T06:50:20.055 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:19 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:19.732+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.868894+0000 front 2024-09-21T06:49:42.868778+0000 (oldest deadline 2024-09-21T06:50:08.168688+0000) 2024-09-21T06:50:20.055 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:19 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:19.732+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-09-21T06:49:52.170471+0000 front 2024-09-21T06:49:52.170349+0000 (oldest deadline 2024-09-21T06:50:16.770511+0000) 2024-09-21T06:50:20.108 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:19 smithi067 podman[100532]: 2024-09-21 06:50:19.90850927 +0000 UTC m=+0.516206220 container init d3026b246a002fc12122c37a79314876c253cf19339f8567459bc85ae4adf7a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-21T06:50:20.108 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:19 smithi067 podman[100532]: 2024-09-21 06:50:19.911895543 +0000 UTC m=+0.519592485 container start d3026b246a002fc12122c37a79314876c253cf19339f8567459bc85ae4adf7a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-21T06:50:20.108 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:20 smithi067 podman[100532]: 2024-09-21 06:50:20.000254277 +0000 UTC m=+0.607951233 container attach d3026b246a002fc12122c37a79314876c253cf19339f8567459bc85ae4adf7a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-21T06:50:20.394 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 21 06:50:20 smithi067 podman[100532]: 2024-09-21 06:50:20.107390675 +0000 UTC m=+0.715087617 container died d3026b246a002fc12122c37a79314876c253cf19339f8567459bc85ae4adf7a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-21T06:50:20.711 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.9.service' 2024-09-21T06:50:20.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:20 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:20.284+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:20.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:20 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:20.284+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:20.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:20 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:20.284+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:20.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:20 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:20.284+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:20.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:20 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:20.284+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.409872+0000 front 2024-09-21T06:49:42.409992+0000 (oldest deadline 2024-09-21T06:50:05.910048+0000) 2024-09-21T06:50:20.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:20 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:20.284+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-09-21T06:49:51.811098+0000 front 2024-09-21T06:49:51.811251+0000 (oldest deadline 2024-09-21T06:50:14.111041+0000) 2024-09-21T06:50:20.751 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:50:20.751 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-21T06:50:20.751 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-21T06:50:20.751 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.10 2024-09-21T06:50:21.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:20 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:20.701+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:50:21.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:20 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:20.701+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:50:21.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:20 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:20.701+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:50:21.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:20 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:20.701+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:50:21.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:20 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:20.701+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:34.667785+0000 front 2024-09-21T06:49:34.667749+0000 (oldest deadline 2024-09-21T06:50:00.567875+0000) 2024-09-21T06:50:21.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:20 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:20.701+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.868894+0000 front 2024-09-21T06:49:42.868778+0000 (oldest deadline 2024-09-21T06:50:08.168688+0000) 2024-09-21T06:50:21.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:20 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:20.701+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-09-21T06:49:52.170471+0000 front 2024-09-21T06:49:52.170349+0000 (oldest deadline 2024-09-21T06:50:16.770511+0000) 2024-09-21T06:50:21.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:20 smithi100 systemd[1]: Stopping Ceph osd.10 for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:50:21.553 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:21 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:21.064+0000 7f32dfa58640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:50:21.554 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:21 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:21.064+0000 7f32dfa58640 -1 osd.10 89 *** Got signal Terminated *** 2024-09-21T06:50:21.554 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:21 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:21.064+0000 7f32dfa58640 -1 osd.10 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:50:21.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:21 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:21.243+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:21.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:21 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:21.243+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:21.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:21 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:21.243+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:21.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:21 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:21.243+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:21.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:21 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:21.243+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.409872+0000 front 2024-09-21T06:49:42.409992+0000 (oldest deadline 2024-09-21T06:50:05.910048+0000) 2024-09-21T06:50:21.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:21 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:21.243+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-09-21T06:49:51.811098+0000 front 2024-09-21T06:49:51.811251+0000 (oldest deadline 2024-09-21T06:50:14.111041+0000) 2024-09-21T06:50:22.053 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:21 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:21.670+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:50:22.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:21 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:21.670+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:50:22.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:21 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:21.670+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:50:22.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:21 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:21.670+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:50:22.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:21 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:21.670+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:34.667785+0000 front 2024-09-21T06:49:34.667749+0000 (oldest deadline 2024-09-21T06:50:00.567875+0000) 2024-09-21T06:50:22.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:21 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:21.670+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.868894+0000 front 2024-09-21T06:49:42.868778+0000 (oldest deadline 2024-09-21T06:50:08.168688+0000) 2024-09-21T06:50:22.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:21 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:21.670+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-09-21T06:49:52.170471+0000 front 2024-09-21T06:49:52.170349+0000 (oldest deadline 2024-09-21T06:50:16.770511+0000) 2024-09-21T06:50:22.482 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:22 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:22.220+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:22.482 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:22 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:22.220+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:22.482 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:22 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:22.220+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:22.482 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:22 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:22.220+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:22.482 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:22 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:22.220+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.409872+0000 front 2024-09-21T06:49:42.409992+0000 (oldest deadline 2024-09-21T06:50:05.910048+0000) 2024-09-21T06:50:22.482 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:22 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:22.220+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-09-21T06:49:51.811098+0000 front 2024-09-21T06:49:51.811251+0000 (oldest deadline 2024-09-21T06:50:14.111041+0000) 2024-09-21T06:50:23.053 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:22 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:22.664+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:50:23.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:22 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:22.664+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:50:23.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:22 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:22.664+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:50:23.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:22 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:22.664+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:50:23.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:22 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:22.664+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:34.667785+0000 front 2024-09-21T06:49:34.667749+0000 (oldest deadline 2024-09-21T06:50:00.567875+0000) 2024-09-21T06:50:23.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:22 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:22.664+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.868894+0000 front 2024-09-21T06:49:42.868778+0000 (oldest deadline 2024-09-21T06:50:08.168688+0000) 2024-09-21T06:50:23.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:22 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:22.664+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-09-21T06:49:52.170471+0000 front 2024-09-21T06:49:52.170349+0000 (oldest deadline 2024-09-21T06:50:16.770511+0000) 2024-09-21T06:50:23.515 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:23 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:23.258+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:23.515 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:23 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:23.258+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:23.515 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:23 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:23.258+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:23.515 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:23 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:23.258+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:23.515 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:23 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:23.258+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.409872+0000 front 2024-09-21T06:49:42.409992+0000 (oldest deadline 2024-09-21T06:50:05.910048+0000) 2024-09-21T06:50:23.515 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:23 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:23.258+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-09-21T06:49:51.811098+0000 front 2024-09-21T06:49:51.811251+0000 (oldest deadline 2024-09-21T06:50:14.111041+0000) 2024-09-21T06:50:23.515 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:23 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:23.258+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-09-21T06:49:59.412332+0000 front 2024-09-21T06:49:59.412203+0000 (oldest deadline 2024-09-21T06:50:22.312061+0000) 2024-09-21T06:50:24.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:23 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:23.707+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:50:24.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:23 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:23.707+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:50:24.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:23 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:23.707+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:50:24.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:23 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:23.707+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:50:24.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:23 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:23.707+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:34.667785+0000 front 2024-09-21T06:49:34.667749+0000 (oldest deadline 2024-09-21T06:50:00.567875+0000) 2024-09-21T06:50:24.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:23 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:23.707+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.868894+0000 front 2024-09-21T06:49:42.868778+0000 (oldest deadline 2024-09-21T06:50:08.168688+0000) 2024-09-21T06:50:24.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:23 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:23.707+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-09-21T06:49:52.170471+0000 front 2024-09-21T06:49:52.170349+0000 (oldest deadline 2024-09-21T06:50:16.770511+0000) 2024-09-21T06:50:24.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:24 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:24.288+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:24.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:24 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:24.288+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:24.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:24 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:24.288+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:24.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:24 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:24.288+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:24.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:24 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:24.288+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.409872+0000 front 2024-09-21T06:49:42.409992+0000 (oldest deadline 2024-09-21T06:50:05.910048+0000) 2024-09-21T06:50:24.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:24 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:24.288+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-09-21T06:49:51.811098+0000 front 2024-09-21T06:49:51.811251+0000 (oldest deadline 2024-09-21T06:50:14.111041+0000) 2024-09-21T06:50:24.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:24 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:24.288+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-09-21T06:49:59.412332+0000 front 2024-09-21T06:49:59.412203+0000 (oldest deadline 2024-09-21T06:50:22.312061+0000) 2024-09-21T06:50:25.053 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:24 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:24.687+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:50:25.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:24 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:24.687+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:50:25.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:24 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:24.687+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:50:25.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:24 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:24.687+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:50:25.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:24 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:24.687+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:34.667785+0000 front 2024-09-21T06:49:34.667749+0000 (oldest deadline 2024-09-21T06:50:00.567875+0000) 2024-09-21T06:50:25.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:24 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:24.687+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.868894+0000 front 2024-09-21T06:49:42.868778+0000 (oldest deadline 2024-09-21T06:50:08.168688+0000) 2024-09-21T06:50:25.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:24 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:24.687+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-09-21T06:49:52.170471+0000 front 2024-09-21T06:49:52.170349+0000 (oldest deadline 2024-09-21T06:50:16.770511+0000) 2024-09-21T06:50:25.734 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:25 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:25.273+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:25.734 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:25 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:25.273+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:25.734 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:25 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:25.273+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:25.734 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:25 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:25.273+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:25.734 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:25 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:25.273+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.409872+0000 front 2024-09-21T06:49:42.409992+0000 (oldest deadline 2024-09-21T06:50:05.910048+0000) 2024-09-21T06:50:25.734 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:25 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:25.273+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-09-21T06:49:51.811098+0000 front 2024-09-21T06:49:51.811251+0000 (oldest deadline 2024-09-21T06:50:14.111041+0000) 2024-09-21T06:50:25.734 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:25 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:25.273+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-09-21T06:49:59.412332+0000 front 2024-09-21T06:49:59.412203+0000 (oldest deadline 2024-09-21T06:50:22.312061+0000) 2024-09-21T06:50:26.053 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:25 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:25.642+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:00.461889+0000 front 2024-09-21T06:49:00.461952+0000 (oldest deadline 2024-09-21T06:49:25.761943+0000) 2024-09-21T06:50:26.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:25 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:25.642+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:10.963475+0000 front 2024-09-21T06:49:10.963363+0000 (oldest deadline 2024-09-21T06:49:34.963889+0000) 2024-09-21T06:50:26.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:25 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:25.642+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:19.065106+0000 front 2024-09-21T06:49:19.064872+0000 (oldest deadline 2024-09-21T06:49:42.564875+0000) 2024-09-21T06:50:26.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:25 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:25.642+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-21T06:49:24.866087+0000 front 2024-09-21T06:49:24.866141+0000 (oldest deadline 2024-09-21T06:49:51.866437+0000) 2024-09-21T06:50:26.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:25 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:25.642+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:34.667785+0000 front 2024-09-21T06:49:34.667749+0000 (oldest deadline 2024-09-21T06:50:00.567875+0000) 2024-09-21T06:50:26.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:25 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:25.642+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.868894+0000 front 2024-09-21T06:49:42.868778+0000 (oldest deadline 2024-09-21T06:50:08.168688+0000) 2024-09-21T06:50:26.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:25 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10[56219]: 2024-09-21T06:50:25.642+0000 7f32d803b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-09-21T06:49:52.170471+0000 front 2024-09-21T06:49:52.170349+0000 (oldest deadline 2024-09-21T06:50:16.770511+0000) 2024-09-21T06:50:26.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:26 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:26.234+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:26.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:26 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:26.234+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:26.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:26 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:26.234+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:26.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:26 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:26.234+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:26.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:26 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:26.234+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.409872+0000 front 2024-09-21T06:49:42.409992+0000 (oldest deadline 2024-09-21T06:50:05.910048+0000) 2024-09-21T06:50:26.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:26 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:26.234+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-09-21T06:49:51.811098+0000 front 2024-09-21T06:49:51.811251+0000 (oldest deadline 2024-09-21T06:50:14.111041+0000) 2024-09-21T06:50:26.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:26 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:26.234+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-09-21T06:49:59.412332+0000 front 2024-09-21T06:49:59.412203+0000 (oldest deadline 2024-09-21T06:50:22.312061+0000) 2024-09-21T06:50:26.791 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:26 smithi100 podman[65091]: 2024-09-21 06:50:26.518765644 +0000 UTC m=+5.575059621 container died 670a59e162e975bb327411686219eb6bf7f2562ec25d6e8022821902e6620c6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:50:27.053 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:26 smithi100 podman[65091]: 2024-09-21 06:50:26.889580151 +0000 UTC m=+5.945874118 container cleanup 670a59e162e975bb327411686219eb6bf7f2562ec25d6e8022821902e6620c6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:50:27.054 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:26 smithi100 bash[65091]: ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10 2024-09-21T06:50:27.450 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:27 smithi100 podman[65106]: 2024-09-21 06:50:27.132620991 +0000 UTC m=+0.608102548 container remove 670a59e162e975bb327411686219eb6bf7f2562ec25d6e8022821902e6620c6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:50:27.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:27 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:27.259+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:27.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:27 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:27.259+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:27.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:27 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:27.259+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:27.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:27 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:27.259+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:27.732 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:27 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:27.259+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.409872+0000 front 2024-09-21T06:49:42.409992+0000 (oldest deadline 2024-09-21T06:50:05.910048+0000) 2024-09-21T06:50:27.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:27 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:27.259+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-09-21T06:49:51.811098+0000 front 2024-09-21T06:49:51.811251+0000 (oldest deadline 2024-09-21T06:50:14.111041+0000) 2024-09-21T06:50:27.733 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:27 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:27.259+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-09-21T06:49:59.412332+0000 front 2024-09-21T06:49:59.412203+0000 (oldest deadline 2024-09-21T06:50:22.312061+0000) 2024-09-21T06:50:27.770 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:27 smithi100 podman[65194]: 2024-09-21 06:50:27.449898654 +0000 UTC m=+0.106738330 container create 0326ef8cefd927c3ff0fe5513c37b8be8915d1d0fd420e025033484661ebdb08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True) 2024-09-21T06:50:27.770 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:27 smithi100 podman[65194]: 2024-09-21 06:50:27.359015803 +0000 UTC m=+0.015855490 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:50:27.770 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:27 smithi100 podman[65194]: 2024-09-21 06:50:27.677325829 +0000 UTC m=+0.334165515 container init 0326ef8cefd927c3ff0fe5513c37b8be8915d1d0fd420e025033484661ebdb08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:50:27.770 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:27 smithi100 podman[65194]: 2024-09-21 06:50:27.680659074 +0000 UTC m=+0.337498754 container start 0326ef8cefd927c3ff0fe5513c37b8be8915d1d0fd420e025033484661ebdb08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10-deactivate, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:50:27.771 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:27 smithi100 podman[65194]: 2024-09-21 06:50:27.768909805 +0000 UTC m=+0.425749492 container attach 0326ef8cefd927c3ff0fe5513c37b8be8915d1d0fd420e025033484661ebdb08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10-deactivate, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-21T06:50:28.053 INFO:journalctl@ceph.osd.10.smithi100.stdout:Sep 21 06:50:27 smithi100 podman[65194]: 2024-09-21 06:50:27.866699232 +0000 UTC m=+0.523538914 container died 0326ef8cefd927c3ff0fe5513c37b8be8915d1d0fd420e025033484661ebdb08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-10-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:50:28.490 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.10.service' 2024-09-21T06:50:28.531 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:50:28.531 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-21T06:50:28.531 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-21T06:50:28.531 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.11 2024-09-21T06:50:28.560 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:28 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:28.259+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:28.560 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:28 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:28.259+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:28.560 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:28 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:28.259+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:28.560 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:28 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:28.259+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:28.560 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:28 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:28.259+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.409872+0000 front 2024-09-21T06:49:42.409992+0000 (oldest deadline 2024-09-21T06:50:05.910048+0000) 2024-09-21T06:50:28.560 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:28 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:28.259+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-09-21T06:49:51.811098+0000 front 2024-09-21T06:49:51.811251+0000 (oldest deadline 2024-09-21T06:50:14.111041+0000) 2024-09-21T06:50:28.560 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:28 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:28.259+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-09-21T06:49:59.412332+0000 front 2024-09-21T06:49:59.412203+0000 (oldest deadline 2024-09-21T06:50:22.312061+0000) 2024-09-21T06:50:28.857 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:28 smithi203 systemd[1]: Stopping Ceph osd.11 for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:50:28.857 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:28 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:28.855+0000 7f31029a2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:50:29.226 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:28 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:28.855+0000 7f31029a2640 -1 osd.11 89 *** Got signal Terminated *** 2024-09-21T06:50:29.226 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:28 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:28.855+0000 7f31029a2640 -1 osd.11 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:50:29.226 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:29 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:29.224+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:29.226 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:29 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:29.224+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:29.226 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:29 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:29.224+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:29.226 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:29 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:29.224+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:29.226 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:29 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:29.224+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.409872+0000 front 2024-09-21T06:49:42.409992+0000 (oldest deadline 2024-09-21T06:50:05.910048+0000) 2024-09-21T06:50:29.226 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:29 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:29.224+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-09-21T06:49:51.811098+0000 front 2024-09-21T06:49:51.811251+0000 (oldest deadline 2024-09-21T06:50:14.111041+0000) 2024-09-21T06:50:30.482 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:29 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:29.224+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-09-21T06:49:59.412332+0000 front 2024-09-21T06:49:59.412203+0000 (oldest deadline 2024-09-21T06:50:22.312061+0000) 2024-09-21T06:50:30.482 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:30 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:30.194+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:30.482 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:30 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:30.194+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:30.482 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:30 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:30.194+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:30.482 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:30 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:30.194+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:30.482 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:30 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:30.194+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.409872+0000 front 2024-09-21T06:49:42.409992+0000 (oldest deadline 2024-09-21T06:50:05.910048+0000) 2024-09-21T06:50:30.483 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:30 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:30.194+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-09-21T06:49:51.811098+0000 front 2024-09-21T06:49:51.811251+0000 (oldest deadline 2024-09-21T06:50:14.111041+0000) 2024-09-21T06:50:30.483 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:30 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:30.194+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-09-21T06:49:59.412332+0000 front 2024-09-21T06:49:59.412203+0000 (oldest deadline 2024-09-21T06:50:22.312061+0000) 2024-09-21T06:50:31.482 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:31 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:31.172+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:31.482 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:31 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:31.172+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:31.482 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:31 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:31.172+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:31.482 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:31 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:31.172+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:31.482 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:31 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:31.172+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.409872+0000 front 2024-09-21T06:49:42.409992+0000 (oldest deadline 2024-09-21T06:50:05.910048+0000) 2024-09-21T06:50:31.482 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:31 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:31.172+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-09-21T06:49:51.811098+0000 front 2024-09-21T06:49:51.811251+0000 (oldest deadline 2024-09-21T06:50:14.111041+0000) 2024-09-21T06:50:31.483 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:31 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:31.172+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-09-21T06:49:59.412332+0000 front 2024-09-21T06:49:59.412203+0000 (oldest deadline 2024-09-21T06:50:22.312061+0000) 2024-09-21T06:50:32.482 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:32 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:32.136+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:32.482 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:32 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:32.136+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:32.482 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:32 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:32.136+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:32.482 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:32 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:32.136+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:32.482 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:32 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:32.136+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.409872+0000 front 2024-09-21T06:49:42.409992+0000 (oldest deadline 2024-09-21T06:50:05.910048+0000) 2024-09-21T06:50:32.482 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:32 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:32.136+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-09-21T06:49:51.811098+0000 front 2024-09-21T06:49:51.811251+0000 (oldest deadline 2024-09-21T06:50:14.111041+0000) 2024-09-21T06:50:32.482 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:32 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:32.136+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-09-21T06:49:59.412332+0000 front 2024-09-21T06:49:59.412203+0000 (oldest deadline 2024-09-21T06:50:22.312061+0000) 2024-09-21T06:50:33.482 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:33 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:33.173+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-21T06:49:03.002782+0000 front 2024-09-21T06:49:03.002867+0000 (oldest deadline 2024-09-21T06:49:28.302882+0000) 2024-09-21T06:50:33.482 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:33 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:33.173+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-09-21T06:49:12.404090+0000 front 2024-09-21T06:49:12.403865+0000 (oldest deadline 2024-09-21T06:49:36.503941+0000) 2024-09-21T06:50:33.482 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:33 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:33.173+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T06:49:21.204765+0000 front 2024-09-21T06:49:21.204850+0000 (oldest deadline 2024-09-21T06:49:42.904822+0000) 2024-09-21T06:50:33.482 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:33 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:33.173+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-09-21T06:49:35.508534+0000 front 2024-09-21T06:49:35.508416+0000 (oldest deadline 2024-09-21T06:49:58.408451+0000) 2024-09-21T06:50:33.483 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:33 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:33.173+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T06:49:42.409872+0000 front 2024-09-21T06:49:42.409992+0000 (oldest deadline 2024-09-21T06:50:05.910048+0000) 2024-09-21T06:50:33.483 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:33 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:33.173+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-09-21T06:49:51.811098+0000 front 2024-09-21T06:49:51.811251+0000 (oldest deadline 2024-09-21T06:50:14.111041+0000) 2024-09-21T06:50:33.483 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:33 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11[56998]: 2024-09-21T06:50:33.173+0000 7f30faf85640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-09-21T06:49:59.412332+0000 front 2024-09-21T06:49:59.412203+0000 (oldest deadline 2024-09-21T06:50:22.312061+0000) 2024-09-21T06:50:34.676 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:34 smithi203 podman[65811]: 2024-09-21 06:50:34.394049644 +0000 UTC m=+5.659424006 container died ff44622e2f6d61807f82d3bdf004b0b9be757fcf856c5dbc2b3deaa84b632c07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731) 2024-09-21T06:50:34.977 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:34 smithi203 podman[65811]: 2024-09-21 06:50:34.766282889 +0000 UTC m=+6.031657242 container cleanup ff44622e2f6d61807f82d3bdf004b0b9be757fcf856c5dbc2b3deaa84b632c07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-21T06:50:34.977 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:34 smithi203 bash[65811]: ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11 2024-09-21T06:50:35.232 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:34 smithi203 podman[65827]: 2024-09-21 06:50:34.975841728 +0000 UTC m=+0.582737606 container remove ff44622e2f6d61807f82d3bdf004b0b9be757fcf856c5dbc2b3deaa84b632c07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-09-21T06:50:35.605 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:35 smithi203 podman[65917]: 2024-09-21 06:50:35.19106995 +0000 UTC m=+0.016083029 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:50:35.605 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:35 smithi203 podman[65917]: 2024-09-21 06:50:35.326979248 +0000 UTC m=+0.151992357 container create b5e38c6a02ce68d2ae2cd3c4fd7badcd4ed89de4903447dbe1a0f4cc439c191a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS) 2024-09-21T06:50:35.606 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:35 smithi203 podman[65917]: 2024-09-21 06:50:35.529317251 +0000 UTC m=+0.354330317 container init b5e38c6a02ce68d2ae2cd3c4fd7badcd4ed89de4903447dbe1a0f4cc439c191a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11-deactivate, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-21T06:50:35.606 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:35 smithi203 podman[65917]: 2024-09-21 06:50:35.532655832 +0000 UTC m=+0.357668906 container start b5e38c6a02ce68d2ae2cd3c4fd7badcd4ed89de4903447dbe1a0f4cc439c191a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-21T06:50:35.606 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:35 smithi203 podman[65917]: 2024-09-21 06:50:35.604192461 +0000 UTC m=+0.429205532 container attach b5e38c6a02ce68d2ae2cd3c4fd7badcd4ed89de4903447dbe1a0f4cc439c191a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-21T06:50:35.982 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 06:50:35 smithi203 podman[65917]: 2024-09-21 06:50:35.72477023 +0000 UTC m=+0.549783307 container died b5e38c6a02ce68d2ae2cd3c4fd7badcd4ed89de4903447dbe1a0f4cc439c191a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-osd-11-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:50:36.312 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@osd.11.service' 2024-09-21T06:50:36.392 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:50:36.393 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-21T06:50:36.393 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-21T06:50:36.393 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.a 2024-09-21T06:50:36.699 INFO:journalctl@ceph.mds.a.smithi067.stdout:Sep 21 06:50:36 smithi067 systemd[1]: Stopping Ceph mds.a for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:50:37.042 INFO:journalctl@ceph.mds.a.smithi067.stdout:Sep 21 06:50:36 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-a[72125]: 2024-09-21T06:50:36.696+0000 7f926cf06640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:50:37.042 INFO:journalctl@ceph.mds.a.smithi067.stdout:Sep 21 06:50:36 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-a[72125]: 2024-09-21T06:50:36.696+0000 7f926cf06640 -1 mds.a *** got signal Terminated *** 2024-09-21T06:50:39.748 INFO:journalctl@ceph.mds.a.smithi067.stdout:Sep 21 06:50:39 smithi067 podman[100648]: 2024-09-21 06:50:39.473014974 +0000 UTC m=+2.878644063 container died 3ecf1f8453a69c9ee050d25ff411b4406bf4410e5593f853196fab42396e7e2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True) 2024-09-21T06:50:40.041 INFO:journalctl@ceph.mds.a.smithi067.stdout:Sep 21 06:50:39 smithi067 podman[100648]: 2024-09-21 06:50:39.829827503 +0000 UTC m=+3.235456581 container cleanup 3ecf1f8453a69c9ee050d25ff411b4406bf4410e5593f853196fab42396e7e2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-09-21T06:50:40.042 INFO:journalctl@ceph.mds.a.smithi067.stdout:Sep 21 06:50:39 smithi067 bash[100648]: ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-a 2024-09-21T06:50:40.270 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.a.service' 2024-09-21T06:50:40.309 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:50:40.309 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-21T06:50:40.309 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-21T06:50:40.310 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.d 2024-09-21T06:50:40.792 INFO:journalctl@ceph.mds.d.smithi067.stdout:Sep 21 06:50:40 smithi067 systemd[1]: Stopping Ceph mds.d for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:50:40.792 INFO:journalctl@ceph.mds.d.smithi067.stdout:Sep 21 06:50:40 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-d[71011]: 2024-09-21T06:50:40.632+0000 7f6fb8bb3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:50:40.792 INFO:journalctl@ceph.mds.d.smithi067.stdout:Sep 21 06:50:40 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-d[71011]: 2024-09-21T06:50:40.632+0000 7f6fb8bb3640 -1 mds.d *** got signal Terminated *** 2024-09-21T06:50:45.010 INFO:journalctl@ceph.mds.d.smithi067.stdout:Sep 21 06:50:44 smithi067 podman[100782]: 2024-09-21 06:50:44.646503972 +0000 UTC m=+4.113885675 container died 4744b69f185ddbab1717dfe553b581b149f39739ad2bc775b305094d002d87f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:50:45.292 INFO:journalctl@ceph.mds.d.smithi067.stdout:Sep 21 06:50:45 smithi067 podman[100782]: 2024-09-21 06:50:45.10858692 +0000 UTC m=+4.575968610 container cleanup 4744b69f185ddbab1717dfe553b581b149f39739ad2bc775b305094d002d87f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-d, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:50:45.292 INFO:journalctl@ceph.mds.d.smithi067.stdout:Sep 21 06:50:45 smithi067 bash[100782]: ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-d 2024-09-21T06:50:45.552 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.d.service' 2024-09-21T06:50:45.587 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:50:45.587 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-21T06:50:45.587 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-21T06:50:45.588 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.g 2024-09-21T06:50:45.908 INFO:journalctl@ceph.mds.g.smithi067.stdout:Sep 21 06:50:45 smithi067 systemd[1]: Stopping Ceph mds.g for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:50:46.292 INFO:journalctl@ceph.mds.g.smithi067.stdout:Sep 21 06:50:45 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-g[71782]: 2024-09-21T06:50:45.905+0000 7fb30efe3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:50:46.292 INFO:journalctl@ceph.mds.g.smithi067.stdout:Sep 21 06:50:45 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-g[71782]: 2024-09-21T06:50:45.905+0000 7fb30efe3640 -1 mds.g *** got signal Terminated *** 2024-09-21T06:50:48.600 INFO:journalctl@ceph.mds.g.smithi067.stdout:Sep 21 06:50:48 smithi067 podman[100904]: 2024-09-21 06:50:48.298507504 +0000 UTC m=+2.490212278 container died c8ff47271516f684a14c25f04356fc32bf089ab337bad7744994b9798699967f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-g, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-21T06:50:49.042 INFO:journalctl@ceph.mds.g.smithi067.stdout:Sep 21 06:50:48 smithi067 podman[100904]: 2024-09-21 06:50:48.814901614 +0000 UTC m=+3.006606377 container cleanup c8ff47271516f684a14c25f04356fc32bf089ab337bad7744994b9798699967f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-21T06:50:49.042 INFO:journalctl@ceph.mds.g.smithi067.stdout:Sep 21 06:50:48 smithi067 bash[100904]: ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-g 2024-09-21T06:50:49.298 INFO:journalctl@ceph.mds.g.smithi067.stdout:Sep 21 06:50:49 smithi067 podman[100916]: 2024-09-21 06:50:49.041003866 +0000 UTC m=+0.738422636 container remove c8ff47271516f684a14c25f04356fc32bf089ab337bad7744994b9798699967f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-g, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-21T06:50:49.302 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.g.service' 2024-09-21T06:50:49.335 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:50:49.336 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-21T06:50:49.336 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-21T06:50:49.336 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.j 2024-09-21T06:50:49.686 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 21 06:50:49 smithi067 systemd[1]: Stopping Ceph mds.j for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:50:50.041 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 21 06:50:49 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-j[70648]: 2024-09-21T06:50:49.684+0000 7fe21e5e1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:50:50.042 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 21 06:50:49 smithi067 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-j[70648]: 2024-09-21T06:50:49.684+0000 7fe21e5e1640 -1 mds.j *** got signal Terminated *** 2024-09-21T06:50:53.292 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 21 06:50:53 smithi067 podman[101026]: 2024-09-21 06:50:53.027936074 +0000 UTC m=+3.467573613 container died 97060575a224078261c95eda1d9c6fc9690ae15c76f56c5dd2a3e356321e3e1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-j, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-21T06:50:53.754 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 21 06:50:53 smithi067 podman[101026]: 2024-09-21 06:50:53.551797264 +0000 UTC m=+3.991434801 container cleanup 97060575a224078261c95eda1d9c6fc9690ae15c76f56c5dd2a3e356321e3e1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-j, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:50:53.754 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 21 06:50:53 smithi067 bash[101026]: ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-j 2024-09-21T06:50:53.981 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.j.service' 2024-09-21T06:50:54.004 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 21 06:50:53 smithi067 podman[101037]: 2024-09-21 06:50:53.752991944 +0000 UTC m=+0.721378358 container remove 97060575a224078261c95eda1d9c6fc9690ae15c76f56c5dd2a3e356321e3e1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-j, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-21T06:50:54.005 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 21 06:50:53 smithi067 systemd[1]: ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.j.service: Deactivated successfully. 2024-09-21T06:50:54.005 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 21 06:50:53 smithi067 systemd[1]: Stopped Ceph mds.j for 2a791768-77e2-11ef-baf5-efdc52797490. 2024-09-21T06:50:54.005 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 21 06:50:53 smithi067 systemd[1]: ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.j.service: Consumed 1.085s CPU time. 2024-09-21T06:50:54.014 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:50:54.015 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-21T06:50:54.015 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-21T06:50:54.015 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.b 2024-09-21T06:50:54.303 INFO:journalctl@ceph.mds.b.smithi100.stdout:Sep 21 06:50:54 smithi100 systemd[1]: Stopping Ceph mds.b for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:50:54.562 INFO:journalctl@ceph.mds.b.smithi100.stdout:Sep 21 06:50:54 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-b[61079]: 2024-09-21T06:50:54.305+0000 7f1bc5199640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:50:54.562 INFO:journalctl@ceph.mds.b.smithi100.stdout:Sep 21 06:50:54 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-b[61079]: 2024-09-21T06:50:54.305+0000 7f1bc5199640 -1 mds.b *** got signal Terminated *** 2024-09-21T06:50:56.299 INFO:journalctl@ceph.mds.b.smithi100.stdout:Sep 21 06:50:55 smithi100 podman[65297]: 2024-09-21 06:50:55.986224868 +0000 UTC m=+1.782468952 container died 1e4cb69455a6927d27b9cb8659c75a4490dd314e090e8af084ddaaca655cb1b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:50:56.554 INFO:journalctl@ceph.mds.b.smithi100.stdout:Sep 21 06:50:56 smithi100 podman[65297]: 2024-09-21 06:50:56.39730855 +0000 UTC m=+2.193552622 container cleanup 1e4cb69455a6927d27b9cb8659c75a4490dd314e090e8af084ddaaca655cb1b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-b, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-21T06:50:56.554 INFO:journalctl@ceph.mds.b.smithi100.stdout:Sep 21 06:50:56 smithi100 bash[65297]: ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-b 2024-09-21T06:50:56.744 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.b.service' 2024-09-21T06:50:56.780 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:50:56.781 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-21T06:50:56.781 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-21T06:50:56.781 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.e 2024-09-21T06:50:57.053 INFO:journalctl@ceph.mds.e.smithi100.stdout:Sep 21 06:50:56 smithi100 systemd[1]: Stopping Ceph mds.e for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:50:57.553 INFO:journalctl@ceph.mds.e.smithi100.stdout:Sep 21 06:50:57 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-e[60530]: 2024-09-21T06:50:57.117+0000 7f1dc0968640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:50:57.554 INFO:journalctl@ceph.mds.e.smithi100.stdout:Sep 21 06:50:57 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-e[60530]: 2024-09-21T06:50:57.117+0000 7f1dc0968640 -1 mds.e *** got signal Terminated *** 2024-09-21T06:50:59.736 INFO:journalctl@ceph.mds.e.smithi100.stdout:Sep 21 06:50:59 smithi100 podman[65430]: 2024-09-21 06:50:59.432938686 +0000 UTC m=+2.432860568 container died 21813274ce3b7459492d7567515ebeaf51d3705b2577cd7a60a6f34e8e6515d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-e, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:51:00.028 INFO:journalctl@ceph.mds.e.smithi100.stdout:Sep 21 06:50:59 smithi100 podman[65430]: 2024-09-21 06:50:59.834501607 +0000 UTC m=+2.834423489 container cleanup 21813274ce3b7459492d7567515ebeaf51d3705b2577cd7a60a6f34e8e6515d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-21T06:51:00.028 INFO:journalctl@ceph.mds.e.smithi100.stdout:Sep 21 06:50:59 smithi100 bash[65430]: ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-e 2024-09-21T06:51:00.165 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.e.service' 2024-09-21T06:51:00.198 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:51:00.198 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-21T06:51:00.198 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-21T06:51:00.198 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.h 2024-09-21T06:51:00.303 INFO:journalctl@ceph.mds.h.smithi100.stdout:Sep 21 06:51:00 smithi100 systemd[1]: Stopping Ceph mds.h for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:51:00.803 INFO:journalctl@ceph.mds.h.smithi100.stdout:Sep 21 06:51:00 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-h[60800]: 2024-09-21T06:51:00.496+0000 7f663e514640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:51:00.803 INFO:journalctl@ceph.mds.h.smithi100.stdout:Sep 21 06:51:00 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-h[60800]: 2024-09-21T06:51:00.496+0000 7f663e514640 -1 mds.h *** got signal Terminated *** 2024-09-21T06:51:04.248 INFO:journalctl@ceph.mds.h.smithi100.stdout:Sep 21 06:51:03 smithi100 podman[65548]: 2024-09-21 06:51:03.992033647 +0000 UTC m=+3.579702995 container died fcaaf2d7561b4d11ce076fbc61fda482b1ce84a67750d32d22c8e671b8d6d257 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-h, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-21T06:51:04.553 INFO:journalctl@ceph.mds.h.smithi100.stdout:Sep 21 06:51:04 smithi100 podman[65548]: 2024-09-21 06:51:04.346340625 +0000 UTC m=+3.934009972 container cleanup fcaaf2d7561b4d11ce076fbc61fda482b1ce84a67750d32d22c8e671b8d6d257 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-h, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True) 2024-09-21T06:51:04.554 INFO:journalctl@ceph.mds.h.smithi100.stdout:Sep 21 06:51:04 smithi100 bash[65548]: ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-h 2024-09-21T06:51:04.772 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.h.service' 2024-09-21T06:51:04.804 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:51:04.804 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-21T06:51:04.804 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-21T06:51:04.804 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.k 2024-09-21T06:51:05.034 INFO:journalctl@ceph.mds.k.smithi100.stdout:Sep 21 06:51:04 smithi100 systemd[1]: Stopping Ceph mds.k for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:51:05.303 INFO:journalctl@ceph.mds.k.smithi100.stdout:Sep 21 06:51:05 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-k[60256]: 2024-09-21T06:51:05.116+0000 7f025139c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:51:05.303 INFO:journalctl@ceph.mds.k.smithi100.stdout:Sep 21 06:51:05 smithi100 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-k[60256]: 2024-09-21T06:51:05.116+0000 7f025139c640 -1 mds.k *** got signal Terminated *** 2024-09-21T06:51:09.384 INFO:journalctl@ceph.mds.k.smithi100.stdout:Sep 21 06:51:09 smithi100 podman[65670]: 2024-09-21 06:51:09.122395334 +0000 UTC m=+4.106533152 container died 9195a712aedbacd2add04f89e453bc318c16e836133c7bd5b5727729bfbefb73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-21T06:51:09.718 INFO:journalctl@ceph.mds.k.smithi100.stdout:Sep 21 06:51:09 smithi100 podman[65670]: 2024-09-21 06:51:09.483484648 +0000 UTC m=+4.467622469 container cleanup 9195a712aedbacd2add04f89e453bc318c16e836133c7bd5b5727729bfbefb73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-k, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-21T06:51:09.718 INFO:journalctl@ceph.mds.k.smithi100.stdout:Sep 21 06:51:09 smithi100 bash[65670]: ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-k 2024-09-21T06:51:09.849 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.k.service' 2024-09-21T06:51:09.880 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:51:09.880 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-21T06:51:09.880 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-21T06:51:09.880 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.c 2024-09-21T06:51:10.167 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 06:51:09 smithi203 systemd[1]: Stopping Ceph mds.c for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:51:10.482 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 06:51:10 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-c[61669]: 2024-09-21T06:51:10.165+0000 7f0a5c32a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:51:10.482 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 06:51:10 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-c[61669]: 2024-09-21T06:51:10.165+0000 7f0a5c32a640 -1 mds.c *** got signal Terminated *** 2024-09-21T06:51:12.851 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 06:51:12 smithi203 podman[66021]: 2024-09-21 06:51:12.575128293 +0000 UTC m=+2.505491900 container died f580d722c33d324aebff6f345c0278e5ea62c8b6bfd60553cb71f6b350436443 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-c, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-21T06:51:13.210 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 06:51:12 smithi203 podman[66021]: 2024-09-21 06:51:12.924397436 +0000 UTC m=+2.854761040 container cleanup f580d722c33d324aebff6f345c0278e5ea62c8b6bfd60553cb71f6b350436443 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-c, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-21T06:51:13.210 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 06:51:12 smithi203 bash[66021]: ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-c 2024-09-21T06:51:13.368 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.c.service' 2024-09-21T06:51:13.403 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:51:13.403 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-21T06:51:13.404 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-21T06:51:13.404 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.f 2024-09-21T06:51:13.722 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 06:51:13 smithi203 systemd[1]: Stopping Ceph mds.f for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:51:13.982 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 06:51:13 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-f[61398]: 2024-09-21T06:51:13.719+0000 7f735677a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:51:13.982 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 06:51:13 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-f[61398]: 2024-09-21T06:51:13.719+0000 7f735677a640 -1 mds.f *** got signal Terminated *** 2024-09-21T06:51:17.982 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 06:51:17 smithi203 podman[66155]: 2024-09-21 06:51:17.726180875 +0000 UTC m=+4.101876565 container died 34dd868b4ddb350322e752d8e712b9d4e3b152af18f3047372eaf504725090ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-f, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:51:18.355 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 06:51:18 smithi203 podman[66155]: 2024-09-21 06:51:18.100710174 +0000 UTC m=+4.476405855 container cleanup 34dd868b4ddb350322e752d8e712b9d4e3b152af18f3047372eaf504725090ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-f, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:51:18.355 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 06:51:18 smithi203 bash[66155]: ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-f 2024-09-21T06:51:18.498 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.f.service' 2024-09-21T06:51:18.531 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:51:18.532 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-21T06:51:18.532 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-21T06:51:18.532 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.i 2024-09-21T06:51:18.982 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 06:51:18 smithi203 systemd[1]: Stopping Ceph mds.i for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:51:18.982 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 06:51:18 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-i[61941]: 2024-09-21T06:51:18.848+0000 7f1f87306640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:51:18.982 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 06:51:18 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-i[61941]: 2024-09-21T06:51:18.848+0000 7f1f87306640 -1 mds.i *** got signal Terminated *** 2024-09-21T06:51:22.009 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 06:51:21 smithi203 podman[66274]: 2024-09-21 06:51:21.735729743 +0000 UTC m=+2.990145128 container died 440a27cc8a72963ef3f84943f4fc4e9591d4a0d47a29b14e28ddde07ee9b93f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-i, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-21T06:51:22.376 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 06:51:22 smithi203 podman[66274]: 2024-09-21 06:51:22.099118553 +0000 UTC m=+3.353533922 container cleanup 440a27cc8a72963ef3f84943f4fc4e9591d4a0d47a29b14e28ddde07ee9b93f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-i, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:51:22.377 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 06:51:22 smithi203 bash[66274]: ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-i 2024-09-21T06:51:22.531 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.i.service' 2024-09-21T06:51:22.563 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:51:22.564 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-21T06:51:22.564 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-21T06:51:22.564 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.l 2024-09-21T06:51:22.982 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 21 06:51:22 smithi203 systemd[1]: Stopping Ceph mds.l for 2a791768-77e2-11ef-baf5-efdc52797490... 2024-09-21T06:51:22.982 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 21 06:51:22 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-l[61130]: 2024-09-21T06:51:22.877+0000 7f5f2e15b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:51:22.982 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 21 06:51:22 smithi203 ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-l[61130]: 2024-09-21T06:51:22.877+0000 7f5f2e15b640 -1 mds.l *** got signal Terminated *** 2024-09-21T06:51:27.163 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 21 06:51:26 smithi203 podman[66395]: 2024-09-21 06:51:26.891979012 +0000 UTC m=+4.113082891 container died 1e20586e4e96774ac74be17325b5a100c14135cf805d02ee4433dee6ffad55a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-l, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True) 2024-09-21T06:51:27.482 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 21 06:51:27 smithi203 podman[66395]: 2024-09-21 06:51:27.253316145 +0000 UTC m=+4.474420026 container cleanup 1e20586e4e96774ac74be17325b5a100c14135cf805d02ee4433dee6ffad55a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-l, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:51:27.482 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 21 06:51:27 smithi203 bash[66395]: ceph-2a791768-77e2-11ef-baf5-efdc52797490-mds-l 2024-09-21T06:51:27.676 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a791768-77e2-11ef-baf5-efdc52797490@mds.l.service' 2024-09-21T06:51:27.707 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:51:27.707 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-21T06:51:27.707 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2a791768-77e2-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T06:51:27.919 INFO:teuthology.orchestra.run.smithi067.stdout:Deleting cluster with fsid: 2a791768-77e2-11ef-baf5-efdc52797490 2024-09-21T06:51:30.721 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2a791768-77e2-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T06:51:30.928 INFO:teuthology.orchestra.run.smithi100.stdout:Deleting cluster with fsid: 2a791768-77e2-11ef-baf5-efdc52797490 2024-09-21T06:51:33.341 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2a791768-77e2-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T06:51:33.542 INFO:teuthology.orchestra.run.smithi203.stdout:Deleting cluster with fsid: 2a791768-77e2-11ef-baf5-efdc52797490 2024-09-21T06:51:35.894 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:51:35.928 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:51:35.961 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:51:35.989 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-21T06:51:35.992 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913544/remote/smithi067/crash 2024-09-21T06:51:35.993 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/crash -- . 2024-09-21T06:51:36.025 INFO:teuthology.orchestra.run.smithi067.stderr:tar: /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T06:51:36.025 INFO:teuthology.orchestra.run.smithi067.stderr:tar: Error is not recoverable: exiting now 2024-09-21T06:51:36.028 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913544/remote/smithi100/crash 2024-09-21T06:51:36.029 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/crash -- . 2024-09-21T06:51:36.084 INFO:teuthology.orchestra.run.smithi100.stderr:tar: /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T06:51:36.085 INFO:teuthology.orchestra.run.smithi100.stderr:tar: Error is not recoverable: exiting now 2024-09-21T06:51:36.086 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913544/remote/smithi203/crash 2024-09-21T06:51:36.087 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/crash -- . 2024-09-21T06:51:36.152 INFO:teuthology.orchestra.run.smithi203.stderr:tar: /var/lib/ceph/2a791768-77e2-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T06:51:36.152 INFO:teuthology.orchestra.run.smithi203.stderr:tar: Error is not recoverable: exiting now 2024-09-21T06:51:36.153 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-21T06:51:36.154 DEBUG:teuthology.orchestra.run.smithi067:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-21T06:51:36.250 INFO:tasks.cephadm:Compressing logs... 2024-09-21T06:51:36.250 DEBUG:teuthology.orchestra.run.smithi067:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T06:51:36.252 DEBUG:teuthology.orchestra.run.smithi100:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T06:51:36.255 DEBUG:teuthology.orchestra.run.smithi203:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T06:51:36.279 INFO:teuthology.orchestra.run.smithi203.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T06:51:36.279 INFO:teuthology.orchestra.run.smithi203.stderr:: No such file or directory 2024-09-21T06:51:36.279 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T06:51:36.283 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.i.log 2024-09-21T06:51:36.283 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-volume.log: /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mon.c.log 2024-09-21T06:51:36.283 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T06:51:36.283 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.11.log 2024-09-21T06:51:36.283 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mon.c.log: /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.2.log 2024-09-21T06:51:36.283 INFO:teuthology.orchestra.run.smithi203.stderr: 82.2% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T06:51:36.283 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.5.log 2024-09-21T06:51:36.283 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph.log 2024-09-21T06:51:36.283 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.l.log 2024-09-21T06:51:36.283 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph.log: 90.3% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T06:51:36.285 INFO:teuthology.orchestra.run.smithi100.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.10.log 2024-09-21T06:51:36.285 INFO:teuthology.orchestra.run.smithi100.stderr:: No such file or directory 2024-09-21T06:51:36.285 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.1.log 2024-09-21T06:51:36.285 INFO:teuthology.orchestra.run.smithi100.stderr:gzip/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.10.log: -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.7.log 2024-09-21T06:51:36.285 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T06:51:36.285 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.b.log 2024-09-21T06:51:36.285 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.k.log 2024-09-21T06:51:36.285 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-volume.log: /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.4.log 2024-09-21T06:51:36.288 INFO:teuthology.orchestra.run.smithi203.stderr: 87.0% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T06:51:36.288 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.8.log 2024-09-21T06:51:36.288 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mgr.z.log 2024-09-21T06:51:36.288 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.8.log: 95.0% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.i.log.gz 2024-09-21T06:51:36.288 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.c.log 2024-09-21T06:51:36.289 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T06:51:36.289 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T06:51:36.289 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph.audit.log: 90.3% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T06:51:36.290 INFO:teuthology.orchestra.run.smithi067.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-21T06:51:36.290 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-21T06:51:36.290 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.0.log 2024-09-21T06:51:36.290 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.9.log 2024-09-21T06:51:36.290 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T06:51:36.294 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T06:51:36.294 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.a.log 2024-09-21T06:51:36.294 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T06:51:36.294 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mon.a.log 2024-09-21T06:51:36.294 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.g.log 2024-09-21T06:51:36.294 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.6.log 2024-09-21T06:51:36.294 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph.log 2024-09-21T06:51:36.294 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.d.log 2024-09-21T06:51:36.294 INFO:teuthology.orchestra.run.smithi067.stderr:gzip/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph.log: -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mgr.x.log 2024-09-21T06:51:36.294 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.j.log 2024-09-21T06:51:36.294 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.3.log 2024-09-21T06:51:36.295 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-09-21T06:51:36.295 INFO:teuthology.orchestra.run.smithi100.stderr: 95.0% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.k.log.gz 2024-09-21T06:51:36.297 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T06:51:36.297 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.0.log: /var/log/ceph/cephadm.log: 87.0% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T06:51:36.297 INFO:teuthology.orchestra.run.smithi067.stderr: 95.1% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.a.log.gz 2024-09-21T06:51:36.298 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mgr.z.log: 90.9% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T06:51:36.298 INFO:teuthology.orchestra.run.smithi203.stderr: 94.9% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.l.log.gz 2024-09-21T06:51:36.298 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.f.log 2024-09-21T06:51:36.298 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T06:51:36.298 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.f.log: 92.7% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mgr.z.log.gz 2024-09-21T06:51:36.298 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/cephadm.log: 92.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T06:51:36.299 INFO:teuthology.orchestra.run.smithi067.stderr: 95.1% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.g.log.gz 2024-09-21T06:51:36.299 INFO:teuthology.orchestra.run.smithi067.stderr: 95.1% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.d.log.gz 2024-09-21T06:51:36.300 INFO:teuthology.orchestra.run.smithi100.stderr: 90.9% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T06:51:36.300 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mgr.y.log 2024-09-21T06:51:36.300 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph.log 2024-09-21T06:51:36.300 INFO:teuthology.orchestra.run.smithi100.stderr: 82.3% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T06:51:36.300 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.h.log 2024-09-21T06:51:36.300 INFO:teuthology.orchestra.run.smithi203.stderr: 94.9% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.c.log.gz 2024-09-21T06:51:36.301 INFO:teuthology.orchestra.run.smithi203.stderr: 95.1% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.f.log.gz 2024-09-21T06:51:36.303 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mon.b.log 2024-09-21T06:51:36.303 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph.log: /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.h.log: 87.0% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T06:51:36.303 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.e.log 2024-09-21T06:51:36.307 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mon.b.log: 92.5% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mgr.y.log.gz 2024-09-21T06:51:36.307 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T06:51:36.315 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.e.log: 95.0% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.e.log.gz 2024-09-21T06:51:36.317 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/cephadm.log: 91.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T06:51:36.320 INFO:teuthology.orchestra.run.smithi067.stderr: 90.1% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T06:51:36.322 INFO:teuthology.orchestra.run.smithi067.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-21T06:51:36.427 INFO:teuthology.orchestra.run.smithi067.stderr: 82.3% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T06:51:36.430 INFO:teuthology.orchestra.run.smithi100.stderr: 93.6% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.h.log.gz 2024-09-21T06:51:36.569 INFO:teuthology.orchestra.run.smithi203.stderr: 91.8% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mon.c.log.gz 2024-09-21T06:51:36.635 INFO:teuthology.orchestra.run.smithi067.stderr: 93.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T06:51:36.680 INFO:teuthology.orchestra.run.smithi067.stderr: 90.9% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T06:51:36.715 INFO:teuthology.orchestra.run.smithi100.stderr: 91.7% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mon.b.log.gz 2024-09-21T06:51:36.833 INFO:teuthology.orchestra.run.smithi100.stderr: 93.8% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.b.log.gz 2024-09-21T06:51:37.129 INFO:teuthology.orchestra.run.smithi067.stderr: 95.1% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mds.j.log.gz 2024-09-21T06:51:38.966 INFO:teuthology.orchestra.run.smithi067.stderr: 90.4% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mgr.x.log.gz 2024-09-21T06:51:42.280 INFO:teuthology.orchestra.run.smithi067.stderr: 90.3% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-mon.a.log.gz 2024-09-21T06:51:59.674 INFO:teuthology.orchestra.run.smithi100.stderr: 91.3% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.10.log.gz 2024-09-21T06:52:03.585 INFO:teuthology.orchestra.run.smithi203.stderr: 90.9% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.8.log.gz 2024-09-21T06:52:14.579 INFO:teuthology.orchestra.run.smithi203.stderr: 90.7% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.11.log.gz 2024-09-21T06:52:17.084 INFO:teuthology.orchestra.run.smithi100.stderr: 90.7% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.1.log.gz 2024-09-21T06:52:21.239 INFO:teuthology.orchestra.run.smithi100.stderr: 90.8% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.7.log.gz 2024-09-21T06:52:23.337 INFO:teuthology.orchestra.run.smithi203.stderr: 90.7% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.2.log.gz 2024-09-21T06:52:37.743 INFO:teuthology.orchestra.run.smithi100.stderr: 90.7% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.4.log.gz 2024-09-21T06:52:37.745 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-09-21T06:52:37.745 INFO:teuthology.orchestra.run.smithi100.stderr:real 1m1.475s 2024-09-21T06:52:37.745 INFO:teuthology.orchestra.run.smithi100.stderr:user 2m46.145s 2024-09-21T06:52:37.745 INFO:teuthology.orchestra.run.smithi100.stderr:sys 0m5.198s 2024-09-21T06:52:40.757 INFO:teuthology.orchestra.run.smithi203.stderr: 90.5% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.5.log.gz 2024-09-21T06:52:40.760 INFO:teuthology.orchestra.run.smithi203.stderr: 2024-09-21T06:52:40.760 INFO:teuthology.orchestra.run.smithi203.stderr:real 1m4.489s 2024-09-21T06:52:40.760 INFO:teuthology.orchestra.run.smithi203.stderr:user 2m51.567s 2024-09-21T06:52:40.760 INFO:teuthology.orchestra.run.smithi203.stderr:sys 0m5.328s 2024-09-21T06:52:42.752 INFO:teuthology.orchestra.run.smithi067.stderr: 90.6% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.0.log.gz 2024-09-21T06:53:12.239 INFO:teuthology.orchestra.run.smithi067.stderr: 90.5% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.3.log.gz 2024-09-21T06:53:21.825 INFO:teuthology.orchestra.run.smithi067.stderr: 90.5% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.6.log.gz 2024-09-21T06:55:04.318 INFO:teuthology.orchestra.run.smithi067.stderr: 90.5% -- replaced with /var/log/ceph/2a791768-77e2-11ef-baf5-efdc52797490/ceph-osd.9.log.gz 2024-09-21T06:55:27.197 INFO:teuthology.orchestra.run.smithi067.stderr: 94.5% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-09-21T06:55:27.199 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-09-21T06:55:27.199 INFO:teuthology.orchestra.run.smithi067.stderr:real 3m50.933s 2024-09-21T06:55:27.199 INFO:teuthology.orchestra.run.smithi067.stderr:user 4m38.114s 2024-09-21T06:55:27.199 INFO:teuthology.orchestra.run.smithi067.stderr:sys 0m10.324s 2024-09-21T06:55:27.199 INFO:tasks.cephadm:Archiving logs... 2024-09-21T06:55:27.199 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913544/remote/smithi067/log 2024-09-21T06:55:27.201 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T06:56:11.212 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913544/remote/smithi100/log 2024-09-21T06:56:11.213 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T06:56:39.476 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913544/remote/smithi203/log 2024-09-21T06:56:39.477 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T06:57:09.525 INFO:tasks.cephadm:Removing cluster... 2024-09-21T06:57:09.525 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2a791768-77e2-11ef-baf5-efdc52797490 --force 2024-09-21T06:57:09.742 INFO:teuthology.orchestra.run.smithi067.stdout:Deleting cluster with fsid: 2a791768-77e2-11ef-baf5-efdc52797490 2024-09-21T06:57:10.287 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2a791768-77e2-11ef-baf5-efdc52797490 --force 2024-09-21T06:57:10.489 INFO:teuthology.orchestra.run.smithi100.stdout:Deleting cluster with fsid: 2a791768-77e2-11ef-baf5-efdc52797490 2024-09-21T06:57:11.051 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2a791768-77e2-11ef-baf5-efdc52797490 --force 2024-09-21T06:57:11.257 INFO:teuthology.orchestra.run.smithi203.stdout:Deleting cluster with fsid: 2a791768-77e2-11ef-baf5-efdc52797490 2024-09-21T06:57:11.818 INFO:tasks.cephadm:Removing cephadm ... 2024-09-21T06:57:11.819 DEBUG:teuthology.orchestra.run.smithi067:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T06:57:11.839 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T06:57:11.858 DEBUG:teuthology.orchestra.run.smithi203:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T06:57:11.875 INFO:tasks.cephadm:Teardown complete 2024-09-21T06:57:11.875 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T06:57:11.885 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T06:57:11.885 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T06:57:11.888 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T06:57:11.901 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T06:57:12.015 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T06:57:12.026 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T06:57:12.027 DEBUG:teuthology.orchestra.run.smithi067:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T06:57:12.030 DEBUG:teuthology.orchestra.run.smithi100:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T06:57:12.032 DEBUG:teuthology.orchestra.run.smithi203:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T06:57:12.045 INFO:teuthology.orchestra.run.smithi067.stderr:bash: line 1: ntpq: command not found 2024-09-21T06:57:12.047 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2024-09-21T06:57:12.047 INFO:teuthology.orchestra.run.smithi203.stderr:bash: line 1: ntpq: command not found 2024-09-21T06:57:12.050 INFO:teuthology.orchestra.run.smithi067.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T06:57:12.050 INFO:teuthology.orchestra.run.smithi067.stdout:=============================================================================== 2024-09-21T06:57:12.050 INFO:teuthology.orchestra.run.smithi067.stdout:^* hv01.front.sepia.ceph.com 3 6 377 52 -25us[ -31us] +/- 45ms 2024-09-21T06:57:12.050 INFO:teuthology.orchestra.run.smithi067.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 54 -23us[ -29us] +/- 55ms 2024-09-21T06:57:12.050 INFO:teuthology.orchestra.run.smithi067.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 49 +56us[ +56us] +/- 72ms 2024-09-21T06:57:12.050 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:57:12.052 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T06:57:12.052 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2024-09-21T06:57:12.053 INFO:teuthology.orchestra.run.smithi100.stdout:^* hv01.front.sepia.ceph.com 3 6 377 49 -17us[ +19us] +/- 45ms 2024-09-21T06:57:12.053 INFO:teuthology.orchestra.run.smithi100.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 50 +14us[ +50us] +/- 55ms 2024-09-21T06:57:12.053 INFO:teuthology.orchestra.run.smithi100.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 50 +56us[ +92us] +/- 72ms 2024-09-21T06:57:12.053 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:57:12.053 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T06:57:12.053 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2024-09-21T06:57:12.053 INFO:teuthology.orchestra.run.smithi203.stdout:^* hv01.front.sepia.ceph.com 3 6 377 50 -8530ns[ +15us] +/- 45ms 2024-09-21T06:57:12.053 INFO:teuthology.orchestra.run.smithi203.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 48 -11us[ -11us] +/- 55ms 2024-09-21T06:57:12.053 INFO:teuthology.orchestra.run.smithi203.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 49 +72us[ +72us] +/- 72ms 2024-09-21T06:57:12.053 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:57:12.053 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T06:57:12.063 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T06:57:12.064 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T06:57:12.084 DEBUG:teuthology.orchestra.run.smithi067:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T06:57:12.225 DEBUG:teuthology.orchestra.run.smithi100:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T06:57:12.297 DEBUG:teuthology.orchestra.run.smithi203:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T06:57:12.362 DEBUG:teuthology.orchestra.run.smithi067:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T06:57:12.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:57:12.398 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T06:57:12.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:57:12.429 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T06:57:12.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:57:12.457 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T06:57:12.468 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T06:57:12.478 INFO:teuthology.task.internal:Duration was 2288.826018 seconds 2024-09-21T06:57:12.479 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T06:57:12.488 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T06:57:12.488 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T06:57:12.491 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T06:57:12.494 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T06:57:12.534 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T06:57:12.536 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T06:57:12.556 INFO:teuthology.orchestra.run.smithi067.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T06:57:12.850 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T06:57:12.851 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi067.front.sepia.ceph.com 2024-09-21T06:57:12.851 DEBUG:teuthology.orchestra.run.smithi067:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T06:57:12.873 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi100.front.sepia.ceph.com 2024-09-21T06:57:12.873 DEBUG:teuthology.orchestra.run.smithi100:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T06:57:12.909 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi203.front.sepia.ceph.com 2024-09-21T06:57:12.909 DEBUG:teuthology.orchestra.run.smithi203:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T06:57:12.938 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T06:57:12.938 DEBUG:teuthology.orchestra.run.smithi067:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T06:57:12.941 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T06:57:12.952 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T06:57:13.004 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T06:57:13.005 DEBUG:teuthology.orchestra.run.smithi067:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T06:57:13.011 DEBUG:teuthology.orchestra.run.smithi100:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T06:57:13.021 DEBUG:teuthology.orchestra.run.smithi203:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T06:57:13.734 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T06:57:13.745 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T06:57:13.745 DEBUG:teuthology.orchestra.run.smithi067:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T06:57:13.784 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T06:57:13.811 DEBUG:teuthology.orchestra.run.smithi203:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T06:57:13.839 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T06:57:13.850 DEBUG:teuthology.orchestra.run.smithi067:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:57:13.853 DEBUG:teuthology.orchestra.run.smithi100:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:57:13.855 DEBUG:teuthology.orchestra.run.smithi203:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:57:13.877 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = core 2024-09-21T06:57:13.882 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern = core 2024-09-21T06:57:13.904 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = core 2024-09-21T06:57:13.931 DEBUG:teuthology.orchestra.run.smithi067:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:57:13.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:57:13.952 DEBUG:teuthology.orchestra.run.smithi100:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:57:13.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:57:13.987 DEBUG:teuthology.orchestra.run.smithi203:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:57:14.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:57:14.002 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T06:57:14.018 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T06:57:14.019 DEBUG:teuthology.misc:Transferring archived files from smithi067:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913544/remote/smithi067 2024-09-21T06:57:14.020 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T06:57:14.071 DEBUG:teuthology.misc:Transferring archived files from smithi100:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913544/remote/smithi100 2024-09-21T06:57:14.071 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T06:57:14.107 DEBUG:teuthology.misc:Transferring archived files from smithi203:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913544/remote/smithi203 2024-09-21T06:57:14.107 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T06:57:14.144 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T06:57:14.145 DEBUG:teuthology.orchestra.run.smithi067:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T06:57:14.147 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T06:57:14.150 DEBUG:teuthology.orchestra.run.smithi203:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T06:57:14.195 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T06:57:14.207 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T06:57:14.208 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T06:57:14.217 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T06:57:14.217 DEBUG:teuthology.orchestra.run.smithi067:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T06:57:14.219 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T06:57:14.221 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T06:57:14.232 INFO:teuthology.orchestra.run.smithi067.stdout: 265341 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 06:57 /home/ubuntu/cephtest 2024-09-21T06:57:14.234 INFO:teuthology.orchestra.run.smithi100.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 06:57 /home/ubuntu/cephtest 2024-09-21T06:57:14.251 INFO:teuthology.orchestra.run.smithi203.stdout: 265340 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 06:57 /home/ubuntu/cephtest 2024-09-21T06:57:14.252 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T06:57:14.262 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T06:57:14.285 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/iogen}} duration: 2288.826018333435 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-09-21T06:57:14.285 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T06:57:14.363 INFO:teuthology.run:pass